Nutrient Status Diagnosis of Infield Oilseed Rape via Deep Learning-Enabled Dynamic Model

The symptoms of the nutrient stress in plant canopies differ among different growth stages. It is a challenge to develop individual diagnosis models to evaluate the nutrient status for a specific growth stage in time. Therefore, this article encoded spatiotemporal information of plants in a single time-series model to evaluate the nutrient status of oilseed rape more efficiently. Specifically, in this article, we combined the convolutional neural network (CNN) and long short-term memory (LSTM) to classify the oilseed rape crops according to their nutrient status. The model was validated on a large number of sequential images acquired from oilseed rape canopies at different growth stages during a two-year experiment. Different pretrained CNNs were used to extract distinctive features from every time step of sequential images and then, these features were considered as the input of LSTM to classify the oilseed rape into nine classes of nutrient statuses. We demonstrated that the LSTM outperformed the traditional machine-learning method and the deep features showed better performance compared with hand-crafted features. The Inceptionv3-LSTM obtained the highest overall classification accuracy of 95% when tested on the dataset of 2017/2018 and it also provided a good generalization when using a cross-dataset validation, with the highest overall accuracy of 92%. Our proposed approach presents a pathway toward automatic nutrient status diagnosis during the whole life cycle of the plants, and the LSTM technique would play an essential role in the near future for time-series analysis for precision agriculture.