Highlights This study addresses two computational challenges in high-throughput phenotyping: scalability and efficiency. Specifically, we focus on extracting crop images and deriving vegetation indices using unmanned aerial systems. To this end, we outline a data processing pipeline, featuring a crop localization algorithm and trie data structure. We demonstrate the efficacy of our approach by computing large-scale and high-precision vegetation indices in a soybean breeding experiment, where we evaluate soybean growth under water inundation and temporal change. Abstract. In agronomy, high-throughput phenotyping (HTP) can provide key information for agronomists in genomic selection as well as farmers in yield prediction. Recently, HTP using unmanned aerial systems (UAS) has shown advantages in both cost and efficiency. However, scalability and efficiency have not been well studied when processing images in complex contexts, such as using multispectral cameras, and when images are collected during early and late growth stages. These challenges hamper further analysis to quantify phenotypic traits for large-scale and high-precision applications in plant breeding. To solve these challenges, our research team previously built a three-step data processing pipeline, which is highly modular. For this project, we present improvements to the previous pipeline to improve canopy segmentation and crop plot localization, leading to improved accuracy in crop image extraction. Furthermore, we propose a novel workflow based on a trie data structure to compute vegetation indices efficiently and with greater flexibility. For each of our proposed changes, we evaluate the advantages by comparison with previous models in the literature or by comparing processing results using both the original and improved pipelines. The improved pipeline is implemented as two MATLAB programs: Crop Image Extraction version 2 (CIE 2.0) and Vegetation Index Derivation version 1 (VID 1.0). Using CIE 2.0 and VID 1.0, we compute canopy coverage and normalized difference vegetation indices (NDVIs) for a soybean phenotyping experiment. We use canopy coverage to investigate excess water stress and NDVIs to evaluate temporal patterns across the soybean growth stages. Both experimental results compare favorably with previous studies, especially for approximation of soybean reproductive stage. Overall, the proposed methodology and implemented experiments provide a scalable and efficient paradigm for applying HTP with UAS to general plant breeding. Keywords: Data processing pipeline, High-throughput phenotyping, Image processing, Soybean breeding, Unmanned aerial systems, Vegetation indices.

Seed characterization and analysis is an important component in plant breeding applications from seed count, shape and weight estimation, to crop yield prediction. Recent research in seed characterization and analysis has adopted high-throughput image-based approaches in which seed detection is the first step. However, traditional seed detection approaches usually require the assistance of specialized hardware such as seed holder and ambient lighting, as well as intensive human support such as seed separation and parameter tuning. To address these challenges, we conducted a baseline study of seed detection using a Convolutional Neural Network based detector (RetinaNet with a backbone network of MobileNet), which is scalable with minimum hardware and human investments. We tested our approach on a benchmark image set, where we collected images of seed samples with different seed sizes and shapes including canola, camelina, and soybean seeds which comprise approximately 5,000 seeds at different density levels. Experimental results of our approach indicated superior detection accuracies for all seed types (COCO mAP@0.5=95.3%) while suggesting improvements on the detection of small seeds such as camelina seeds (COCO mAR@10=37.0%).

Fine-Grained Recognition aims to classify sub-category objects such as bird species and car models from imagery. In High-throughput Phenotyping, the required task is to classify individual plant cultivars to assist plant breeding, which has posed three challenges: 1) it is easy to overfit complex features and models, 2) visual conditions change during and between image collection opportunities, and 3) analysis of thousands of cultivars require high-throughput data collection and analysis. To tackle these challenges, we propose a simple but intuitive descriptor, Radial Object Descriptor, to represent plant cultivar objects based on contour. This descriptor is invariant under scaling, rotation, and translation, as well as robust under changes to the plant's growth stage and camera's view angle. Furthermore, we complement this mid-level feature by fusing it with the low-level features (Histogram of Oriented Gradients) and deep features (ResNet-18), respectively. We extensively test our fusion approaches using two real world experiments. One experiment is on a novel benchmark dataset (HTP-Soy) in which we collect ~ 2, 000 high-resolution aerial images of outdoor soybean plots. Another experiment is on three datasets of indoor rosette plants. For both experiments, our fusion approaches achieve superior accuracies while maintaining better generalization as compared with traditional approaches.

In agronomy, High-throughput Phenotyping (HTP) can provide key information for agronomist in genomic selection as well as farmers in yield prediction. Recently, HTP using Unmanned Aerial Systems (UAS) has shown advantages in both cost and efficiency. However, scalability and efficiency have not been well studied when processing images in complex contexts such as multispectral bands and early/late growth stages. The listed challenges hamper further analysis to quantify phenotypic traits for large-scale and high-precision applications in plant breeding. To solve these challenges, the research team previously built a 3-step data processing pipeline, which is highly modular. For this project we present improvements to the previous methods used for canopy segmentation and crop plot localization for crop image extraction. Furthermore, we propose a novel workflow based on a trie data structure to compute vegetation indices efficiently and with greater flexibility. For each of our proposed changes, we evaluate advantages by comparing with processing results using the original model. Based on these proposed methods, we implement two MATLAB programs, namely Multi-Layer Mosaic version 2 (MLM2) and Vegetation Indices Derivation version 1 (VID1). Using MLM2 and VID1, we compute canopy coverage and Normalized Difference Vegetation Indices (NDVIs) for a soybean phenotyping experiment. We use canopy coverage to investigate water depression and NDVIs to evaluate temporal patterns across the soybean growth stages. Both experimental results compare favorably with previous studies, especially on soybean reproductive stage approximation. Overall, the proposed methodology and implemented experiments provide a scalable and efficient paradigm for applying HTP in UAS to general plant breeding.