Many advanced, image-based precision agricultural technologies for plant breeding, field crop research, and site-specific crop management hinge on the reliable detection and phenotyping of plants across highly variable morphological growth stages. Convolutional neural networks (CNNs) have shown promise for image-based plant phenotyping and weed recognition, but their ability to recognize growth stages, often with stark differences in appearance, is uncertain. Amaranthus palmeri (Palmer amaranth) is a particularly challenging weed plant in cotton (Gossypium hirsutum) production, exhibiting highly variable plant morphology both across growth stages over a growing season, as well as between plants at a given growth stage due to high genetic diversity. In this paper, we investigate eight-class growth stage recognition of A. palmeri in cotton as a challenging model for You Only Look Once (YOLO) architectures. We compare 26 different architecture variants from YOLO v3, v5, v6, v6 3.0, v7, and v8 on an eight-class growth stage dataset of A. palmeri. The highest mAP@[0.5:0.95] for recognition of all growth stage classes was 47.34% achieved by v8-X, with inter-class confusion across visually similar growth stages. With all growth stages grouped as a single class, performance increased, with a maximum mean average precision (mAP@[0.5:0.95]) of 67.05% achieved by v7-Original. Single class recall of up to 81.42% was achieved by v5-X, and precision of up to 89.72% was achieved by v8-X. Class activation maps (CAM) were used to understand model attention on the complex dataset. Fewer classes, grouped by visual or size features improved performance over the ground-truth eight-class dataset. Successful growth stage detection highlights the substantial opportunity for improving plant phenotyping and weed recognition technologies with open-source object detection architectures.

Stomatal anatomy and physiology define CO2 availability for photosynthesis and regulate plant water use. Despite being key drivers of yield and dynamic responsiveness to abiotic stresses, conventional measurement techniques of stomatal traits are laborious and slow, limiting adoption in plant breeding. Advances in instrumentation and data analyses present an opportunity to screen stomatal traits at scales relevant to plant breeding. We present a high-throughput field-based phenotyping approach, FieldDino, for screening of stomatal physiology and anatomy. The method allows coupled measurements to be collected in <15 s and consists of: (1) stomatal conductance measurements using a handheld porometer; (2) in situ collection of epidermal images with a digital microscope, 3D-printed leaf clip and Python-based app; and (3) automated deep learning analysis of stomatal features. The YOLOv8-M model trained on images collected in the field achieved strong performance metrics with an mAP@0.5 of 97.1% for stomatal detection. Validation in large field trials of 200 wheat genotypes with two irrigation treatments captured wide diversity in stomatal traits. FieldDino enables stomatal data collection and analysis at unprecedented scales in the field. This will advance research on stomatal biology and accelerate the incorporation of stomatal traits into plant breeding programs for resilience to abiotic stress. ### Competing Interest Statement The authors have declared no competing interest.

Abstract Automating the analysis of plants using image processing would help remove barriers to phenotyping and large-scale precision agricultural technologies, such as site-specific weed control. The combination of accessible hardware and high-performance deep learning (DL) tools for plant analysis is becoming widely recognised as a path forward for both plant science and applied precision agricultural purposes. Yet, a lack of collaboration in image analysis for plant science, despite the open-source origins of much of the technology, is hindering development. Here, we show how tools developed for specific attributes of phenotyping or weed recognition for precision weed control have substantial overlapping data structure, software/hardware requirements and outputs. An open-source approach to these tools facilitates interdisciplinary collaboration, avoiding unnecessary repetition and allowing research groups in both basic and applied sciences to capitalise on advancements and resolve respective bottlenecks. The approach mimics that of machine learning in its nascence. Three areas of collaboration are identified as critical for improving efficiency, (1) standardized, open-source, annotated dataset development with consistent metadata reporting; (2) establishment of accessible and reliable training and testing platforms for DL algorithms; and (3) sharing of all source code used in the research process. The complexity of imaging plants and cost of annotating image datasets means that collaboration from typically distinct fields will be necessary to capitalize on the benefits of DL for both applied and basic science purposes.

Site-specific weed control (SSWC) can provide considerable reductions in weed control costs and herbicide usage. Despite the promise of machine vision for SSWC systems and the importance of ground speed in weed control efficacy, there has been little investigation of the role of ground speed and camera characteristics on weed detection performance. Here, we compare the performance of four camera-software combinations using the open-source OpenWeedLocator platform - (1) default settings on a Raspberry Pi HQ camera, (2) optimised software settings on a HQ camera, (3) optimised software settings on the Raspberry Pi v2 camera, and (4) a global shutter Arducam AR0234 camera - at speeds ranging from 5 km h$^{-1}$ to 30 km h$^{-1}$. A combined excess green (ExG) and hue, saturation, value (HSV) thresholding algorithm was used for testing under fallow conditions using tillage radish (Raphanus sativus) and forage oats (Avena sativa) as representative broadleaf and grass weeds, respectively. ARD demonstrated the highest recall among camera systems, with up to 95.7% of weeds detected at 5 km h$^{-1}$ and 85.7% at 30 km h$^{-1}$. HQ1 and V2 cameras had the lowest recall of 31.1% and 26.0% at 30 km h$^{-1}$, respectively. All cameras experienced a decrease in recall as speed increased. The highest rate of decrease was observed for HQ1 with 1.12% and 0.90% reductions in recall for every km h$^{-1}$ increase in speed for tillage radish and forage oats, respectively. Detection of the grassy forage oats was worse (P<0.05) than the broadleaved tillage radish for all cameras. Despite the variations in recall, HQ1, HQ2, and V2 maintained near-perfect precision at all tested speeds. The variable effect of ground speed and camera system on detection performance of grass and broadleaf weeds, indicates that careful hardware and software considerations must be made when developing SSWC systems.

Weeds are a significant threat to agricultural productivity and the environment. The increasing demand for sustainable weed control practices has driven innovative developments in alternative weed control technologies aimed at reducing the reliance on herbicides. The barrier to adoption of these technologies for selective in-crop use is availability of suitably effective weed recognition. With the great success of deep learning in various vision tasks, many promising image-based weed detection algorithms have been developed. This paper reviews recent developments of deep learning techniques in the field of image-based weed detection. The review begins with an introduction to the fundamentals of deep learning related to weed detection. Next, recent advancements in deep weed detection are reviewed with the discussion of the research materials including public weed datasets. Finally, the challenges of developing practically deployable weed detection methods are summarized, together with the discussions of the opportunities for future research. We hope that this review will provide a timely survey of the field and attract more researchers to address this inter-disciplinary research problem.

As highly adaptable plants, weeds have evolved numerous mechanisms to evade control in agroecosystems. For example, reliance on herbicides has resulted in widespread evolution of resistance in many species. Minimising weed adaptation is a major driver for integrated weed management strategies. Crop mimicry is a notable example of weed adaptation, where weed species evolve to avoid control by mimicking aspects of the crop phenotype. Visual selection by hand weeding has been documented to select for crop mimics that are difficult to distinguish from the crop at the vegetative stage. With recent advancements in weed recognition technologies, image-based weed recognition for in-crop, site-specific weed control is on the cusp of becoming widely adopted. Whilst the control methods used in site-specific weed control will be varied (e.g., spot spraying or lasers), they will share weed recognition technology. Visual selection via image-based deep learning represents a selection pressure for weeds that can evade detection by mimicking crops. This mimicry may reduce weed recognition accuracy and thus weed control efficacy over time and result in difficult to manage mimetic weed phenotypes. Therefore, it is timely to explore the potential for selection of crop mimics by image-based weed recognition algorithms.

The use of a fallow phase is an important tool for maximizing crop yield potential in moisture limited agricultural environments, with a focus on removing weeds to optimize fallow efficiency. Repeated whole field herbicide treatments to control low-density weed populations is expensive and wasteful. Site-specific herbicide applications to low-density fallow weed populations is currently facilitated by proprietary, sensor-based spray booms. The use of image analysis for fallow weed detection is an opportunity to develop a system with potential for in-crop weed recognition. Here we present OpenWeedLocator (OWL), an open-source, low-cost and image-based device for fallow weed detection that improves accessibility to this technology for the weed control community. A comprehensive GitHub repository was developed, promoting community engagement with site-specific weed control methods. Validation of OWL as a low-cost tool was achieved using four, existing colour-based algorithms over seven fallow fields in New South Wales, Australia. The four algorithms were similarly effective in detecting weeds with average precision of 79% and recall of 52%. In individual transects up to 92% precision and 74% recall indicate the performance potential of OWL in fallow fields. OWL represents an opportunity to redefine the approach to weed detection by enabling community-driven technology development in agriculture.

The past 50 yr of advances in weed recognition technologies have poised site-specific weed control (SSWC) on the cusp of requisite performance for large-scale production systems. The technology offers improved management of diverse weed morphology over highly variable background environments. SSWC enables the use of nonselective weed control options, such as lasers and electrical weeding, as feasible in-crop selective alternatives to herbicides by targeting individual weeds. This review looks at the progress made over this half-century of research and its implications for future weed recognition and control efforts; summarizing advances in computer vision techniques and the most recent deep convolutional neural network (CNN) approaches to weed recognition. The first use of CNNs for plant identification in 2015 began an era of rapid improvement in algorithm performance on larger and more diverse datasets. These performance gains and subsequent research have shown that the variability of large-scale cropping systems is best managed by deep learning for in-crop weed recognition. The benefits of deep learning and improved accessibility to open-source software and hardware tools has been evident in the adoption of these tools by weed researchers and the increased popularity of CNN-based weed recognition research. The field of machine learning holds substantial promise for weed control, especially the implementation of truly integrated weed management strategies. Whereas previous approaches sought to reduce environmental variability or manage it with advanced algorithms, research in deep learning architectures suggests that large-scale, multi-modal approaches are the future for weed recognition.

Increasing concern for the ongoing availability and efficacy of herbicides is driving interest in the development of alternative physical and thermal weed control methods. Fortunately, improvements in weed detection through advancements in computing hardware and deep learning algorithms are creating an opportunity to use novel weed control tools, such as lasers, in large-scale cropping systems. For alternative control options, there are two key weed control timing opportunities, early and late post-crop emergence. Weed density for the early timing is typically higher, with a shorter window for control. Conversely, late post-emergent treatment of surviving and late-emerging weeds would occur in lower densities of larger and more variably sized weeds, given a prior weed control effort, but with a longer available weed control period. Research in laser weeding to date has primarily focused on early growth stage weeds and the ability of this approach to control larger weeds remains unknown. This study used a 25 W, 975 nm fiber-coupled diode laser to evaluate the opportunity for control of annual ryegrass (Lolium rigidum Gaudin) and the influence of four different growth stages (three-leaf, seven-leaf, mid-tillering, and late-tillering). Annual ryegrass plants at each growth stage were treated using a laser-focused to a 5 mm diameter with five different irradiation durations developing energy densities of 1.3, 2.5, 6.4, 19.1, and 76.4 J mm−2. At the three-leaf stage, all plants were controlled at 76.4 J mm−2 and 93.3% controlled at 19.1 J mm−2. Complete control of seven-leaf plants was only achieved at 76.4 J mm−2. Although laser treatments did not control mid-tillering stage plants, 76.4 J mm−2 reduced biomass by 60.2%. No similar reductions in biomass were recorded for the largest plants. This initial research assists in the development of novel weed control options in the context of large-scale conservation cropping systems. Future research should investigate the influence of laser treatments on additional weed species and the impact of increased laser power on larger weeds.

The scale of herbicide resistance within a cropping region can be estimated and monitored using surveys of weed populations. The current approach to herbicide resistance surveys is time consuming, logistically challenging, and costly. Here we review past and current approaches used in herbicide resistance surveys with the aims of i) defining effective survey methodologies, ii) highlighting opportunities for improving efficiencies through the use of new technologies and iii) identifying the value of repeated region-wide herbicide resistance surveys. One of the most extensively surveyed area of the world's cropping regions is the Australian grain production region, with in excess of 2900 fields randomly surveyed in each of 3 surveys conducted over the past 15 years. Consequently, recommended methodologies are based on what has been learned from the Australian experience. Traditional seedling-based herbicide screening assays remain the most reliable and widely applicable method for characterizing resistance in weed populations. The use of satellite or aerial imagery to plan collections and image analysis to rapidly quantify screening results could complement traditional resistance assays by increasing survey efficiency and sampling accuracy. Global management of herbicide resistant weeds would benefit from repeated and standardized surveys that track herbicide resistance evolution within and across cropping regions. This article is protected by copyright. All rights reserved.

Weeds are a significant threat to the agricultural productivity and the environment. The increasing demand for sustainable agriculture has driven innovations in accurate weed control technologies aimed at reducing the reliance on herbicides. With the great success of deep learning in various vision tasks, many promising image-based weed detection algorithms have been developed. This paper reviews recent developments of deep learning techniques in the field of image-based weed detection. The review begins with an introduction to the fundamentals of deep learning related to weed detection. Next, recent progresses on deep weed detection are reviewed with the discussion of the research materials including public weed datasets. Finally, the challenges of developing practically deployable weed detection methods are summarized, together with the discussions of the opportunities for future research.We hope that this review will provide a timely survey of the field and attract more researchers to address this inter-disciplinary research problem.

Australian conservation cropping systems are practiced on very large farms (approximately 3,000 ha) where herbicides are relied on for effective and timely weed control. In many fields, though, there are low weed densities (e.g., <1.0 plant 10 m(-2)) and whole-field herbicide treatments are wasteful. For fallow weed control, commercially available weed detection systems provide the opportunity for site-specific herbicide treatments, removing the need for whole-field treatment of fallow fields with low weed densities. Concern about the sustainability of herbicide-reliant weed management systems remain and there has not been interest in the use of weed detection systems for alternative weed control technologies, such as targeted tillage. In this paper, we discuss the use of a targeted tillage technique for site-specific weed control in large-scale crop production systems. Three small-scale prototypes were used for engineering and weed control efficacy testing across a range of species and growth stages. With confidence established in the design approach and a demonstrated 100% weed-control potential, a 6-m wide pre-commercial prototype, the "Weed Chipper," was built incorporating commercially available weed-detection cameras for practical field-scale evaluation. This testing confirmed very high (90%) weed control efficacies and associated low levels (1.8%) of soil disturbance where the weed density was fewer than 1.0 plant 10 m(-2) in a commercial fallow. These data established the suitability of this mechanical approach to weed control for conservation cropping systems. The development of targeted tillage for fallow weed control represents the introduction of site-specific, nonchemical weed control for conservation cropping systems.

Robotic weed control through weed detection has become increasingly important due to mounting pressure on herbicides from resistance and the large impact of weeds on agricultural productivity. One of the major challenges is accurate classification of weed species for selective targeting in crop situations, whilst the existing studies are often conducted in well-controlled settings with consistent lighting, species and backgrounds. Therefore, in this study, we propose a novel graph-based deep learning architecture, namely Graph Weeds Net (GWN), which aims to recognize multiple types of weeds from conventional RGB images collected from complex rangelands. GWN collects regional patterns in line with set image scopes and formulates multi-scale graph representations for weed classification. Additionally, GWN provides suggestions for key regions, creating opportunities for further within-image actions for robotic in-field systems. The architecture was evaluated on a recently published benchmark dataset, achieving the state-of-the-art performance with a top-1 accuracy 98.1%.

AbstractThe widespread use of herbicides in cropping systems has led to the evolution of resistance in major weeds. The resultant loss of herbicide efficacy is compounded by a lack of new herbicide sites of action, driving demand for alternative weed control technologies. While there are many alternative methods for control, identifying the most appropriate method to pursue for commercial development has been hampered by the inability to compare techniques in a fair and equitable manner. Given that all currently available and alternative weed control methods share an intrinsic energy consumption, the aim of this review was to compare methods based on energy consumption. Energy consumption was compared for chemical, mechanical, and thermal weed control technologies when applied as broadcast (whole-field) and site-specific treatments. Tillage systems, such as flex-tine harrow (4.2 to 5.5 MJ ha−1), sweep cultivator (13 to 14 MJ ha−1), and rotary hoe (12 to 17 MJ ha−1) consumed the least energy of broadcast weed control treatments. Thermal-based approaches, including flaming (1,008 to 4,334 MJ ha−1) and infrared (2,000 to 3,887 MJ ha−1), are more appropriate for use in conservation cropping systems; however, their energy requirements are 100- to 1,000-fold greater than those of tillage treatments. The site-specific application of weed control treatments to control 2-leaf-stage broadleaf weeds at a density of 5 plants m−2 reduced energy consumption of herbicidal, thermal, and mechanical treatments by 97%, 99%, and 97%, respectively. Significantly, this site-specific approach resulted in similar energy requirements for current and alternative technologies (e.g., electrocution [15 to 19 MJ ha−1], laser pyrolysis [15 to 249 MJ ha−1], hoeing [17 MJ ha−1], and herbicides [15 MJ ha−1]). Using similar energy sources, a standardized energy comparison provides an opportunity for estimation of weed control costs, suggesting site-specific weed management is critical in the economically realistic implementation of alternative technologies.