This study presents an annotated multi-sensor, multimodal, and hyperspectral dataset designed to support deep learning-based classification and segmentation of bulky waste. The dataset comprises four distinct sensor modalities: high-resolution visible RGB images (VIS), hyperspectral near-infrared (NIR), temporally resolved thermal infrared (IR), and terahertz (THz) imaging with depth information, providing complementary multimodal information. An image registration process aligns all modalities to a common reference frame, enabling near pixel-precise fusion across sensors. WoodVIT contains 56 registered multi-sensor scenes, partitioned into 22,659 annotated patches with two main classes (wood and non-wood) and 16 subclass labels. It includes pixel-masks and patch-wise annotations to facilitate both segmentation and classification tasks. The primary benchmark task is binary discrimination of wood versus non-wood. The dataset also includes challenging scenarios involving occlusion and concealed contaminants (e.g., embedded metals) to motivate robust multimodal fusion approaches. We provide predefined train/validation/test splits and report baseline results using convolutional neural networks and fusion architectures to establish reference performance. WoodVIT is publicly available to support research on multi-sensor learning for waste sorting.

PurposeThe study aims to develop a method for detecting and discriminating grapevine yellows (GY) diseases, including Flavescence dorée (FD), Bois noir (BN), and Palatinate grapevine yellows (PGY), using hyperspectral imaging (HSI). Specifically, it seeks to address the challenge of symptom misclassification among visually similar diseases, such as distinguishing between GY diseases and other biotic and abiotic stresses.MethodsHyperspectral images of detached leaves from field with GY diseases, Grapevine Leafroll-associated Virus (V), leafhopper (LH) infestation, iron deficiency (Fe), or magnesium deficiency (Mg) were acquired using a mobile platform in the laboratory or in the field. The images were taken in the spectral range of 400-1,000 nm. Classification models were trained on the mean spectra of the leaves to distinguish between different classes.ResultsThe model achieved F1-scores ranging from 54.2% to 96.8% for white cultivars and exceeded 95% for all six classes for black cultivars. When limited to GY classes and nonsymptomatic leaves, the F1-scores were 89.8%, 78.7%, 63.4%, and 53.4% for nonsymptomatic, FD, PGY, and BN, respectively.ConclusionThe study demonstrates the feasibilityof using HSI to detect and discriminate different biotic and abiotic stresses on grapevine leaves, including distinguishing between symptom-similar GY diseases. The developed mobile platform and classification models show promise for largescale monitoring and diagnosis of GY diseases, potentially improving disease management and reducing the risk of symptom misclassification.

Vineyard managers traditionally count grape clusters manually for yield estimation, a process that is both time-consuming and labor-intensive. Recent advances in computer vision enable autonomous tracking, yet state-of-the-art methods often rely on re-identification networks that require expensive, hard-to-obtain instance ID annotations. This study addresses this challenge by evaluating the real-time tracking performance and counting accuracy of SORT, DeepSORT, ByteTrack, and the newly proposed SORT+ and DeepSORT+ algorithms. SORT+ and DeepSORT+ incorporate a novel matching cascade that leverages the complementary strengths of Mahalanobis, IoU, and Euclidean distances. Crucially, this approach allows SORT+ to achieve robust performance without the need for additional training data. The extended matching cascade offers large improvements for SORT, making the training-free SORT+ comparable to the deep-learning-based DeepSORT. It increases MOTA and IDF1 by 5% to 6%, while decreasing ID switches by 62%. SORT+ improves the counting accuracy from 33% to 96%. DeepSORT+ shows further performance gains, decreasing ID switches by 12% compared to DeepSORT. This work illustrates the feasibility of using unmanned aerial vehicles (UAVs) to autonomously track and count grape clusters in challenging real-world vineyard settings. By potentially improving yield estimation and non-destructive robotic farming, these findings support sustainable farming practices and economic growth.

Precise georeferencing of grapevines is essential for plant-level monitoring in precision viticulture. We address the task of assigning side-view images, captured from a vehicle with GNSS logging, to individual vines previously mapped with high-precision GNSS. The challenge is to select, for each vine, the frame where it is most centrally and fully visible. We present a method for automated image assignment, combining a vision-based detection and tracking method with optional GNSS-based correction and validation. Selected rows were manually annotated as ground truth references for evaluation. Results show that our method reliably assigns the correct frames in most cases, highlighting its potential for scalable precision agriculture. An advantage of this approach is that it does not require manual offset correction to account for varying growing directions. Die pr & auml;zise Georeferenzierung von Rebst & ouml;cken ist entscheidend f & uuml;r das pflanzenindividuelle Monitoring im Pr & auml;zisionsweinbau. Diese Arbeit adressiert die Zuordnung seitlich aufgenommener, GNSS-verorteter Bilder zu zuvor hochgenau vermessenen Rebst & ouml;cken. Ziel ist, f & uuml;r jeden Rebstock das Bild zu identifizieren, auf dem er zentral und vollst & auml;ndig sichtbar ist. Vorgestellt wird eine Methode zur automatischen Bildzuordnung, die visuelle Objekt-Detektion und Tracking mit GNSS-basierter Korrektur kombiniert. Zur Evaluation wurden ausgew & auml;hlte Rebzeilen manuell als Referenzdaten annotiert. Die Ergebnisse zeigen eine zuverl & auml;ssige Zuordnung und verdeutlichen das Potenzial f & uuml;r skalierbare Anwendungen im Pr & auml;zisionsweinbau. Ein wesentlicher Vorteil der Methode ist, dass keine manuelle Offset-Korrektur zur Ber & uuml;cksichtigung variierender Wuchsrichtungen erforderlich ist.

Sensor-based sorting describes a family of systems that enable the removal of individual objects from a material stream. The technology is widely used in various industries such as agriculture, food, mining, and recycling. Examples of sorting tasks include the removal of fungus-infested grains, the enrichment of copper content in copper mining or the sorting of plastic waste according to the type of plastic. Sorting decisions are made based on information acquired by one or more sensors. A particular strength of the technology is the flexibility in sorting decisions, which is achieved by using various sensors and programming the data analysis. However, a comprehensive understanding of the process is necessary for the development of new sorting systems that can address previously unresolved tasks. This survey is aimed at innovative researchers and practitioners who are unfamiliar with sensor-based sorting or have only encountered certain aspects of the overall process. The references provided serve as starting points for further exploration of specific topics.

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

It is crucial for winegrowers to make informed decisions about the optimum time to harvest the grapes to ensure the production of premium wines. Global warming contributes to decreasing acidity and increasing sugar levels in grapes, resulting in bland wines with high contents of alcohol. Predicting quality in viticulture is thus pivotal. To assess the average ripeness, typically a sample of one hundred berries representative for the entire vineyard is collected. However, this process, along with the subsequent detailed must analysis, is time consuming and expensive. This study focusses on predicting essential quality parameters like sugar and acid content in Vitis vinifera (L.) varieties ‘Chardonnay’, ‘Riesling’, ‘Dornfelder’, and ‘Pinot Noir’. A small near-infrared spectrometer was used measuring non-destructively in the wavelength range from 1 100 nm to 1 350 nm while the reference contents were measured using high-performance liquid chromatography. Chemometric models were developed employing partial least squares regression and using spectra of all four grapevine varieties, spectra gained from berries of the same colour, or from the individual varieties. The models exhibited high accuracy in predicting main quality-determining parameters in independent test sets. On average, the model regression coefficients exceeded 93% for the sugars fructose and glucose, 86% for malic acid, and 73% for tartaric acid. Using these models, prediction accuracies revealed the ability to forecast individual sugar contents within an range of ± 6.97 g/L to ± 10.08 g/L, and malic acid within ± 2.01 g/L to ± 3.69 g/L. This approach indicates the potential to develop robust models by incorporating spectra from diverse grape varieties and berries of different colours. Such insight is crucial for the potential widespread adoption of a handheld near-infrared sensor, possibly integrated into devices used in everyday life, like smartphones. A server-side and cloud-based solution for pre-processing and modelling could thus avoid pitfalls of using near-infrared sensors on unknown varieties and in diverse wine-producing regions.

The optical bulk material sorting is a key technology on our way toward a circular economy and efficient recycling. However, controlling the sorting accuracy has so far been severely limited, as the achievable accuracy of conventional sorters is strongly determined by the mass flow and the mixing ratio of the incoming particle stream. To enable closed-loop control, in the previous work, we introduced a modification to the sorter design, in which controlled fractions of the already sorted mass flows are returned to the inlet of the sorter. In this article, we now propose two open-loop and two closed-loop feedback (CLF) stochastic model predictive controllers (MPCs) for the control of sorting systems with recirculation operating under dynamically changing conditions. In addition, we propose to integrate a desired minimum accuracy as a chance constraint into our controllers' stochastic formulation. Our evaluations using a coupled discrete element-computational fluid dynamics (DEM-CFD) simulation show that our controllers considerably improve on the system without recirculation and outperform the previously known controllers. Furthermore, we found that they are able to maintain a predefined minimum quality even in highly dynamic scenarios, making the approach highly valuable for tasks where achieving a certain quality at any point in time is crucial.

Conventional agriculture relies heavily on herbicides for weed control. Smart farming, particularly through the use of mechanical weed control systems, has the potential to reduce the herbicide usage and the associated negative impact on our environment. The growing accessibility of multispectral cameras in recent times poses the question if their added expenses justify the potential advantages they offer. In this study we compare the weed and crop detection performance between RGB and multispectral VIS-NIR imaging data. Therefore, we created and annotated a multispectral instance segmentation dataset for sugar beet crop and weed detection. We trained Mask-RCNN models on the RGB images and on images composed of different vegetation indices calculated from the multispectral data. The outcomes are thoroughly analysed and compared across various scenarios. Our findings indicate that the use of vegetation indices can significantly improve the weed detection performance in many situations.

Yield prediction in viticulture is an especially challenging research direction within the field of yield prediction. The characteristics that determine annual grapevine yields are plentiful, difficult to obtain, and must be captured multiple times throughout the year. The processes currently used in grapevine yield prediction are based mainly on manually captured data and rigid statistical measures derived from historical insights. Experts for data acquisition are scarce, and statistical models cannot meet the requirements of a changing encontributes a concept on how to overcome those drawbacks, by (1) proposing a deep learning driven approach for feature the future of yield prediction in viticulture.

This data set comprises images of bulk material of brick on a conveyor belt. The images were recorded on the small-scale optical belt sorter Tablesort. A thorough description of the system can be found in Georg Maier, Florian Pfaff, Christoph Pieper, Robin Gruna, Benjamin Noack, Harald Kruggel-Emden,Thomas Längle, Uwe D. Hanebeck, Jürgen Beyerer, Experimental Evaluation of a Novel Sensor-Based Sorting Approach Featuring Predictive Real-Time Multiobject Tracking,Transactions on Industrial Electronics, February 2020. See also the project website. This dataset is part of a batch of recordings of construction and demolition waste consisting of brick and sand-lime brick on optical sorters. Please use the search function with the keywords "Images of Construction and Demolition Waste" (in quotes) to find them. The data set was recorded with a Bonito CL-400C. Please see the debayer script on GitHub https://github.com/geomai/debayer_bonito. The frame rate is approximately 192 Hz.To this date, publications that used this data include Marcel Reith-Braun, Albert Bauer, Maximilian Staab, Florian Pfaff, Georg Maier, Robin Gruna, Thomas Längle, Jürgen Beyerer, Harald Kruggel-Emden and Uwe D. Hanebeck, GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs, 22nd IFAC World Congress, Yokohama, Japan, July 2023. AcknowledgmentThe IGF project 20354 N of the research association Forschungs-Gesellschaft Verfahrens-Technik e.V. (GVT) was supported via the AiF in a program to promote the Industrial Community Research and Development (IGF) by the Federal Ministry for Economic Affairs and Climate Action on the basis of a resolution of the German Bundestag.

This data set comprises images of bulk material of brick and sand-lime brick on a conveyor belt. The images were recorded on the small-scale optical belt sorter Tablesort. A thorough description of the system can be found in Georg Maier, Florian Pfaff, Christoph Pieper, Robin Gruna, Benjamin Noack, Harald Kruggel-Emden,Thomas Längle, Uwe D. Hanebeck, Jürgen Beyerer, Experimental Evaluation of a Novel Sensor-Based Sorting Approach Featuring Predictive Real-Time Multiobject Tracking,Transactions on Industrial Electronics, February 2020. See also the project website. This dataset is part of a batch of recordings of construction and demolition waste consisting of brick and sand-lime brick on optical sorters. Please use the search function with the keywords "Images of Construction and Demolition Waste" (in quotes) to find them. The data set was recorded with a Bonito CL-400C. Please see the debayer script on GitHub https://github.com/geomai/debayer_bonito. The frame rate is approximately 192 Hz.To this date, publications that used this data include Marcel Reith-Braun, Albert Bauer, Maximilian Staab, Florian Pfaff, Georg Maier, Robin Gruna, Thomas Längle, Jürgen Beyerer, Harald Kruggel-Emden and Uwe D. Hanebeck, GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs, 22nd IFAC World Congress, Yokohama, Japan, July 2023. AcknowledgmentThe IGF project 20354 N of the research association Forschungs-Gesellschaft Verfahrens-Technik e.V. (GVT) was supported via the AiF in a program to promote the Industrial Community Research and Development (IGF) by the Federal Ministry for Economic Affairs and Climate Action on the basis of a resolution of the German Bundestag.

Optical sorters separate particles of different classes by first detecting them while they are transported, e.g., on a conveyor belt, and subsequently bursting out particles of undesired classes using compressed air nozzles. Currently, the most promising results are achieved by predictive tracking, a multitarget tracking approach based on extracted midpoints from area-scan camera images that analyzes the particles’ motion and activates the nozzles accordingly. However, predictive tracking requires expert knowledge for setup and preceding object detection. Moreover, particle shapes are only considered implicitly, and the need to solve an association problem rises the computational complexity of the algorithm. In this paper, we present GridSort, an image-based approach that forecasts the scene at the nozzle array using a convolutional long short-term memory neural network and subsequently extracts nozzle activations, thus circumventing the aforementioned weaknesses. We show how GridSort can be trained in an unsupervised fashion and evaluate it using a coupled discrete element–computational fluid dynamics simulation of an optical sorter. We compare our method with predictive tracking in terms of sorting accuracy and demonstrate that it is an easy-to-apply alternative while achieving state-of-the-art results.

This data set comprises images of bulk material of brick and sand-lime brick on a conveyor belt. The images were recorded on the small-scale optical belt sorter Tablesort. A thorough description of the system can be found in Georg Maier, Florian Pfaff, Christoph Pieper, Robin Gruna, Benjamin Noack, Harald Kruggel-Emden,Thomas Längle, Uwe D. Hanebeck, Jürgen Beyerer, Experimental Evaluation of a Novel Sensor-Based Sorting Approach Featuring Predictive Real-Time Multiobject Tracking,Transactions on Industrial Electronics, February 2020. See also the project website. This dataset is part of a batch of recordings of construction and demolition waste consisting of brick and sand-lime brick on optical sorters. Please use the search function with the keywords "Images of Construction and Demolition Waste" (in quotes) to find them. The data set was recorded with a Bonito CL-400C. Please see the debayer script on GitHub https://github.com/geomai/debayer_bonito. The frame rate is approximately 192 Hz.To this date, publications that used this data include Marcel Reith-Braun, Albert Bauer, Maximilian Staab, Florian Pfaff, Georg Maier, Robin Gruna, Thomas Längle, Jürgen Beyerer, Harald Kruggel-Emden and Uwe D. Hanebeck, GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs, 22nd IFAC World Congress, Yokohama, Japan, July 2023. AcknowledgmentThe IGF project 20354 N of the research association Forschungs-Gesellschaft Verfahrens-Technik e.V. (GVT) was supported via the AiF in a program to promote the Industrial Community Research and Development (IGF) by the Federal Ministry for Economic Affairs and Climate Action on the basis of a resolution of the German Bundestag.

Deep learning techniques are commonly utilized to tackle various computer vision problems, including recognition, segmentation, and classification from RGB images. With the availability of a diverse range of sensors, industry-specific datasets are acquired to address specific challenges. These collected datasets have varied modalities, indicating that the images possess distinct channel numbers and pixel values that have different interpretations. Implementing deep learning methods to attain optimal outcomes on such multimodal data is a complicated procedure. To enhance the performance of classification tasks in this scenario, one feasible approach is to employ a data fusion technique. Data fusion aims to use all the available information from all sensors and integrate them to obtain an optimal outcome. This paper investigates early fusion, intermediate fusion, and late fusion in deep learning models for bulky waste image classification. For training and evaluation of the models, a multimodal dataset is used. The dataset consists of RGB, hyperspectral Near Infrared (NIR), Thermography, and Terahertz images of bulky waste. The results of this work show that multimodal sensor fusion can enhance classification accuracy compared to a single-sensor approach for the used dataset. Hereby, late fusion performed the best with an accuracy of 0.921 compared to intermediate and early fusion, on our test data.

This data set comprises images of bulk material of brick and sand-lime brick on a conveyor belt. The images were recorded on the small-scale optical belt sorter Tablesort. A thorough description of the system can be found in Georg Maier, Florian Pfaff, Christoph Pieper, Robin Gruna, Benjamin Noack, Harald Kruggel-Emden,Thomas Längle, Uwe D. Hanebeck, Jürgen Beyerer, Experimental Evaluation of a Novel Sensor-Based Sorting Approach Featuring Predictive Real-Time Multiobject Tracking,Transactions on Industrial Electronics, February 2020. See also the project website. This dataset is part of a batch of recordings of construction and demolition waste consisting of brick and sand-lime brick on optical sorters. Please use the search function with the keywords "Images of Construction and Demolition Waste" (in quotes) to find them. The data set was recorded with a Bonito CL-400C. Please see the debayer script on GitHub https://github.com/geomai/debayer_bonito. The frame rate is approximately 192 Hz.To this date, publications that used this data include Marcel Reith-Braun, Albert Bauer, Maximilian Staab, Florian Pfaff, Georg Maier, Robin Gruna, Thomas Längle, Jürgen Beyerer, Harald Kruggel-Emden and Uwe D. Hanebeck, GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs, 22nd IFAC World Congress, Yokohama, Japan, July 2023. AcknowledgmentThe IGF project 20354 N of the research association Forschungs-Gesellschaft Verfahrens-Technik e.V. (GVT) was supported via the AiF in a program to promote the Industrial Community Research and Development (IGF) by the Federal Ministry for Economic Affairs and Climate Action on the basis of a resolution of the German Bundestag.

This data set comprises images of bulk material of brick and sand-lime brick on a conveyor belt. The images were recorded on the small-scale optical belt sorter Tablesort. A thorough description of the system can be found in Georg Maier, Florian Pfaff, Christoph Pieper, Robin Gruna, Benjamin Noack, Harald Kruggel-Emden,Thomas Längle, Uwe D. Hanebeck, Jürgen Beyerer, Experimental Evaluation of a Novel Sensor-Based Sorting Approach Featuring Predictive Real-Time Multiobject Tracking,Transactions on Industrial Electronics, February 2020. See also the project website. This dataset is part of a batch of recordings of construction and demolition waste consisting of brick and sand-lime brick on optical sorters. Please use the search function with the keywords "Images of Construction and Demolition Waste" (in quotes) to find them. The data set was recorded with a Bonito CL-400C. Please see the debayer script on GitHub https://github.com/geomai/debayer_bonito. The frame rate is approximately 192 Hz.To this date, publications that used this data include Marcel Reith-Braun, Albert Bauer, Maximilian Staab, Florian Pfaff, Georg Maier, Robin Gruna, Thomas Längle, Jürgen Beyerer, Harald Kruggel-Emden and Uwe D. Hanebeck, GridSort: Image-based Optical Bulk Material Sorting Using Convolutional LSTMs, 22nd IFAC World Congress, Yokohama, Japan, July 2023. AcknowledgmentThe IGF project 20354 N of the research association Forschungs-Gesellschaft Verfahrens-Technik e.V. (GVT) was supported via the AiF in a program to promote the Industrial Community Research and Development (IGF) by the Federal Ministry for Economic Affairs and Climate Action on the basis of a resolution of the German Bundestag.