Grapevine winter pruning is a labor-intensive and repetitive process that significantly influences grape yield and quality at harvest and produced wine. Due to its complexity and repetitive nature, the task demands skilled labor that needs to be trained, as in many other agricultural sectors. This paper encompasses an approach that targets using a robotic system to perform autonomous grapevine winter pruning using a vision system and artificial intelligence. In our previous work, we presented a 2D neural network that segmented images of grapevines into 5 different classes of plant organs during their dormant season. In this paper, we expand into the third dimension, introducing point clouds into our algorithm. The 3D approach creates instance-segmented point clouds using depth images and segmentation masks obtained with our 2D neural network. After the 3D reconstruction, the system extracts thickness measurement and uses agronomic knowledge to place pruning points for balanced pruning. The study not only delineates the integration of 2D and 3D methods but also scrutinizes their efficacy in pruning point identification. The real-world performance of the created system was evaluated and statistically analyzed on data collected during field trials in the winter pruning season 2022/2023, where the system was used in a potted vineyard to prune a set of test vines, where the positive success rate is 54.2%. Moreover, as one of the main contributions, the paper underscores a unique facet of adaptability, presenting a customizable framework that empowers end-users to fine-tune parameters according to the expected balanced pruning. This adaptability extends to variables such as the number of nodes to retain on pruned spurs and the preferred cane thickness, encapsulating the versatility of the 3D approach.

Satellite imagery is essential for Earth observation, offering land use and land cover monitor capabilities. In agriculture, Sentinel-2 multispectral data enable analyses of crop health, irrigation, and yield forecasting. However, mapping crops such as vineyards and olive orchards is challenging due to phenological and spectral variability, alongside Sentinel-2 spatial and temporal limitations. This study evaluates the segmentation capabilities of a deep learning-based method for these crops using Sentinel-2 data. To address dataset limitations, advanced techniques of pre-processing, data augmentation, and class balancing were applied. The model integrates s Swin Transformer-Based encoder with a modified U-Net decoder, using dilated convolutions and attention mechanisms. Raster composites using three band combinations (B432, B843, and B864) were evaluated using data from northern Portugal with B864 showing the best performance with an IoU of 0.79 and F1-score of 0.88 for vineyards. This approach offers a solution that contributes to efficient crop and land-use monitoring.

Grapevine winter pruning is a labor-intensive and repetitive process that significantly influences the quality and quantity of the grape harvest and produced wine of the following season. It requires a careful and expert detection of the point to be cut. Because of its complexity, repetitive nature and time constraint, the task requires skilled labor that needs to be trained. This extended abstract presents the computer vision pipeline employed in project Vinum, using detectron2 as a segmentation network and keypoint visual odometry to merge different observation into a single pointcloud used to make informed pruning decisions.

The VINUM project seeks to address the shortage of skilled labor in modern vineyards by introducing a cutting-edge mobile robotic solution. Leveraging the capabilities of the quadruped robot, HyQReal, this system, equipped with arm and vision sensors, offers autonomous navigation and winter pruning of grapevines reducing the need for human intervention. At the heart of this approach lies an architecture that empowers the robot to easily navigate vineyards, identify grapevines with unparalleled accuracy, and approach them for pruning with precision. A state machine drives the process, deftly switching between various stages to ensure seamless and efficient task completion. The system’s performance was assessed through experimentation, focusing on waypoint precision and optimizing the robot’s workspace for single-plant operations. Results indicate that the architecture is highly reliable, with a mean error of 21.5cm and a standard deviation of 17.6cm for HyQReal. However, improvements in grapevine detection accuracy are necessary for optimal performance. This work is based on a computer-vision-based navigation method for quadruped robots in vineyards, opening up new possibilities for selective task automation. The system’s architecture works well in ideal weather conditions, generating and arriving at precise waypoints that maximize the attached robotic arm’s workspace. This work is an extension of our short paper presented at the Italian Conference on Robotics and Intelligent Machines (I-RIM), 2022 [1].

A blockchain, during its lifetime, records large amounts of data. In a robotics environment, the old information is useful for human evaluation, or to perform analysis, but it is not useful for robots that require only current information to continue their work. This causes a storage problem in Blockchain nodes as in the case of nodes attached to robots that are usually built around embedded solutions. This paper presents a time-segmentation solution for devices with limited storage capacity, integrated into a particular robot-directed Blockchain called RobotChain. The experiments conducted show that the goal of restricting each node's capacity is reached without compromising all the benefits that arise from the use of Blockchains in these contexts and it allows for cheap nodes to use this Blockchain, reducing storage costs and allowing faster deployment of new nodes.

Mobile manipulators that combine manipulability and mobility, are increasingly being used for various unstructured application scenarios in the field, e.g. vineyards. Therefore, coordinated motion of the manipulator and mobile base is an essential feature of the overall performance. In this paper, we explore a whole-body coordinated motion controller of a robot which is composed of a 2-DoFs non-holonomic wheeled mobile base with a 7-DoFs manipulator (non-holonomic wheeled mobile manipulator, NWMM). This robotic platform is designed to efficiently undertake complex grapevine pruning automation. In this control framework, a task priority coordinated motion of the NWMM is guaranteed. Tasks with lower priority are projected into the null space of the top-priority tasks so that higher-priority tasks are completed without interruption from lower-priority tasks. The proposed controller was evaluated in a grapevine spur pruning experiment scenario.

Grapevine winter pruning is a complex task, that requires skilled workers to execute it correctly. The complexity of this task is also the reason why it is time consuming. Considering that this operation takes about 80–120 hours/ha to be completed, and therefore is even more crucial in large-size vineyards, an automated system can help to speed up the process. To this end, this paper presents a novel multidisciplinary approach that tackles this challenging task by performing object segmentation on grapevine images, used to create a representative model of the grapevine plants. Second, a set of potential pruning points is generated from this plant representation. We will describe (a) a methodology for data acquisition and annotation, (b) a neural network fine-tuning for grapevine segmentation, (c) an image processing based method for creating the representative model of grapevines, starting from the inferred segmentation and (d) potential pruning points detection and localization, based on the plant model which is a simplification of the grapevine structure. With this approach, we are able to identify a significant set of potential pruning points on the canes, that can be used, with further selection, to derive the final set of the real pruning points.

This is a dataset for Grapevine segmentation for the purposes of winter pruning, created in a joint work between Istituto Italiano di Tecnologia and Università Cattolica del Sacro Cuore, as part of the Vinum project. The images were captured in the simulated grapevine garden in Università Cattolica, located in Piacenza, Italy, and it it is split in two parts, following agronomic trials being run on the university. The first part, is a group of seven specimen that is the Control group, and the second group of eight specimens is being performed shoot thinning. Each plant has around 5 spurs, and the photos are taken from both sides of the grapevine specimen, meaning that there is one picture where the plant grows from left to right, and the other side where the plant grows from right to left. The resolution of the images is 4608x3456, and we have a total of 149 images. It is annotated using the COCO Segmentation format, and some of the statistics of the dataset such as the number of annotations and number of images can be seen next. Control Complex Total Images: 69 Annotated Images: 69 Annotations: 1838 Categories: 5 Annotations Per Category Main Cordon: 79 Cane: 440 Node: 1100 Arm: 103 Spur: 116 Annotated Images Per Category Main Cordon: 69 Cane: 69 Node: 69 Arm: 62 Spur: 68 Shoot Thining Simple Total Images: 79 Annotated Images: 79 Annotations: 1635 Categories: 5 Annotations Per Category Main Cordon: 84 Cane: 341 Node: 912 Arm: 154 Spur: 144 Annotated Images Per Category Main Cordon: 79 Cane: 79 Node: 79 Arm: 79 Spur: 77

Grapevine winter pruning is a complex task, that requires skilled workers to execute it correctly. The complexity makes it time consuming. It is an operation that requires about 80-120 hours per hectare annually, making an automated robotic system that helps in speeding up the process a crucial tool in large-size vineyards. We will describe (a) a novel expert annotated dataset for grapevine segmentation, (b) a state of the art neural network implementation and (c) generation of pruning points following agronomic rules, leveraging the simplified structure of the plant. With this approach, we are able to generate a set of pruning points on the canes, paving the way towards a correct automation of grapevine winter pruning.

In recent years Sim2Real approaches have brought great results to robotics. Techniques such as model-based learning or domain randomization can help overcome the gap between simulation and reality, but in some situations simulation accuracy is still needed. An example is agricultural robotics, which needs detailed simulations, both in terms of dynamics and visuals. However, simulation software is still not capable of such quality and accuracy. Current Sim2Real techniques are helpful in mitigating the problem, but for these specific tasks they are not enough.

The final version of the paper “Robotchain: Using Tezos Technology for Robot Event Management” can be found in Ledger Vol. 4, S1 (2019) 32-41, DOI 10.5915/LEDGER.2019.175. There were two reviewers involved in the review process, none of whom have requested to waive their anonymity at present, and are thus listed as Reviewers A and B. After initial review (1A), the editors requested that the authors respond to the reviewer concerns and make revisions (1B), which were carried out by the authors, completing the peer-review process.

Robots are important equipment in the modern day factory environment. To maintain and improve factory productivity, ledgers containing robotic actions may be used to identify possible bottleneck points in an assembly line or to serve as a record of in unintentional behaviours, be it of a malicious nature or not. In this paper we present Robotchain, a possible solution using blockchain technology, that prevents unwanted changes in a robotic action ledger and provides a way to use the said ledger in order to aid in production efficiency or other management requirements. This paper also presents an initial experimental study of the Tezos blockchain in order to understand the challenges related to using its advanced blockchain technology for the Robotchain implementation.

Object grasping is a task that humans do without major concerns. This results from self learning and by observing of other skilled humans doing such task with previous information. However, grasping novel objects in unknown positions for a robot is a complex task which encounters many problems, such as sub-optimal performance rates and the time consumption. In this paper we present a method that complements the state-of-the-art grasping algorithms with two segmentation steps, the first one which removes the largest planar surface in the point cloud of the world before the grasp detector receives them and the second one that complements this segmentation with another segmentation that calculates where the object is located and segments the point cloud by executing a crop around the object. The proposed method significantly improves the grasping success rate (100% improvement over the baseline approach) and simultaneously is able to reduce the time consumption by 23%.

There are multiple approaches for SLAM, but we found the the ones implemented in ROS had problems when a robot drove over small obstacles. This paper presents a proposal to make a more robust SLAM by running three SLAM methods in parallel and using their information to produce a better estimate of the robot's surroundings. The proposed method defines its output by making the three methods vote for the value of each pixel in the map. To deal with the increased computational complexity, the method is implemented in the GPU. The performed experiments show that our method shows smaller error than any of the three fused methods alone both when there are ground obstacles that induce map errors and also when no obstacles are present, thus presenting in fact an increase in robustness.

Navigation is a well established field with robust algorithms that can work out-of-the-box in systems like ROS. Nonetheless, there are situations in which the current navigation approaches are laking in terms of optimality. Examples arise when too much "safe space" is assigned around a given object that can completely prevent a robot from using a given path and forces the use of an alternative path that can be much longer. In this paper we propose the dynamic adaptation of robot navigation strategies depending on the type of obstacles that are met during navigation. We do this in real time using a convolutional neural network for obstacle recognition and a path planning parameter adjustment depending on the obstacle category. We present experiments illustrating the difference in paths that can be obtained by using the proposed approach versus standard approaches implemented in ROS.

Precision viticulture (PV) and precision agriculture (PA) requires the acquisition and processing of a vast collection of data coming typically from large scale and heterogeneous sensor networks. Unfortunately, sensor integration is far from being simple due to the number of incompatible network specifications and platforms. The adoption of a common, standard communication interface would allow the engineer to abstract the relation between the sensor and the network. This would reduce the development efforts and emerge as an important step towards the adoption of ''plug-and-play'' technology in PA/PV sensor networks. This paper explores this need and introduces a framework for smart data acquisition in PA/PV that relies on the IEEE 1451 family of standards, which addresses the transducer-to-network interoperability issues. The framework includes a ZigBee end device (sMPWiNodeZ), as an IEEE 1451 WTIM (Wireless Transducer Interface Module), and an IEEE 1451 NCAP (Network Capable Application Processor) that acts as gateway to an information service provider and WSN (Wireless Sensor Network) coordinator. The paper discusses the proposed IEEE 1451 system architecture and its benefits in PA/PV and closes with results/lessons learned from in-field trials towards smarter WSN.

Wireless sensor networks have found multiple applications in precision viticulture. Despite the steady progress in sensing devices and wireless technologies, some of the crucial items needed to improve the usability and scalability of the networks, such as gateway infrastructures and in-field processing, have been comparatively neglected. This paper describes the hardware, communication capabilities and software architecture of an intelligent autonomous gateway, designed to provide the necessary middleware between locally deployed sensor networks and a remote location within the whole-farm concept. This solar-powered infrastructure, denoted by iPAGAT (Intelligent Precision Agriculture Gateway), runs an aggregation engine that fills a local database with environmental data gathered by a locally deployed ZigBee wireless sensor network. Aggregated data are then retrieved by external queries over the built-in data integration system. In addition, embedded communication capabilities, including Bluetooth, IEEE 802.11 and GPRS, allow local and remote users to access both gateway and remote data, as well as the Internet, and run site-specific management tools using authenticated smartphones. Field experiments provide convincing evidence that iPAGAT represents an important step forward in the development of distributed service-oriented information systems for precision viticulture applications.

This paper describes a Viticulture Service-Oriented Framework (VSOF) which turns around context elements or tags that are placed in the field and which can be decoded by mobile devices such as mobile phones or PDAs. The tags are used to automatically associate a field location to the relevant database tables or records and also to access contextual information or services. By pointing a mobile device to a tag, the viticulturalist may download data such as climatic data or upload information such as disease and pest incidence in a simple way, without having to provide coordinates or any other references, and without having to return to a central office. This work is part of an effort to implement a large-scale distributed cooperative network in the Douro Demarcated Region in Northeast Portugal, a region in which the effort makes particular sense due to the extremely variable topography and mesoclimates. The possibility of exchanging contextualized information and accessing contextualized services in the field, using well-known devices such as cell phones, may contribute to increase the rate of adoption of information technology in viticulture, and contribute to more efficient and closer-to-the-crops practices.

The deployment of large mesh-type wireless networks is a challenge due to the multitude of arising issues. Perpetual operation of a network node is undoubtedly one of the major goals of any energy-aware protocol or power-efficient hardware platform. Energy harvesting has emerged as the natural way to keep small stationary hardware platforms running, even when operating continuously as network routing devices. This paper analyses solar radiation, wind and water flow as feasible energy sources that can be explored to meet the energy needs of a wireless sensor network router within the context of precision agriculture, and presents a multi-powered platform solution for wireless devices. Experimental results prove that our prototype, the MPWiNodeX, can manage simultaneously the three energy sources for charging a NiMH battery pack, resulting in an almost perpetual operation of the evaluated ZigBee network router. In addition to this, the energy scavenging techniques double up as sensors, yielding data on the amount of solar radiation, water flow and wind speed, a capability that avoids the use of specific sensors.

This paper is part of a long-term effort to introduce precision viticulture in the region of Demarcated Region of Douro. It presents the architecture, hardware and software of a platform designed for that purpose, called MPWiNodeZ. A major feature of this platform is its power-management subsystem, able to recharge batteries with energy harvested from the surrounding environment from up to three sources. It allows the system to sustain operation as a general-purpose wireless acquisition device for remote sensing in large coverage areas, where the power to run the devices is always a concern. The MPWiNodeZ, as a ZigBee™ network element, provides a mesh-type array of acquisition devices ready for deployment in vineyards. In addition to describing the overall architecture, hardware and software of the monitoring system, the paper also reports on the performance of the module in the field, emphasising the energy issues, crucial to obtain self-sustained operation. The testing was done in two stages: the first in the laboratory, to validate the power management and networking solutions under particularly severe conditions, the second stage in a vineyard. The measurements about the behaviour of the system confirm that the hardware and software solutions proposed do indeed lead to good performance. The platform is currently being used as a simple and compact yet powerful building block for generic remote sensing applications, with characteristics that are well suited to precision viticulture in the DRD region. It is planned to be used as a network of wireless sensors on the canopy of vines, to assist in the development of grapevine powdery mildew prediction models.