Real-Time CNN-based Computer Vision System for Open-Field Strawberry Harvesting Robot

Strawberry production in open-field conditions requires a lot of human labor, which is increasingly difficult to recruit. A robotic solution could potentially operate in the field around the clock with minimal supervision. For strawberry farmers, automation of harvesting would eliminate the personnel risk and provide security for operations in the long term. A robot which would be capable of replacing physical human labor in horticultural production requires an accurate and fast perception system. In this paper, we focus on the task of detecting of garden strawberries to guide the picking by a strawberry harvesting robot. We have developed a real-time implementation of strawberry and peduncle detection system that runs on an edge device. This paper outlines the vision system requirements, hardware selection, model selection, training process and results. After consideration of the overall requirements of the system, we decided to use YOLOv5 to detect both the berries and peduncles for the picking system. Training data was collected and annotated, and the detection model was trained. The network had 91.5% average precision (AP) for detecting strawberries and an 43.6% AP for detecting peduncles. One of the reasons for performance discrepancy was the difficulty to detect peduncles from afar. Overall, the vision algorithm reached the performance that was required to guide the robot to a strawberry and detect the corresponding strawberry-peduncle pairs. However, for densely clustered berries the method often failed to detect the correct peduncle and needs to be improved.