This research focuses on the development of a drone equipped with pruning tools and a stereo vision camera to accurately detect and measure the spatial positions of tree branches. YOLO is employed for branch segmentation, while two depth estimation approaches, monocular and stereo, are investigated. In comparison to SGBM, deep learning techniques produce more refined and accurate depth maps. In the absence of ground-truth data, a fine-tuning process using deep neural networks is applied to approximate optimal depth values. This methodology facilitates precise branch detection and distance measurement, addressing critical challenges in the automation of pruning operations. The results demonstrate notable advancements in both accuracy and efficiency, underscoring the potential of deep learning to drive innovation and enhance automation in the agricultural sector.

This paper introduces a method for Pinus Radiata cutpoint modelling and detection that, to the best of our knowledge, offers the first solution capable of determining high-precision pruning cutpoints for autonomous forestry applications. We utilise a sparse convolutional neural network to produce a tree skeleton from 3D point cloud data, then determine branch collar regions and cutpoints using a novel detection algorithm based on real-world forest measurements. Our proposed method is robust in complex environments, producing branch cutpoints even through occlusions from pine needles and other fine structures, and achieving cutpoint detection times of around 500 ms. We evaluate our method using a synthetic tree dataset and real P. Radiata forest data, with cutpoint estimates assessed against industry 'best practices' for P. Radiata forestry pruning. Our evaluation using synthetic and real-world data demonstrates the precision of our approach when compared to prior tree pruning methods.

Manual pruning of radiata pine trees presents significant safety risks due to their substantial height and the challenging terrains in which they thrive. To address these risks, this research proposes the development of a drone-based pruning system equipped with specialized pruning tools and a stereo vision camera, enabling precise detection and trimming of branches. Deep learning algorithms, including YOLO and Mask R-CNN, are employed to ensure accurate branch detection, while the Semi-Global Matching algorithm is integrated to provide reliable distance estimation. The synergy between these techniques facilitates the precise identification of branch locations and enables efficient, targeted pruning. Experimental results demonstrate that the combined implementation of YOLO and SGBM enables the drone to accurately detect branches and measure their distances from the drone. This research not only improves the safety and efficiency of pruning operations but also makes a significant contribution to the advancement of drone technology in the automation of agricultural and forestry practices, laying a foundational framework for further innovations in environmental management.

This research focuses on the development of a deep learning based method to enable a drone equipped with a stereo vision camera to accurately detect and measure the spatial positions of tree branches. YOLO is employed for branch segmentation, while two depth estimation approaches, monocular and stereo, are investigated. In comparison to Semi-Global Block Matching(SGBM), deep learning techniques produce more refined and accurate depth maps. In the absence of ground-truth data, a fine-tuning process with deep neural networks is applied to generate the depth map that most closely approximates the ground-truth. This methodology achieves accurate branch detection and precise distance measurement, addressing key challenges in automating pruning operations. The results indicate substantial improvements in accuracy, though further optimization is required to enhance processing speed, demonstrating the potential of deep learning to advance automation in agricultural systems.

Capturing outdoor visual-inertial datasets is a challenging yet vital aspect of developing robust visual-inertial odometry (VIO) algorithms. A significant hurdle is that high-accuracy-ground-truth systems (e.g., motion capture) are not practical for outdoor use. One solution is to use a "semi-synthetic" approach that combines rendered images with real IMU data. This approach can produce sequences containing challenging imagery and accurate ground truth but with less simulated data than a fully synthetic sequence. Existing methods (used by popular tools/datasets) record IMU measurements from a visual-inertial system while measuring its trajectory using motion capture, then rendering images along that trajectory. This work identifies a major flaw in that approach, specifically that using motion capture alone to estimate the pose of the robot/system results in the generation of inconsistent visual-inertial data that is not suitable for evaluating VIO algorithms. However, we show that it is possible to generate high-quality semi-synthetic data for VIO algorithm evaluation. We do so using an open-source full-batch optimisation tool to incorporate both mocap and IMU measurements when estimating the IMU's trajectory. We demonstrate that this improved trajectory results in better consistency between the IMU data and rendered images and that the resulting data improves VIO trajectory error by 79% compared to existing methods. Furthermore, we examine the effect of visual-inertial data inconsistency (as a result of trajectory noise) on VIO performance to provide a foundation for future work targeting real-time applications.

Pinus Radiata trees form pollen-producing catkins that can be harvested for pharmaceutical uses. Unmanned Aerial Vehicles (UAVs) may be well suited to the task of autonomously harvesting these catkins. We propose a method to reliably detect and track P. Radiata catkins in three dimensions that can be used for real-time guidance of a UAV. The method applies the YOLOv5 deep learning algorithm to detect catkins in the X-Y plane. A novel optimisation of the MeanShift algorithm is utilised to assist existing contour detection algorithms in segmenting individual catkins in the Z plane. A Kanade-Lucas-Tomasi tracker was used with RANSAC for accurate frame-to-frame tracking. The method achieved a Mean Average Precision of 0.87 on images taken at a commercial pine pollen farm. The method detected the depth of catkins at distances of up to 1200 mm to an accuracy of 2 mm, or 8 mm for occluded catkins. Detected catkins can be reliably tracked at speeds of 1ms. An average frame rate of 22 frames per second was achieved on an Intel i5 CPU, with the Meanshift optimisation performing up to 41 times faster than existing implementations. These results indicate that the proposed method could be used to successfully assist in the automated harvesting of P. Radiata catkins.

We present a novel outlier detector for visual odometry algorithms operating in dynamic vegetated environments. The outlier detector utilises the difference in optical flow patterns caused by camera motion and plants moving in the wind. The proposed method is compared to existing methods on real and synthetic data containing wide ranges of scene motion. Our results show that the proposed method works well as a pre-processing step to RANSAC, improving pose estimation accuracy in dynamic vegetated environments with minimal computation overhead.

Visual odometry estimates a robot’s pose by analysing images it captured and is integral to autonomous navigation. However, when devices operate outdoors, the large changes in brightness may cause localisation failure. This shortcoming motivates us to find methods that ensure visual odometry can function in high dynamic range environments.To mitigate the negative impact of operating in high dynamic range environments, researchers proposed automatic exposure algorithms to select a camera exposure time that maximises their selected image quality metric. However, direct comparisons between the selected image metric and visual odometry performance is scarce. Many of the proposed algorithms go through multiple steps, making it difficult to determine whether the improved pose estimation accuracy is the result of optimising the metric.In this submission, we generated image sequences in a simulation environment and captured real-world photos at different exposure levels. We estimate trajectories for each data set and compare the resulting errors with the image quality metrics. Our tests showed that none of the selected image quality metrics had a strong correlation with absolute translation error. The findings suggest that the current automatic exposure algorithms have yet to find an image quality metric that truly impacts visual odometry performance. Ongoing research should consider alternative metrics or metric combinations.

This paper examines the effect of moving vegetation on visual-inertial-odometry performance. The evaluation was performed using a custom dataset consisting of fifteen sequences in a forest scene. The sequences are comprised of three trajectories, each with five different wind/scene motion levels — allowing the relationship between scene motion and state-estimation accuracy to be examined. The dataset was created using a "semi-synthetic" approach that combines real IMU data and synthetic images to provide increased realism while maintaining precise control over the scene. The dataset is used to assess the effect of scene motion on multiple open-source visual-inertial odometry algorithms in a variety of configurations. Additionally, the results are validated using real-world sequences containing imagery of vegetation moving in the wind. The results show that scene motion as a result of moving vegetation can worsen visual-inertial odometry performance by up to a factor of ten.

Motion capture is commonly used to track the 3D pose of a camera in order to provide accurate ground truth data for computer vision algorithms such as visual odometry, SLAM and object tracking. However, it is challenging to manually align the coordinate frame of a camera and a motion capture-tracked object. This paper proposes a method for estimating the transformation between these two coordinate frames. The suggested approach does not require a special calibration object, instead using the motion capture markers directly. Thus, it has fewer opportunities for human-introduced bias than some existing methods. The proposed method achieves a reprojection error of 7.4px compared to 12.0px for previous work and 84.9px for manual alignment. These results indicate that the proposed calibration method can significantly improve the accuracy of ground truth data obtained from motion capture.

A novel method is proposed to acquire tree metrics for forestry managers in New Zealand. This method involves using a UAV to capture images of trees, and then using photogrammetry to generate a 3D model which includes both a mesh and a point cloud. The point cloud is then analysed, and data and metrics are extracted. Once the metrics have been extracted, they are saved for analysis and exported to a virtual reality (VR) environment. The VR environment will allow end users to remotely interact with sections of forest replicated with photogrammetry and visualise the metrics created through point cloud analysis.

For a UAV to operate autonomously, it needs an estimate of its position. This estimate is commonly obtained by fusing IMU measurements with another pose estimation system like visual odometry (VO) or simultaneous localisation and mapping (SLAM). Although there is existing research examining the use of these pose estimation algorithms with UAVs, results are often given in terms of estimation accuracy and processing time, rather than position control performance. While it is often more practical to obtain and compare these results, they do not necessarily indicate how a UAV will perform when using the proposed algorithm. This paper examines the relationship between the properties of pose estimation systems-namely framerate, latency and error-and UAV position control performance. A number of experiments are performed using the Gazebo simulation environment and the PX4 software-in-the-loop (SITL) flight stack. Two important conclusions can be drawn from the results. The first is that the minimum desirable framerate is dependent on the level of noise present in the estimate; after this threshold has been reached, framerate has little effect on position control performance. The second is that estimation error is the most important factor to consider when developing a pose estimation algorithm for UAVs, as it has the biggest impact on RMSE. This suggests that for UAVs with sufficient processing power, using accuracy-focused SLAM algorithms rather than efficiency-focused VO algorithms could provide better position control performance.