Marine biofouling on vessel hulls poses major ecological, economic, and biosecurity risks. Traditional survey methods rely on diver inspections, which are hazardous and limited in scalability. This work investigates automated classification of biofouling severity on the Level of Fouling (LoF) scale using both custom computer vision models and large multimodal language models (LLMs). Convolutional neural networks, transformer-based segmentation, and zero-shot LLMs were evaluated on an expert-labelled dataset from the New Zealand Ministry for Primary Industries. Computer vision models showed high accuracy at extreme LoF categories but struggled with intermediate levels due to dataset imbalance and image framing. LLMs, guided by structured prompts and retrieval, achieved competitive performance without training and provided interpretable outputs. The results demonstrate complementary strengths across approaches and suggest that hybrid methods integrating segmentation coverage with LLM reasoning offer a promising pathway toward scalable and interpretable biofouling assessment.

Categorical Distributional Reinforcement Learning (CDRL) has demonstrated superior sample efficiency in learning complex tasks compared to conventional Reinforcement Learning (RL) approaches. However, the practical application of CDRL is encumbered by challenging projection steps, detailed parameter tuning, and domain knowledge. This paper addresses these challenges by introducing a pioneering Continuous Distributional Model-Free RL algorithm tailored for continuous action spaces. The proposed algorithm simplifies the implementation of distributional RL, adopting an actor-critic architecture wherein the critic outputs a continuous probability distribution. Additionally, we propose an ensemble of multiple critics fused through a Kalman fusion mechanism to mitigate overestimation bias. Through a series of experiments, we validate that our proposed method provides a sample-efficient solution for executing complex continuous-control tasks.

Monocular depth estimation is a rudimentary problem for robotic perception systems and downstream applications. However, depth estimation from a single image is an inherently ill-posed problem due to data loss related to projection from 3D to 2D. Recent studies address the discrepancy between camera parameters by using learning-based methods and unifying the camera model to canonical camera space or bipolar representations, thus addressing the problem of training a metric depth model over different datasets with different camera parameters. In addition, the previous study, OrchardDepth, introduced the sparse-dense depth consistency loss function to learn the dense depth distribution through the city autonomous driving scene to improve model performance in the orchard. Instead of enforcing strict consistency between the sparse and dense depth, this work introduced the KL divergence to encourage the network to adapt to the depth distributions of different sensors and penalize deviations from reliable regions while tolerating errors in unreliable areas. Furthermore, we further enhance the depth consistency loss by integrating bins into the supervised discretised depth distribution. This method significantly improves the robustness and performance of our previous method. In addition, it improves the absolute relative error in the orchard dataset by 17.3% and 16.2% in contrast to SILog Loss and OrchardDepth baseline, respectively. Thus enhancing the new training paradigm for depth estimation in the orchard scene.

Aotearoa's apple industry struggles to maintain the skilled workforce required for fruitlet thinning each year. Skilled labourers play a pivotal role in managing crop loads by precisely thinning fruitlets to a desired number to achieve the desired spacing for high-quality apple growth. This complex task requires accurate mapping of the fruitlets along each branch. This paper presents a novel vision system capable of mapping the orientation and clustering information of apple fruitlets. Fruitlet pose estimation has been validated against data collected from a real-world commercial apple orchard. The results show an improved counting accuracy of 83.97% on prior implementations, an orientation estimate accuracy of 88.1%, and a clustering accuracy of 94.3%. Future work will utilise this information to determine which fruitlets to remove and then robotically thin them from the canopy.

The global warehousing industry is experiencing rapid growth, with the market size projected to grow at an annual rate of 8.1 Research, 2021]. This expansion has led to a surge in demand for efficient pallet detection and localisation systems. While automation can significantly streamline warehouse operations, the development of such systems often requires extensive manual data annotation, with an average of 35 seconds per image, for a typical computer vision project. This paper presents a novel approach to enhance pallet detection and localisation using purely synthetic data and geometric features derived from their side faces. By implementing a domain randomisation engine in Unity, the need for time-consuming manual annotation is eliminated while achieving high-performance results. The proposed method demonstrates a pallet detection performance of 0.995 mAP50 for single pallets on a real-world dataset. Additionally, an average position accuracy of less than 4.2 cm and an average rotation accuracy of 8.2 were achieved for pallets within a 5-meter range, with the pallet positioned head-on.

Humanoid robots are designed to be relatable to humans for applications such as customer support and helpdesk services. However, many such systems, including Softbank's Pepper, fall short because they fail to communicate effectively with humans. The advent of Large Language Models (LLMs) shows the potential to solve the communication barrier for humanoid robotics. This paper outlines the comparison of different Automatic Speech Recognition (ASR) APIs, the integration of Whisper ASR and ChatGPT with the Pepper robot and the evaluation of the system (Pepper-GPT) tested by 15 human users. The comparison result shows that, compared to the Google ASR and Google Cloud ASR, the Whisper ASR performed best as its average Word Error Rate (1.716 (2.639 s) are both the lowest. The participants' usability investigations show that 60 "excellent", while the rest rated this system as "good" in the subsequent experiments. It is proved that while some problems still need to be overcome, such as the robot's multilingual ability and facial tracking capacity, users generally responded positively to the system, feeling like talking to an actual human.

The use of synthetic data in machine learning saves a significant amount of time when implementing an effective object detector. However, there is limited research in this domain. This study aims to improve upon previously applied implementations in the task of instance segmentation of pallets in a warehouse environment. This study proposes using synthetically generated domain-randomised data as well as data generated through Unity to achieve this. This study achieved performance improvements on the stacked and racked pallet categories by 69 data. Additionally, it was found that there was a considerable impact on the performance of a model when it was evaluated against images in a darker environment, dropping as low as 3 80 used YOLOv8 and SAM, but this proved to have unstable performance. The use of domain-randomised data proved to have negligible performance improvements when compared to the Unity-generated data.

Reinforcement Learning (RL) has been widely used to solve tasks where the environment consistently provides a dense reward value. However, in real-world scenarios, rewards can often be poorly defined or sparse. Auxiliary signals are indispensable for discovering efficient exploration strategies and aiding the learning process. In this work, inspired by intrinsic motivation theory, we postulate that the intrinsic stimuli of novelty and surprise can assist in improving exploration in complex, sparsely rewarded environments. We introduce a novel sample-efficient method able to learn directly from pixels, an image-based extension of TD3 with an autoencoder called NaSA-TD3. The experiments demonstrate that NaSA-TD3 is easy to train and an efficient method for tackling complex continuous-control robotic tasks, both in simulated environments and real-world settings. NaSA-TD3 outperforms existing state-of-the-art RL image-based methods in terms of final performance without requiring pre-trained models or human demonstrations.

Apple fruitlet thinning is critical in cultivating high-quality apples, requiring an expert workforce to manage the orchard. The thinning process requires precise mapping of fruitlet clusters across the tree branches to manage the desired load for each tree. This paper presents Archie Snr, which was developed to autonomously assess the current load of the tree and thin the excess apples as an expert thinner would. The platform has been extensively evaluated in a real-world commercial orchard. The results show the platform can generate an average load count accuracy of 82.1% with a recall of 93.3%. The system was then able to successfully thin 66.14% of the fruitlets from the canopy.

Reinforcement Learning (RL) training is predominantly conducted in cost-effective and controlled simulation environments. However, the transfer of these trained models to real-world tasks often presents unavoidable challenges. This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation. The benchmarking results of three RL algorithms trained on intricate in-hand manipulation tasks within practical real-world contexts are presented. Our study not only demonstrates the practicality of RL training in authentic real-world scenarios, facilitating direct real-world applications, but also provides insights into the associated challenges and considerations. Additionally, our experiences with the employed experimental methods are shared, with the aim of empowering and engaging fellow researchers and practitioners in this dynamic field of robotics.

The primary goal of our project is to generate artificial humanoid avatars, specifically talking faces from a single image and a text for enhanced human-robotic interaction. We put a specific emphasis on avatars that exhibit precise lip motion, head movement, and dynamic facial expressions. We believe that these attributes are essential components, making avatars significantly more engaging to human users. Contrary to traditional 3D modelling techniques that are commonly used in many modern state-of-the-art systems, our project aims to build avatars from machine-learned image augmentations. While numerous studies have been conducted on talking face generation systems, most have explored lip-motion in isolation from emotional facial shifts. Additionally, many methods depend heavily on audio or video inputs. In this paper, we propose an emotional talking face generation, called EmoFaceGen, which generates realistic talking face videos with emotions. Our system is unique in that it creates emotional talking face videos taking a single facial image and a text as inputs, then producing a talking face video with emotions as an output. The text input is converted to an audio source using the Text-To-Speech method for our project. Based on our findings, EmoFaceGen provides a more realistic talking face representation compared to other open-source models, highlighting a positive direction in overcoming present challenges in this area especially when considering the memory and hardware limitations associated with conventional 3D graphics methods.

Cane pruning grapevines is a complex manual task requiring expert vine assessment to determine which canes to prune. This paper presents Archie Jnr, which was developed to autonomously assess the structure of the vine and prune the lower-quality canes as an expert pruner would. The platform has been extensively evaluated in a real-world commercial vineyard using a three-cane pruning method. The results show the effectiveness of the vision system for generating accurate assessments of a vine's canes. The platform is also shown to be capable of successfully pruning 71.1% of the 311 total canes that required pruning across 25 vines.

This research sets out to assess the viability of using game engines to generate synthetic training data for machine learning in the context of pallet segmentation. Using synthetic data has been proven in prior research to be a viable means of training neural networks and saves hours of manual labour due to the reduced need for manual image annotation. Machine vision for pallet detection can benefit from synthetic data as the industry increases the development of autonomous warehousing technologies. As per our methodology, we developed a tool capable of automatically generating large amounts of annotated training data from 3D models at pixel-perfect accuracy and a much faster rate than manual approaches. Regarding image segmentation, a Mask R-CNN pipeline was used, which achieved an AP50 of 86% for individual pallets.

With the rising popularity of autonomous navigation research, Formula Student (FS) events are introducing a Driverless Vehicle (DV) category to their event list. This paper presents the initial investigation into utilising Deep Reinforcement Learning (RL) for end-to-end control of an autonomous FS race car for these competitions. We train two state-of-the-art RL algorithms in simulation on tracks analogous to the full-scale design on a Turtlebot2 platform. The results demonstrate that our approach can successfully learn to race in simulation and then transfer to a real-world racetrack on the physical platform. Finally, we provide insights into the limitations of the presented approach and guidance into the future directions for applying RL toward full-scale autonomous FS racing.

Following a global trend, the lack of reliable access to skilled labour is causing critical issues for the effective management of apple orchards. One of the primary challenges is maintaining skilled human operators capable of making precise fruitlet thinning decisions. Thinning requires accurately measuring the true crop load for individual apple trees to provide optimal thinning decisions on an individual basis. A challenging task due to the dense foliage obscuring the fruitlets within the tree structure. This paper presents the initial design, implementation, and evaluation details of the vision system for an automatic apple fruitlet thinning robot to meet this need. The platform consists of a UR5 robotic arm and stereo cameras which enable it to look around the leaves to map the precise number and size of the fruitlets on the apple branches. We show that this platform can measure the fruitlet load on the apple tree to with 84% accuracy in a real-world commercial apple orchard while being 87% precise.

Model Free Reinforcement Learning (MFRL) has shown significant promise for learning dexterous robotic manipulation tasks, at least in simulation. However, the high number of samples, as well as the long training times, prevent MFRL from scaling to complex real-world tasks. Model- Based Reinforcement Learning (MBRL) emerges as a potential solution that, in theory, can improve the data efficiency of MFRL approaches. This could drastically reduce the training time of MFRL, and increase the application of RL for real- world robotic tasks. This article presents a study on the feasibility of using the state-of-the-art MBRL to improve the training time for two real-world dexterous manipulation tasks. The evaluation is conducted on a real low-cost robot gripper where the predictive model and the control policy are learned from scratch. The results indicate that MBRL is capable of learning accurate models of the world, but does not show clear improvements in learning the control policy in the real world as prior literature suggests should be expected.

Aotearoa New Zealand has a strong and growing apple industry but struggles to access workers to complete skilled, seasonal tasks such as thinning. To ensure effective thinning and make informed decisions on a per-tree basis, it is crucial to accurately measure the crop load of individual apple trees. However, this task poses challenges due to the dense foliage that hides the fruitlets within the tree structure. In this paper, we introduce the vision system of an automated apple fruitlet thinning robot, developed to tackle the labor shortage issue. This paper presents the initial design, implementation,and evaluation specifics of the system. The platform straddles the 3.4 m tall 2D apple canopy structures to create an accurate map of the fruitlets on each tree. We show that this platform can measure the fruitlet load on an apple tree by scanning through both sides of the branch. The requirement of an overarching platform was justified since two-sided scans had a higher counting accuracy of 81.17 % than one-sided scans at 73.7 %. The system was also demonstrated to produce size estimates within 5.9% RMSE of their true size.

Aotearoa (New Zealand) has a strong and growing winegrape industry struggling to access workers to complete skilled, seasonal tasks such as pruning. Maintaining high-producing vines requires training agricultural workers that can make quality cane pruning decisions, which can be difficult when workers are not readily available. A novel vision system for an autonomous cane pruning robot is presented that can assess a vine to make quality pruning decisions like an expert. The vision system is designed to generate an accurate digital 3D model of a vine with skeletonised cane structures to estimate key pruning metrics for each cane. The presented approach has been extensively evaluated in a real-world vineyard as a commercial platform would be expected to operate. The system is demonstrated to perform consistently at extracting dimensionally accurate digital models of the vines. Detailed evaluation of the digital models shows that 51.45% of the canes were modelled entirely, with a further 35.51% only missing a single internode connection. The quantified results demonstrate that the robotic platform can generate dimensionally accurate metrics of the canes for future decision-making and automation of pruning.

As technology progresses, smart automated systems will serve an increasingly important role in the agricultural industry. Current existing vision systems for yield estimation face difficulties in occlusion and scalability as they utilize a camera system that is large and expensive, which are unsuitable for orchard environments. To overcome these problems, this paper presents a size measurement method combining a machine learning model and depth images captured from three low cost RGBD cameras to detect and measure the height and width of tomatoes. The performance of the presented system is evaluated on a lab environment with real tomato fruits and fake leaves to simulate occlusion in the real farm environment. To improve accuracy by addressing fruit occlusion, our three-camera system was able to achieve a height measurement accuracy of 0.9114 and a width accuracy of 0.9443.

Capturing sediment topography is one component in understanding how large animals in soft sediment ecosystems affect ecosystem processing. Traditional approaches for sampling soft-sediment systems are too invasive and destructive for the fragile sediment structures. In contrast, non-invasive approaches, such as LIDAR and monocular structure-from-motion (SfM) systems, can be expensive and cumbersome for data processing. We developed a low cost and practical framework for measuring morphological characteristics of soft sediment topography at the millimetre scale. Using an RGB-D (red, green, blue and depth) device and a semi-opaque container, the sediment surface can be imaged rapidly while operating in outdoor environments. The RGB-D device imaged 0.3 m(2) intertidal sediment surfaces, creating depth imagery over seven sites and 82 surfaces. First, using simulated surface data, we formulated and tested four variations of a geometrical detrending model to extract and measure the undulating surface microtopography. We then computed nine morphological characteristics, including arithmetic mean roughness, mean sediment peaks and rugosity, for the detrending models. We then used the two best performing models, linear and quadratic detrending models, to compare extracted morphological characteristics from the field data. We found a strong correlation between the models for extracted surface measures and similar extracted sediment trends. The results showed that RGB-D real-time imaging is a promising rapid scanning tool for collecting field data in intertidal regions and can be expanded to other fragile sediment surfaces outside the marine environment. The low costs and real-time feedback makes it an attractive data collection tool for environments where data collection is challenging.