Hou Rongo is a multi-sensory, interactive exhibition revitalising Moriori culture and evoking the feeling of being on Rekohu (Chatham Island). Many different exhibits were created for this event, in close collaboration and co-design with Ta Imi Moriori (community). In this work we discuss two of the exhibits that use visual computing technologies to allow visitors to interact with the forests and birds of Rekohu. The first, AR Rakau Momori, allows visitors to closely examine culturally significant rakau momori in augmented reality. The second, Manu Moriori, creates a forest filled with colourful birds drawn by visitors and reflects the importance of birds in Moriori life and culture. Although just a small part of the broader exhibition, these examples illustrate how careful co-design can lead to interactive experiences that reflect culture and place.
Zero-shot image denoising, the process of removing noise from images without ground truth, is becoming increasingly important across various fields. Current denoising methods often downsample noisy images and employ residual and consistency loss functions to learn and subtract noise from base image. However, these methods struggle to discern the superiority between different downsampled images. To address this limitation, we propose an alpha-conditional loss function, combined with a 3x3 window downsampler, and a light-weight convolutional neural network, which effectively handles various noise types and levels. Notably, our method, named ZS-ACL, is computationally efficient by consisting of just 6K model parameters, thus distinguishing itself from others in the field. The experimental results on established real-world datasets demonstrate that ZS-ACL either outperforms or matches existing approaches in various scenarios, even with significantly fewer parameters. It learns from only a single image, thus presenting an efficient dataset-free denoising solution. Moreover, it showcases versatility and robustness by achieving better results for varying noise levels. The code is made available on Github at https://github.com/kshahmir49/ZS-ACL.
Event cameras, which capture pixel-level intensity changes asynchronously, offer minimal motion blur, high temporal resolution, and low power usage compared to traditional cameras. However, with limited availability and the high cost of real event data, synthetic event data has become a crucial alternative for applications such as retrospective gaze analysis in medical research, user behavior studies, and neurocognitive research. It allows simulation of experimental conditions, optimization of gaze estimation algorithms, and detailed scenario testing, including rare events, without costly real-world data collection. In this paper, we demonstrate that synthetic event data can match the performance of real event data in saccadic gaze prediction. To support this, we introduce a synthetic event-based gaze dataset focused on saccadic eye movements and propose a novel Dual ResUNet architecture designed to process consecutive frames with synthetic encoded events for gaze estimation. Our network achieves feature-level domain adaptation and temporally consecutive centroid prediction, delivering 99.20% accuracy within a 100-pixel radius. Our research underscores the potential of synthetic temporally encoded images for precise gaze vector prediction while effectively minimizing the domain gap with real event data.
Atmospheric turbulence generates wavefront distortions in light rays from distant sources, thus reducing the resolution of astronomical images. Adaptive Optics (AO) systems enhance telescopes' imaging capabilities by compensating for these aberrations in real time. This paper presents the design and laboratory prototyping of a high-order AO system intended for the University of Canterbury's 0.61 m Boller & Chivens (B&C) Telescope. The AO system incorporates a 12 x 12 actuator Micro-Electro-Mechanical Systems Deformable Mirror to simulate atmospheric conditions and static aberration screens, and a Thorlabs DM and Shack-Hartmann Wavefront Sensor for closed loop correction. The system was tested in the laboratory under simulated atmospheric conditions, with a focus on correcting higher-order aberrations, such as defocus and astigmatism. Experimental results demonstrate significant improvements in the Full Width at Half Maximum of the Point Spread Function, indicating the system's potential for improving image quality. The prototype's design allows for simple integration with the B&C Telescope, paving the way for its future implementation at the University of Canterbury's Mount John Observatory.
This research focuses on the development of a deep learning based method to enable a drone equipped with a stereo vision camera to accurately detect and measure the spatial positions of tree branches. YOLO is employed for branch segmentation, while two depth estimation approaches, monocular and stereo, are investigated. In comparison to Semi-Global Block Matching(SGBM), deep learning techniques produce more refined and accurate depth maps. In the absence of ground-truth data, a fine-tuning process with deep neural networks is applied to generate the depth map that most closely approximates the ground-truth. This methodology achieves accurate branch detection and precise distance measurement, addressing key challenges in automating pruning operations. The results indicate substantial improvements in accuracy, though further optimization is required to enhance processing speed, demonstrating the potential of deep learning to advance automation in agricultural systems.
Virtual try-on has emerged as a prominent field of research because of its wide application in ecommerce. The key challenge in this task is producing realistic try-on images, where the garment is warped to accurately fit the person's body. The try-on generation quality is of significance to provide desirable and realistic results. In this work, we improve the realism and accuracy of virtual try-on by enhancing the architecture to accept attention conditioning. Instead of using multiple ControlNets, we introduce a novel Multi Zero Cross Residual Attention Block (MZB) that replaces the standard attention mechanism, enabling the simultaneous incorporation of multiple conditioning inputs. These include DinoV2 embeddings for rich semantic cloth representation, color histograms to preserve garment color fidelity, and skin maps to ensure coherence between the garment and exposed skin areas. This design allows the diffusion model to effectively handle complex garment-body interactions with greater precision. Through extensive experiments, our method demonstrates superior control over garment alignment and visual realism, achieving competitive results on standard virtual try-on benchmarks.
This paper introduces a method for Pinus Radiata cutpoint modelling and detection that, to the best of our knowledge, offers the first solution capable of determining high-precision pruning cutpoints for autonomous forestry applications. We utilise a sparse convolutional neural network to produce a tree skeleton from 3D point cloud data, then determine branch collar regions and cutpoints using a novel detection algorithm based on real-world forest measurements. Our proposed method is robust in complex environments, producing branch cutpoints even through occlusions from pine needles and other fine structures, and achieving cutpoint detection times of around 500 ms. We evaluate our method using a synthetic tree dataset and real P. Radiata forest data, with cutpoint estimates assessed against industry 'best practices' for P. Radiata forestry pruning. Our evaluation using synthetic and real-world data demonstrates the precision of our approach when compared to prior tree pruning methods.
Multi-object tracking (MOT) is the task of tracking multiple entities across video frames and maintaining identity labels over time. This has many applications, including in animal research and behavioural studies. Animal tracking applications have many unique challenges, such as frequent occlusions, interactions, and homogenous appearance. The most common paradigm for MOT is tracking-by-detection, which relies on powerful object detection models to detect objects and associate the detected objects between frames. These models rely on well-performing detection models; however, the impact of poor detection accuracy has not been quantified. In this study, we trained six variants of YOLOv10 on a dataset of meerkats in a zoo environment and tested four tracking-by-detection methods, BoT-SORT, DeepSORT, ByteTrack, and OC-SORT, to measure the effect of varying detection performances. Our experiments showed that detection accuracy significantly affects tracking metrics, such as identity consistency, HOTA, and MOTA, particularly in challenging scenarios involving occlusions and rapid movement. We also highlight the common detection and tracking challenges with animal datasets and compare the performances of the evaluated tracking methods. Ultimately, this study highlights the importance of optimising detection models to achieve reliable MOT performance in complex environments, contributing to more accurate animal behaviour monitoring and ecological research.
Visual attention is crucial for identifying the most salient regions in an image. However, when the objective involves higher-level visual tasks, such as target object detection, it becomes necessary to incorporate high-level information to guide the search for the target. This process, known as top-down saliency detection, leverages guidance sources like contextual information and target features to identify regions of interest that are more likely to contain the target object. In this paper, we propose a model that generates top-down saliency maps by adjusting the feature map weights of a universal visual attention model based on contextual information. While contextual information has traditionally been used to understand the gist of an image, it has not been integrated into the creation of saliency maps for target object detection. We demonstrate that incorporating contextual information into a visual attention model enhances target object detection performance. The proposed model, tested on six datasets, shows significant improvements in detecting target objects compared to models that do not utilize contextual information.
The demand for Artificial Intelligence (AI)-powered solutions in New Zealand's aquaculture sector, particularly in mussel farms, is fast growing due to the need for enhanced productivity, reduced carbon footprint, and a safer worksite. An effective crop monitoring system that relies on high-resolution images from boat-mounted cameras requires an effective multi-object matching module. This paper presents a new Genetic Programming (GP)-based multi-object matching approach for mussel floats in farm images, addressing site-specific challenges such as a large number of tracklets, near-identical appearances, and low-overlapping scenes. Current object matching methods, including appearance models, motion models, and transformer models, struggle under these conditions. Evaluated on mussel farm images from the Marlborough Sounds of New Zealand, the proposed GP method demonstrates significantly higher matching accuracy, achieving 96.4% compared to 39.6% by a state-of-the-art transformer model. Our GP method achieves accurate multi-object matching in both slow and fast-moving scenarios, outperforming the state-of-the-art particularly in the latter.
Recent advancements in vision transformers and self-supervised learning are expanding the capabilities of computer vision models. This study explores the application of a DINOv2-based unsupervised approach for the re-identification of kaka, a forest parrot endemic to New Zealand. We measure the performance of our vision transformer against a canonical SIFT-based method to establish its utility in accurately identifying individual birds. Using video recordings of wild birds captured at purpose-built feeders over three distinct periods, we present evaluations of our models using extracted images. The results demonstrate that our DINOv2-based model achieves high accuracy, outperforming our SIFT-based approach. Deep learning models are often considered unexplainable. We offer a window into our model utilising patch embeddings to highlight key features of the kaka. These findings suggest that a vision transformer-based method is an effective non-invasive tool for improving conservation efforts to monitor growing populations of threatened parrots such as the kaka.
Foreign object contamination remains a problem in food processing despite the existing countermeasures to detect and remove unintended objects during the food production process. Most imaging techniques only work well in detecting specific objects but may not be suitable in more challenging problem settings, such as objects with different shapes, con-ductivities, and densities in aqueous food media. This paper introduces a new approach for detecting objects embedded in aqueous food media using surface potential measurements obtained from a three-dimensional cylindrical sensing pipe and convolutional neural network (CNN) classifiers. The simulated electric potential measurements from the surface of the sensing pipe are constructed as two-dimensional images, providing essential information for the CNN classifier to infer the presence of objects and to distinguish opposing conductivities based on distinctive spatial characteristics. Experimental results evaluated on unseen object shapes and orientations suggest that surface potential measurements with low noise interference and CNN classifiers can be used to reliably detect the presence of objects and classify the conductivity of the objects.
The use of autonomous systems within the retail environment is currently hampered by the challenges object detection networks face when trying to locate and classify products. These challenges arise from densely packed product arrangements, limited training data, and frequent product up-dates. Existing object detection methods such as R-CNN and YOLO exhibit limitations in scalability and accuracy when applied to these complex environments. This work explores a new vision algorithm designed specifically for retail applications by integrating two distinct networks to address the dual object detection tasks of localization and classification. The first task utilizes RetinaNet to localize densely packed products, relying on its ability to detect small overlapping objects. Then, a Siamese Neural Network (SNN) is used to generate embeddings of local-ized product images. SNNs learn to create feature embed dings with minimal training data and can support the addition of new classifications without requiring network retraining. This separation of localization and classification tasks allows each network to be fine-tuned for its specific purpose, resulting in a final object detection algorithm that can adequately perform in the retail landscape.
Accurate 3D modelling of grapevines is crucial for precision viticulture, particularly for informed pruning decisions and automated management techniques. However, the intricate structure of grapevines poses significant challenges for traditional skeletonization algorithms. This paper presents an adaptation of the Smart-Tree algorithm for 3D grapevine modelling, addressing the unique characteristics of grapevine structures. We introduce a graph-based method for disambiguating skeletonization. Our method delineates individual cane skeletons, which are crucial for precise analysis and management. We validate our approach using annotated real-world grapevine point clouds, demonstrating an improvement of 15.8% in the F1 score compared to the original Smart-Tree algorithm. This research contributes to advancing 3D grapevine modelling techniques, potentially enhancing both the sustainability and profitability of grape production through more precise and automated viticulture practices.
The tracking of torsional eye movements, where the eye rotates about the axis of vision, is important for the diagnosis of certain vestibular disorders, but presents additional challenges when compared to the tracking of linear eye movements. This paper proposes a method for the measurement of ocular torsion that combines techniques from several previous studies, attempting to make the measurement robust to a highly off-axis centre of vision and obscurations of the pupil and iris. The proposed method detects feature points across a region of interest (ROI) including both the iris and the sclera (the white part of the eye surrounding the iris), using the Affine-SIFT (ASIFT) feature detection method, and tracks the feature points across frames using the Lucas-Kanade Optical Flow Method. The proposed method was only able to achieve an average error of 0.42 degrees compared to 0.17 degrees achieved in prior research. However, an additional 23.5% of feature points were able to be tracked compared to feature matching on the iris alone, showing that the proposed method has the potential to provide more accurate tracking than conventional methods. Future improvements proposed to the method include a deep segmentation method to automatically determine the ROI in each frame, and further testing with a calibration step to determine the source of error in the method.
Robustly identifying overhead conductors from aerial images taken by Unmanned Aerial Vehicles (UAV) is a necessary first step in assessing their condition. However, the detection method must reject other visually similar spurious lines such as other conductors, edges of a roof, or even gaps in the concrete slabs. Our proposed method makes use of a conductor's unique winding characteristic in the Fourier space to deliver at least a 98.2% conductor detection rate in our datasets. The method can also be generalized to other conductor types given their strand winding characteristics, and is reliable for those edge cases where suspicious false conductors are both close to and parallel with the target conductor. In these difficult, but surprisingly common scenarios, the conductor's unique frequency identity proves a resilient feature to use. This proposed conductor detection pipeline has been put in practical use by a commercial lines company.
This paper aims to analyse the effectiveness of deep learning models for the classification of microsatellite instability (MSI) in colorectal cancer (CRC) using whole slide images (WSIs). MSI tumours have complex and heterogeneous morphological characteristics, such as Tumor Infiltrating Lym-phocytes (TILs), mucinous differentiation, and medullary growth patterns. The complexity of these morphological characteristics makes automated classification challenging but crucial for aiding pathologists in identifying the presence of MSI in CRC. In the proposed method we develop and train multiple deep learning models based on the various ResNet architectures, to classify MSI and microsatellite stable (MSS) cases. We leverage image pre-processing techniques, including greyscale conversion, Contrast Limited Adaptive Histogram Equalization (CLAHE), and stain normalization, to enhance key MSI-related features in WSIs. The models are trained and evaluated on a New Zealand cohort of CRC cases, and the results demonstrate the potential of deep learning in improving diagnostic accuracy, while also highlighting the challenges posed by preprocessing and model selection. This work provides insights into the optimisation of deep learning approaches for MSI classification and suggests directions for future research.
The automation of fruit picking using robotic systems is rapidly advancing within precision agriculture, relying heavily on the accurate detection and segmentation of fruit objects through computer vision. This study explores the use of generative artificial intelligence (AI) to create extensive fruit image datasets, concentrating on cherries, bananas, oranges, apples, and pineapples. Five generative models-Stable Diffusion v1.4, Stable Diffusion v1.5, Stable Diffusion v2, Stable Diffusion XL Refiner 1.0, and SDXL Turbo-were employed to generate 1000 images per fruit type, each set against a white background. Thresholding techniques were used to automatically extract fruit boundaries, which were then utilized to train segmentation models. The performance of these models was evaluated based on their accuracy and processing speed. SDXL Turbo consistently delivered the highest accuracy across all fruit types, though it required more processing time per image. Stable Diffusion XL Refiner 1.0 also exhibited strong accuracy but balanced performance differently. In contrast, Stable Diffusion v2 showed significant shortcomings, particularly in producing accurate images for cherries and oranges. This comparative analysis highlights the potential of advanced generative models in enhancing synthetic dataset creation for fruit object detection and segmentation in agricultural robotics. Future research will focus on refining these models to improve accuracy, broaden their applicability to a wider variety of fruits and environments, and optimize the trade-off between image quality and generation
Deep learning is a cutting-edge methodology that has been extensively applied in real-world applications to solve computer vision tasks. Nonetheless, the inherent challenges of deep learning models lie in their black-box nature, rendering them opaque and hard to interpret. Recently, attention-based vision transformers have been introduced to overcome the black-box behaviour of deep learning models. Despite these advances, the decision-making process of the vision transformer is still challenging to interpret. Learning classifier systems is a state-of-the-art rule-based evolutionary machine learning technique that stands out for its ability to provide interpretable decisions. These systems generate niche-based solutions, require less memory, and can be trained using small data sets. We hypothesize integrating attention mechanisms into learning classifier systems, aiming to identify critical components in problem instances, link features to create simple patterns, and model hierarchical relationships in the data. The experimental results for binary-class image classification (cat and dog) tasks demonstrate that the novel system successfully ignores the irrelevant parts and pays attention to the salient features of cats and dogs. Crucially, the proposed system exhibits comparable performance accuracy to that of the state-of-the-art learning classifier systems.
This work introduces a new method for quantifying and sizing Brassica Oleracea instances via the use of unmanned aerial vehicle (UAV) imagery for instance segmentation. The goal of the project is to create a deep learning-based model that can recognize and classify Brassica Oleracea occurrences in images taken by unmanned aerial vehicles. The study uses the Mask R-CNN architecture and transfer learning strategies to train the model using a dataset of COCO-formatted annotated data and images of cabbage that was obtained from Mendeley Data. To improve resilience and variability, augmentation methods are used in dataset preparation. Evaluation criteria including mean Average Precision (mAP), precision, recall, and confidence levels are used to assess the model's performance. Findings show that the model performed very well, obtaining a remarkable mAP value of 98.6%. Moreover, inferencing test data shows how reliable the model is in identifying Brassica Oleracea occurrences in a variety of settings. The model continuously obtains high identification rates despite obstacles such as item incompleteness and illumination fluctuations, demonstrating its dependability and applicability for practical use in agricultural settings. The suggested strategy has the potential to advance crop management techniques and precision agriculture methods, enhancing agricultural sustainability and production.
