Learned image compression (LIC) achieves state-of-the-art compression performance, yet practical deployments often overlook latency and energy—critical constraints for on-device pipelines and bandwidth-limited applications such as drones and satellite imaging. We address this gap with a codec-aware heterogeneous deployment partitioned at the codec’s natural bottleneck—the quantised latent—and mapped across devices: large analysis/synthesis transforms to the GPU (high parallel throughput), the compact, pipelined hyperprior to the FPGA (low-power dataflow), and entropy coding to the CPU. The system remains bit-exact with the reference codec and integrates directly with pretrained CompressAI models via exported ONNX subgraphs. Across two pretrained codecs and 1,000 test images, the heterogeneous mapping improves encoder runtime by about 10% and reduces encoder energy by 80% on average relative to GPU-only execution. For the decoder, runtime remains within 1% of the GPU while energy decreases by approximately 78%. Overall, it achieves an energy–delay product about 80% lower than GPU-only and 29% lower than FPGA-only, providing a superior latency–energy trade-off for LIC deployment.
This paper presents a watermarking assessment in the context of the emerging JPEG Trust Part 3: Media Asset Watermarking (ISO/IEC 21617-3). The specification complements the JPEG Trust family of standards by enabling a robust content–metadata binding through watermarking, while remaining algorithm-agnostic via explicit signalling of watermark presence, identification of the embedding/extraction technique, and references to external repositories that store Trust Records. It further describes a recommended payload structure and a benchmarking methodology covering imperceptibility and robustness, including modern attacks such as advanced compression (e.g., JPEG XL, JPEG AI) and AI-driven object manipulation. Four state-of-the-art methods from industry and academia (InvisMark, Watermark Anything, TrustMark, and VINE-R) are then evaluated on COCO dataset, highlighting strengths on classical perturbations and persistent vulnerabilities to emerging AI-based attacks, underscoring the need for standard and effective evaluation.
Computer vision algorithms, specifically convolutional neural networks (CNNs) and feature extraction algorithms, have become increasingly pervasive in many vision tasks. As algorithm complexity grows, it raises computational and memory requirements, which poses a challenge to embedded vision systems with limited resources. Heterogeneous architectures have recently gained momentum as a new path forward for energy efficiency and faster computation, as they allow for the effective utilisation of various processing units, such as Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Field Programmable Gate Array (FPGA), which are tightly integrated into a single platform to enhance system performance. However, partitioning algorithms over each accelerator requires careful consideration of hardware limitations and scheduling. We propose two low-high power heterogeneous systems and a method of partitioning CNNs and a feature extraction algorithm (SIFT) onto the hardware. We benchmark feature detection and image classification algorithms on heterogeneous systems and their discrete accelerator counterparts. We demonstrate that both systems outperform FPGA/GPU-only accelerators. Experimental results show that for the SIFT algorithm, there is 18% runtime improvement over the GPU. In the case of MobilenetV2 and ResNet18 networks, the high power system achieves 17.75%/5.55% runtime and 6.25%/2.08% energy improvements respectively, against their discrete counterparts. The low-power system achieves 6.32%/16.21% runtime and 7.32%/3.27% energy savings. The results show that effective partitioning and scheduling of imaging algorithms on heterogeneous systems is a step towards better efficiency over traditional FPGA/GPU-only accelerators.
As the demand for satellite imagery increases, efficient data processing at the sensor level is essential to overcome bandwidth and latency limitations. This study investigates the use of AI-driven atmospheric cloud segmentation directly onboard satellites to optimise sensor data utilisation, reduce transmission loads, and enable autonomy in Earth observation missions. Various deep learning models were tested on embedded hardware to assess their feasibility under power and computational constraints. While some models achieved high accuracy, their resource demands made them impractical for real-time deployment. A lightweight approach, particularly using optimized versions of CloudNet, demonstrated an effective balance between efficiency and performance. The research highlights the impact of model selection and optimization techniques in enabling real-time cloud detection on small / cube satellites, offering insights into practical onboard AI implementation and providing meaningful guidance on selecting models best suited for different satellite applications based on hardware limitations.
Content generation with generative AI has become a common practice in recent years. Manipulated images have become widespread today due to the ease with which they can be modified using sophisticated tools, which is a significant challenge. AI-driven visual content creation enhances creativity and efficiency. However, today, it is also one of the considerable sources of misinformation, hate crimes, counterfeiting, fraud, and manipulated content. Therefore, there is an urgent need for robust detection and verification mechanisms. Traditional image manipulation detection methods often focus on either image features or metadata analysis. Both have limitations and alone are insufficient against more advanced AI-based manipulation. We propose a novel framework that leverages the recent JPEG Trust international standard (ISO/IEC 21617-1) with deep learning-based detection and localisation techniques to address AI-manipulated image detection challenges. The proposed framework consists of two components: A) a component that enables users to record the provenance metadata about AI-powered image processing and support ethical use using the existing JPEG Trust standard and its extension, and B) a component that enables verification of the image's authenticity through detection tools. The proposed framework aims to improve the trustworthiness of AI-powered image processing activities within the media consumption chain as it provides a robust two-layer verification system that strengthens confidence in image authenticity. To demonstrate the capability of this framework, we describe the adoption of the framework for two case studies: 1) earth observation applications with satellite imagery and 2) digitised cultural heritage.
The proliferation of synthetic media generation technologies, such as generative AI, has led to a surge of media content generation and consumption. While this progress opens new opportunities, especially in creative industries, it also causes challenges, including piracy, fake media distribution, and concerns about trust and privacy. In the creative sector, media modifications are often part of the production pipelines and in many application domains, creators need or want to declare the type of modifications that were performed on the media asset. The cryptographically signed association of provenance information with the media asset itself provides a trust link between the owner or editor of a media asset and its consumers. The absence of such assertions may reveal the lack of trustworthiness in media assets or worse, the intention to hide the existence of manipulations. This paper describes the JPEG Trust framework (ISO/IEC 21617) that aims to establish trust in digital media creation, modification, annotation, distribution and consumption. The framework provides standardized protocols to extract indicators to assess trustworthiness, means to annotate media provenance, and securely link the assets and associated annotations together.
JPEG Trust is a new international standard that responds to the pressing need of assessing trust in digital media assets. JPEG Trust provides a comprehensive framework addressing key elements such as provenance, authenticity, integrity, and copyright declaration. Built on top of established JPEG and other industry standards, the framework ensures compatibility across digital media ecosystems. This paper provides an overview of the JPEG Trust framework and demonstrates its potential to enhance the reliability and trustworthiness of digital media content.
Food recognition plays a vital role in various domains including dietary monitoring, nutrition analysis and food service automation. Real-world food recognition is a challenging task, as the contents of a plate of food can be complex intermixed objects, making it difficult to define their individual structures. Currently, technology offers a wide range of feasible options for dietary assessment and image-based methods hold the capability of substituting for traditional methods such as food records, food frequency questionnaires, and 24-hour recalls, which can have low accuracy and provide unreliable results. Faced with these issues, deep learning methods have shown better accuracy and ability to identify ingredients and types of food compared to traditional approaches for image classification. However, many deep learning methods rely on powerful computational resources which have limitations in terms of cost, energy consumption, and size. The rapid evolution of embedded hardware systems has significantly influenced the domain of computer vision and offers promising solutions to these challenges. This paper presents a method that utilises deep-learning methods for detection and segmentation that are optimised for resource-constrained embedded platforms. These networks are tailored to efficiently process food images while ensuring low latency and energy efficiency. Additionally, strategies such as model quantisation, pruning, and compression are employed to reduce the computational complexity and memory footprint, making them suitable for deployment on embedded devices with limited resources, such as a Raspberry Pi. The method consists of a custom recognition pipeline that makes use of YOLOv81 and EdgeSAM2 approaches for detection and segmentation, which are trained on the foodSeg1033 dataset. The resulting system provides a fast, accurate way to recognise foods without requiring expensive, energy intensive hardware.
Water hyacinth (WH) is a notorious invasive species that significantly threatens ecosystems worldwide. Despite WH's well-documented threats and effects, its spatial distribution is not yet fully understood, especially in complex environments such as wetland systems. This knowledge gap is primarily due to the lack of accurate techniques with high spatial resolution and reliable in situ field data for quantification and monitoring. To address this research gap, we conducted a study to map the spatiotemporal distribution of invasive WH in Anzali International Wetland, Iran, using Sentinel-2 Multispectral Instrument 2022 data. Specifically, our study aimed to identify multispectral remote sensing variables and in situ field data using machine learning (ML) methods to detect and map WH growth cycles. In the first phase of our study, we compared three ML models for detecting WH and discriminating from other classes. Our results demonstrate that ML algorithms can detect WH accurately. In the second phase, we used four images dominated by four growth stages: early, mid, high, and decaying stages to train our ML classifier. We used the random forest algorithm for training our training samples achieving an overall classification accuracy of over 98%. These findings were further supported by statistical analysis, such as F1 (above 96%) and intersection over union (above 92%), indicating the high-performance quality of the used algorithm. Our study provides valuable insights into using ML algorithms for mapping WH growth cycles, which can significantly contribute to effectively managing and monitoring invasive species worldwide.
Acceleration of image segmentation using deep learning methods on satellite imagery has become ubiquitous in various applications areas such as land cover classification, disaster monitoring and vegetation detection. However, the increase in satellite image resolution and large data volume required for remote sensing applications has resulted in a substantial increase in computational resource usage and demand for real-time processing. This paper investigates quantum computing as a novel approach to meet these computational demands, exploiting its parallel processing strengths. We evaluate hybrid quantum models (COQCNN, MQCNN, FQCNN) against classical CNN and U-Net architectures in remote sensing classification. Although COQCNN and MQCNN underperformed, FQCNN reached 53.26% accuracy, outperforming the classical CNN by 8%. Despite quicker convergence, quantum models struggle with complex feature segmentation, a task where U-Net excels. This study highlights quantum convolutions as a potential path to enhance convergence while addressing challenges like noise from multiple quantum channels affecting accuracy.
Despite tremendous advancement in computer vision, especially with deep learning, understanding scenes in the wild remains challenging. Even modern image classification models often misclassify when presented with out-of-distribution inputs despite having been trained on tens of millions of images or more. Moreover, training modern deep-learning classifiers requires a lot of energy due to the need to iterate many times over the training set, constantly updating billions of model parameters. Owing to problems with generalisability and robustness as well as efficiency, there is growing interest in computer vision to mimic biological vision (e.g., human vision) in the hope that doing so will require fewer resources for training both in terms of energy and in terms of data sets while increasing robustness and generalisability. This paper proposes a biologically plausible neuromorphic vision system that is based on a spiking neural network and is evaluated on the classification of hand-written digits from the MNIST dataset. The experimental outcome indicates improved robustness of the proposed approach over state-of-the-art considering non-digit detection.
The proliferation of artificial intelligence (AI) has revolutionised various fields, including image processing and manipulation. While AI-driven image manipulation techniques such as DALL-E1 and Stable Diffusion2 offer unprecedented capabilities for creativity and visual enhancement, they also pose significant challenges in terms of authenticity, integrity, and misinformation. Advancements in AI Image modification have blurred the line between reality and fiction, raising concerns about the potential for misinformation, propaganda, and manipulation in various domains, including journalism, advertising, and social media. These techniques enable the creation of visually convincing yet falsified images which pose challenges for current state-of-the-art techniques such as early/late fusion3 which often struggle to discern subtle alterations made by AI algorithms and as such report poor detection results, necessitating the development of advanced detection methods capable of discerning AI manipulations. This paper presents a dataset of images containing AI generated modifications and a method for the detection of AI manipulations in images. The dataset consists of over 14,000 AI manipulated images and their ground truth masks which indicate where on the images the manipulations take place. The modified images were created using a state-of-the-art text-guided generation method4 that can generate modifications within an image from a text prompt. This dataset facilitates the development of a new deep-learning based image manipulation detection model which can reliably determine the existence of edited content in an image and localise the area that has been modified.
Deep Neural Network (DNN) algorithms have become ubiquitous within the vision domain, encompassing various tasks, including object detection, segmentation, and classification. However, executing complex DNNs in real-time systems demands improved energy efficiency, runtime, and accuracy. Traditional embedded imaging designs, typically implemented on homogeneous architectures, face hardware limitations, prompting the need for heterogeneous computing architectures. These architectures combine CPUs, GPUs, FPGAs, and other accelerators, enabling applications to use the most efficient architecture for a given algorithm. The challenge lies in scheduling and partitioning algorithms across accelerators with different computing paradigms and tool-sets. This requires balancing computational power, memory bandwidth, and communication overhead. Effective scheduling involves considering task dependencies, resource availability, and synchronisation. Current deep learning libraries often target single architectures and lack mechanisms to intelligently partition sub-operations across the most suitable processors. This paper introduces a scheduler for heterogeneous vision systems that finely partitions and maps sub-operations of convolutional neural networks and image processing algorithms. Leveraging state-of-the-art compiler frameworks such as PyTorch, TVM, and ONNX, the proposed scheduler optimally distributes tasks across heterogeneous components. Experimental results show that the heterogeneous platform achieves on average 1.12x & 1.08x improvements in kernel runtime and energy consumption, compared to the best-performing discrete hardware counterparts, GPU and FPGA. The study demonstrates that partitioning algorithms based on their runtime and energy properties and optimally scheduling them improves energy and runtime efficiency compared to homogeneous components executing the complete algorithm.
Measuring hyperplasia in Atlantic salmon gills can give important insight into fish health and environmental conditions such as water quality. This paper proposes a novel histology image classification technique to identify hyperplastic regions using an emerging signal decomposition technique, Empirical Wavelet Transform (EWT) in combination with a fully connected neural network (FCNN). Due to its adaptive nature, we hypothesise and show that EWT effectively represents unique features of gill histopathology whole slide images that help in the classification task. Our hybrid approach is unique and significantly outperformed regular deep learning-based methods considering a joint speed-accuracy metric.
Trusted distribution and consumption of media content has become a challenging issue, especially with the advancement of machine learning-based techniques such as deep fake. To address such challenges, this paper proposes a new metadata schema which is embedded within a larger framework that facilitates trusted media distribution. This schema is realised through a distributed media blockchain core in conjunction with algorithms to detect media modifications. Such a framework is expected to improve trust in media consumption, ensuring media integrity, authenticity and provenance.
G. J. Michaelson合作论文数 Computer Science
School of Mathematical & Computer Sciences
Heriot-Watt University2