This paper presents features for quantitatively diagnosing the progression of gastric atrophy from stomach X-ray images, aiming to support mass screening for gastric cancer. Specifically, the features focus on the gastric areae shadow pattern, which appears as a mesh-like and low-contrast shadow that becomes prominent in cases of moderate to severe atrophy. In moderate atrophy, this pattern appears in the lower stomach region and gradually extends upward as the condition progresses. To capture this trend, the stomach area is divided into upper and lower regions, and the features are extracted by measuring the area occupied by the gastric areae shadow pattern in each region. An evaluation conducted on 157 gastric X-ray images demonstrated that the proposed features increased in correlation with atrophy progression and could effectively distinguish between normal and abnormal cases without reliance on specific classifiers.
Multi-Query Person Retrieval (mQPRS) improves security by enabling the retrieval of multiple individuals in various scenarios while providing real-time situational awareness on edge devices. This paper introduces mQPRS using soft biometrics on the NVIDIA Jetson ORIN AGX platform for the first time. It employs YOLOv8s for person detection and proposes the DensePAR model for recognising person attributes, alongside a new ranking strategy. A low-rank approximation is applied to the DensePAR model to manage computational limits on edge devices. An ablation study assesses the effects of model compression on performance using the GPU and Nvidia Jetson Orin platforms. The system achieves a 74
This work explores a state-of-the-art foundation vision model, DINOv3 ViT-B/16, to the domain of musculoskeletal (MSK) radiographs using self-supervised learning. Unlike prior work focused on supervised pipelines, we investigate whether and how domain-specific self-supervised adaptation reshapes feature representations to better support downstream anatomical region classification for registry construction. Leveraging a unified dataset of 88,303 plain radiographs across nine anatomical regions, we compare four modeling strategies, including supervised and frozen approaches, as well as self-supervised fine-tuning. Our results demonstrate that off-the-shelf DINOv3 features with a simple linear probe achieve 97.0
We present a novel framework for automatically evaluating building conditions nationwide in the United States by leveraging large language models (LLMs) and Google Street View (GSV) imagery. By fine-tuning Gemma 3 27B on a modest human-labeled dataset, our approach achieves strong alignment with human mean opinion scores (MOS), outperforming even individual raters relative to the MOS benchmark in terms of SRCC and PLCC. To enhance efficiency, we apply knowledge distillation, transferring the capability of Gemma 3 27B to a smaller Gemma 3 4B model, which attains comparable performance with a 3 × speedup. Further, we distill the knowledge into a CNN-based model (EfficientNetV2-M) and a transformer (SwinV2-B), delivering close performance while achieving a 30 × speed gain. Our framework offers a flexible and efficient solution for large-scale building condition assessment, enabling high accuracy with minimal human labeling effort.
The aurora borealis/australis is generated by the interaction between the solar wind and the Earth’s geomagnetic field. Auroral observations provide clues on the mechanisms that transfer energy from the solar wind into the terrestrial environment. However, these auroral displays, which are captured by ground-based optical instruments, are usually in the polar or arctic regions. Weather can obscure the optical instruments from capturing the auroral displays, which limits the number of days good optical data can be recorded. Recent advances in artificial intelligence have provided frameworks that can eliminate clouds on satellite images, for example. This study builds on these developments to address the problem of cloud removal in ground-based camera recordings with the goal of preserving the integrity of the auroral data. We train a deep learning model based on the U-net architecture to learn to reconstruct the aurora borealis from ground-based videos. To manufacture training data, we generate synthetic clouds using Perlin noise. Then, we combine an image with a clear sky with these synthetic clouds to generate a “cloudy” image. The synthetic clouds interlaced with an image frame of a clear sky appear similar to days with real clouds. The model is trained to remove the synthetic clouds. When given real cloudy data, the model effectively extracts auroral features from cloudy days: the filtered image on a real day features fewer clouds with a clearer and more prominent auroral structure when compared to the original image.
Video super-resolution (VSR) aims to restore high-resolution videos from low-resolution inputs, enhancing visual quality under real-world degradation. Existing diffusion-based VSR methods often rely on specially designed network architectures or text prompts as conditional inputs, which limits their flexibility and applicability—especially in scenarios where explicit text descriptions are unavailable. To tackle this challenge, we propose a Conditional Modeling ADapter that enables zero-modification reuse of pre-trained text-to-video diffusion models for the VSR task, called CMAD. The adapter transforms low-resolution video features into pseudo-text tokens via a pre-trained Vision Transformer (ViT) and a lightweight adapter module, allowing these features to be injected as encoder hidden states, which serve as the main conditioning interface within the diffusion model. Unlike previous methods that require redesigning or retraining task-specific backbone architectures, our design enables the model to interpret video inputs directly through its original language conditioning interface—without any modification to the pre-trained diffusion network. Experiments on several VSR benchmarks, including REDS, UDM10, and VID4, demonstrate that our method achieves reconstruction performance comparable to specialized super-resolution models, without text guidance. Moreover, CMAD provides a simple and efficient solution for adapting large-scale pre-trained diffusion models to video super-resolution tasks. The source code and models will be released.
While most acknowledge the many potential applications for augmented and virtual reality head-mounted displays, many people face visual discomfort while using these displays. In order to better understand the visual discomfort associated with optical see-through augmented reality, we investigated the differences in visual discomfort across two popular augmented reality devices (Magic Leap 2 and HoloLens 2) during perceived depth changes in four depths (.35 m, .75 m, 1.5 m, 4 m), comparing to real stimulus at those same depths. Utilizing a four-alternative forced-choice task, we found that users experienced a significant increase in visual discomfort when using the HoloLens 2 compared to the Magic Leap 2. When asked to compare the real and virtual conditions, participants significantly preferred the real condition and experienced higher visual discomfort in the virtual condition. Furthermore, when asked to subjectively order the depths from most fatiguing to least fatiguing, the nearest depth of 0.35 m was significantly more fatiguing than the other depths.
Traffic jams occur frequently and cause psychological, economic and environmental issues. Hence, understanding traffic congestion is important for both travelers and city planners. While dedicated sensors to detect road usage can provide accurate measurements for selected roads, they are expensive to set up and maintain. On the other hand, public traffic surveillance cameras are ubiquitous and can often be used to detect the current traffic flow and to allow the cities to improve traffic conditions. Many computer vision systems have to be trained for this purpose on large datasets. To support the development and evaluation of such systems, we introduce a new large-scale traffic surveillance video dataset. Recorded over a period of two months, the dataset comprises more than 17000 videos captured from 150 publicly available traffic cameras. It exhibits substantial diversity in both technical properties, e.g. resolution, frame rate, and compression artifacts, and visual characteristics, including camera perspectives and time-of-day variations. Unlike datasets collected in controlled settings, our dataset is captured using real-world traffic surveillance infrastructure, making it a realistic and challenging benchmark for advancing computer vision methods in traffic analysis. The dataset is available at https://www.nes.uni-due.de/research/data/
In recent years, a variety of computer vision methods have been proposed for hand pose estimation. The reported error rates on public benchmark datasets are often quite low, in the order of a few millimeters per keypoint. However, such accuracies are obtained by experimenting with specific datasets. In this paper, we pose the question of whether these accuracies are reliable estimates of performance over general real-world data, outside of the datasets that were used. We propose a method for assessing whether a test set has realistic or unrealistic similarities to a training set. In principle, test results are representative of real-world performance if the test cases are randomly and independently sampled from the space of all possible cases that the system could be applied to. Conversely, test results cannot be trusted as representative of real-world performance if the test images are unrealistically similar to the training set, more similar than we would expect truly random test cases to be. In our experiments, we find that such an unrealistic degree of similarity indeed exists in three public benchmark datasets, namely the NYU Hand Pose dataset, the MSRA dataset, and the BigHand dataset. We believe that the approach proposed in this paper can inform how to interpret the reported accuracies of state-of-the-art hand pose estimation systems, and can be a valuable guideline towards constructing less biased and more representative datasets in the future.
The process of studying plant cell traits has traditionally been conducted manually, requiring meticulous review to categorize and capture each separate cell trait of a plant. Our research aims to simplify and automate the categorization process by incorporating pre-trained deep learning-based models to detect and categorize plant cells in light micrographs obtained via an optical microscope. By manually labeling plant imaging data, we trained a Deep learning model based on YOLO model architecture to allow prediction on new plant stem micrographs. The model was trained to detect three categories of cells (vessels, fibers, and parenchyma) in the micrograph, achieving an overall precision of 0.74, a recall of 0.79, and an mAP50 of 0.77 in the test set. At the class level, vessels and fibers produced a high precision (mAP50 = 0.84 and 0.86, respectively), while parenchyma cells proved to be more challenging due to their smaller size and ambiguous boundaries, with mAP50 = 0.60 but a higher coverage of 0.80. These results highlight both the strengths and limitations of the approach, and suggest that future improvements in detecting subtle or rare cell types could further enhance performance. In summary, our results demonstrate the significant usefulness of automated cell detection and recognition in the field of hydraulic plant anatomy.
Audio description (AD) plays a crucial role in making video content accessible to visually impaired audiences, yet current approaches often rely on expensive supervised training or struggle to capture temporal and narrative consistency. We introduce a training-free framework that integrates vision–language models (VLMs) with large language models (LLMs) through three complementary mechanisms: semantic-constrained prompting to reduce irrelevant content, adaptive character reasoning for accurate entity grounding, and a memory structure that aligns fine-grained shot-level cues with longer scene-level context. This design allows the system to generate temporally coherent and context-aware AD without requiring additional training data. Evaluation on the MAD-eval-Named and TV-AD benchmarks demonstrates consistent improvements over state-of-the-art training-free methods, with gains in both lexical and semantic quality metrics.
Recent results on pose estimation from ellipsoid-ellipse correspondences, which can be readily obtained from an object detector, allow a direct computation of the camera pose from object-level correspondences. Unfortunately, standard bounding boxes (either horizontal or minimal enclosing boxes) are symmetric, which introduces an inherent ambiguity in the correspondence, yielding multiple or even infinite solutions. Furthermore, the current state of the art requires minimum two such correspondences to provide sufficient constraints for camera rotation. Our contributions make object-based pose estimation efficient in practice: First, a novel object detection method is proposed, called Directional Object Bounding Box (DOBB), which is capable of detecting the object’s own direction together with its minimal enclosing box (OBB), yet independently from it, which not only breaks the symmetry of OBBs, but also provides the necessary additional geometric information for our pose estimation method. Second, a novel object-based robust camera pose estimation pipeline is proposed where a minimal solution can be obtained from a single object for outlier filtering when vertical direction and the object orientation w.r.t. that axis are known; followed by a closed-form least squares solution for multiple inlier objects to compute the camera pose. Comparative tests confirm the state-of-the-art performance of the proposed DOBB-based pose estimation method on the standard KITTI360 and 7-Scenes datasets.
To promote occupational health management and prevent Computer Vision Syndrome among teleworkers, this paper presents a method for measuring screen-based computer work duration using a web camera. The existing method determines working states solely based on whether the PC user appears in the image, and therefore cannot accurately classify situations in which the user is taking a break while remaining at their desk. The proposed method captures an image of the user every minute and applies YOLOv8n-pose to detect the human body and facial landmarks such as the eyes and nose. Based on the size of the detected bounding box of human body and the face orientation calculated from the coordinates of the nose and eyes, a classifier determines whether the user is working or a screen break. The total work duration is calculated by counting consecutive one-minute intervals identified as “working”. To evaluate the proposed method, experiments were conducted on 21 users with a total of 3,790 webcam images. Experimental results confirmed that the proposed method was capable of accurately identifying the state of participants taking a break while remaining at their desks, and achieved a measurement error within 5 min for 12 participants with stable working postures. On the other hand, accurate classification was difficult in cases where participants adopted working postures not represented in the training data, or where only the gaze direction differed between working and on-break states. Consequently, 3 participants exhibited measurement errors exceeding 60 min.
Human pose estimation is a fundamental task in computer vision with numerous applications in human-machine interaction, activity recognition, monitoring, surveillance, security, animation, and augmented reality. While existing pose estimation models perform well on humans in upright, unobstructed positions, humans are frequently in horizontal positions with occlusions like blankets in real-world scenarios. Much research has been done in using infrared, depth, and pressure images for in-bed pose estimation, but the comparatively inexpensive pose estimation with RGB images remains unexplored. In this paper, we address the task of accurately estimating in-bed human poses with severe occlusions induced by blankets using only RGB images. To overcome the under-representation of such poses and occlusions in conventional pose estimation datasets, a synthetic dataset is generated and used in the training process. We demonstrate the effectiveness of our method by fine-tuning a pre-trained pose estimation model with our synthetic dataset and evaluating the results on a real-world in-bed pose estimation dataset. We also integrate a domain adaptation method in the training process of the pose estimation model and evaluate the resulting increase in accuracy.
360-degree surround view systems play a critical role in driver assistance across low-speed maneuvering scenarios. However, conventional rendering pipelines often suffer from artifacts such as the Manhattan effect, missing pixels, and non-photorealistic rendering, leading to diminished perception. To address these challenges, we propose a novel image synthesis pipeline leveraging generative modeling to enhance rendering quality. Surround-view RGBD images are fused into a point cloud and rendered from alternate viewpoints, formulating novel view synthesis as an image-to-image translation problem. A denoising diffusion model refines the generated views and suppresses rendering artifacts, producing outputs with significantly improved perceptual realism. Qualitative evaluations on diverse driving scenarios showcase the effectiveness of the proposed pipeline.
While transformer-based models dominate NLP and vision applications, their underlying mechanisms to map input space to label space semantically are not well understood. We study the sources of representation vulnerabilities in vision transformers (ViT), where perceptually identical images have very different representations and semantically unrelated images have identical representations. Our analysis reveals that imperceptible input changes result in significant representation changes, particularly in later layers, suggesting performance instabilities. Our comprehensive study shows adversarial effects, while subtle in early layers, propagate and amplify through the network, becoming most pronounced in middle to late layers. This insight motivates NeuroShield-ViT, a novel defense mechanism that strategically neutralizes vulnerable neurons in earlier layers to prevent the cascade of adversarial effects. We demonstrate NeuroShield-ViT’s effectiveness across various attacks, particularly against strong iterative attacks, and showcase its remarkable zero-shot generalization capabilities. Without fine-tuning, our method achieves 77.8
The generation of high-fidelity 3D point clouds is a critical yet challenging task, particularly for complex natural objects. This paper presents a systematic comparative analysis of five state-of-the-art generative architectures, including l-GANs, PointFlow, a Diffusion Probabilistic Model (DPM), and the autoregressive CanonicalVAE, on the specific task of synthesizing eucalyptus tree point clouds. To facilitate this investigation, we introduce Euca3D, a novel benchmark dataset of 440 real-world eucalyptus tree scans. Through rigorous quantitative and qualitative evaluation, our findings reveal that DPM and CanonicalVAE emerge as the most effective models. While DPM demonstrates superior performance in surface fidelity as measured by the Chamfer Distance, CanonicalVAE excels at preserving global structural integrity and perceptual quality, achieving the best results on Earth Mover’s Distance-based metrics. Visually, the point clouds generated by CanonicalVAE are of exceptional quality, nearly indistinguishable from the reference data. This study demonstrates that generating high-quality synthetic point cloud data of complex natural forms is achievable, establishing a strong foundation for data augmentation to enhance downstream tasks in forestry and computer vision.
The state of the art in hand pose estimation can benefit from larger, annotated datasets, covering ideally the full spectrum of hand poses. However, annotating keypoint locations in large numbers of hand images can be prohibitively time-consuming and expensive. We propose a new type of model for hand pose estimation that can be used to speed up manual annotations of hand images. The key idea is that, when the human annotator provides correct locations for some keypoints, that provides additional information (for which we use the term “partial annotation”) that can be exploited. An appropriately designed model can benefit from this additional information to produce more accurate estimates compared to the estimate that it produces just based on the image. In this paper, we propose such a model and demonstrate that using partial annotations indeed improves accuracy compared to estimates that are based solely on the hand image itself.
A crucial first step in the early detection and tracking of neurodevelopmental problems is the accurate segmentation of infant brain magnetic resonance imaging (MRI). However, the effectiveness of traditional segmentation techniques is limited by challenges such low tissue contrast, high anatomical variability, and tiny tissue volumes, especially in cerebrospinal fluid (CSF) and white matter (WM). To improve segmentation accuracy and focus on relevant anatomical characteristics, a convolutional neural network-based method in this paper that incorporates attention processes within a U-Net framework is proposed. Dice Similarity Coefficient (DSC), Modified Hausdorff Distance (MHD), and Average Surface Distance (ASD) were used to quantitatively assess WM, gray matter (GM), and CSF. With decreased MHD and ASD values across all tissue types, the proposed model surpassed a few cutting-edge techniques in boundary correctness and obtained competitive DSC scores (0.84 for WM, 0.92 for GM, and 0.86 for CSF). These findings show that the proposed approach offers a favourable trade-off between boundary localization and volumetric accuracy. According to the results, our method has a great deal of promise for use in clinical settings where accurate brain tissue segmentation is necessary for early infancy diagnosis and quantitative analysis.
Accurate segmentation of organoids in bright-field microscopy is essential for high-content screening in drug discovery and disease modeling, yet separating touching instances remains challenging. We present a training-free framework that combines Phase Congruency, an illumination-invariant image feature detector, with Persistent Homology, a topological method that extracts stable topological features such as H_1 cycles that correspond to organoid boundaries. Representative cycles derived from Phase Congruency responses provide interpretable separation contours that align close with true edge pixels, enabling robust separation of touching organoids without shape priors or supervised learning. Evaluated on a recent public dataset (OrgaSegment), our method achieves higher overlap which outperforms both deep-learning baselines and a previously proposed Phase Congruency-based segmentation method. Our results demonstrate that Phase Congruency combined with topological data analysis offers a highly accurate, interpretable, and generalizable strategy for organoid segmentation that does not require expensive data generation for training.
