A known flaw of statistical machine learning is the reliance on spurious correlations. Due to this, standard machine learning models often have degraded performance on minority sub-groups despite the presence of consistent causal signals. Group robustness solutions aim to address this by largely focusing on minimizing the expected error on the worst-performing minority group, often at the expense of overall performance. These state-of-the-art approaches also rely on accurate group annotations, which are difficult to obtain in practice, and makes the model's performance highly sensitive to label noise. In this work, we address these gaps with a simple yet effective solution: we show that integrating group variance minimization with the standard ERM objective is an upper bound on the worst-group loss, enabling a multi-objective optimization framework that improves both average and worst-group performance while effectively balancing their trade-off. We further propose a generalized cross entropy based multi-objective optimization training paradigm which demonstrates Pareto optimal performance under 10% noisy label conditions, and also maintains strong performance for both metrics across varying noise levels while state-of-the-art methods succeed at one metric but fail at another. We evaluate our method on standard benchmark - Waterbirds and CelebA datasets and demonstrate Pareto-optimal performance in both noise-free and noisy settings.
Colorectal cancer is a major cause of cancer mortality, with a significant proportion of polyps being missed during colonoscopy. To address this challenge, we propose EvoPolypNet, an automated polyp segmentation framework optimized through evolutionary neural architecture search (NAS). EvoPolypNet integrates convolutional feature extraction, attention mechanisms, multi-scale processing, and feature recalibration within an encoder-decoder architecture. The evolutionary NAS employs crossover, mutation, and elitism to explore diverse architectural configurations, guided by a defined search space. Evaluated on CVC-ClinicDB and Kvasir-SEG datasets, EvoPolypNet achieves superior performance with an mDice of 0.939 and mIoU of 0.898 on CVC-ClinicDB and demonstrates strong generalization on the unseen Kvasir-SEG dataset with an mDice of 0.919. The resulting model contains only 2.3 million parameters, enabling high segmentation accuracy with computational efficiency suitable for clinical deployment. Visual comparisons further validate the model's precision in delineating polyp boundaries. EvoPolyp-Net establishes a new benchmark for polyp segmentation by balancing performance, generalization, and efficiency.
Recent advances in immersive technologies have fueled efforts to bring real-world experiences into virtual environments, particularly in the art and performance domains. One emerging direction is the digital recreation of dance performances from a single monocular image. This task presents significant challenges due to the need for accurate 3D reconstruction, realistic geometry fusion, and photorealistic relighting under novel illumination. In this work, we propose a complete pipeline that reconstructs 3D dance scenes from a single image and enables real-time relighting in a browser-based renderer. The system combines monocular geometry estimation for the static stage and high-fidelity human mesh recovery for the performer. After mesh alignment and integration, the final scene is relit interactively using physically-based shading, environment map illumination, and dynamic directional lights, with realistic shadows computed via custom shader programming. The method operates fully on the client side, without requiring camera calibration or multi-view supervision. To assess the perceptual quality of our relit scenes, we conduct a user study focused on realism and material-light interaction. Results demonstrate that our approach generates compelling 3D representations of performance scenes, enabling realistic visualization under varied lighting conditions from minimal input.
Autism Spectrum Disorder ( ASD) often includes challenges in verbal communication, particularly for minimally verbal ( mv*) individuals. Mv* individuals with ASD often find it difficult to communicate with people since their vocalizations are highly personalized and often non-verbal, making it extremely difficult for people to understand their needs. Recent advances in machine learning and deep learning have made it possible for researchers to explore new avenues and provide personalized care for individuals with ASD. This research explores transfer learning and Mel spectrograms to classify non-verbal vocalizations from mv* ASD individuals, aiming to improve understanding and communication. We conduct three experiments to verify the robustness and generalizability of our approach in different setups. Using a dataset of 7077 vocalizations from eight mv* individuals collected from noisy, real-world settings, we convert the audio signals to Mel spectrograms and evaluate six transfer learning models and a custom convolutional neural network model. Our experiments demonstrated that DenseNet161, ResNet101, and VGG19, particularly when combined in an ensemble, effectively classify vocalizations into six categories: self-talk, frustration, delight, dysregulation, social, and request. Results suggest that this ensemble achieves state-of-the-art performance with an F1 score of 0.79, outperforming traditional machine learning methods and highlighting the potential of artificial intelligence in enhancing care for mv* ASD individuals.
We present Consistent3D, a training-free framework for high-fidelity 3D reconstruction from sparse views, leveraging point-cloud-guided video diffusion model to generate geometrically consistent novel views for 3D Gaussian Splatting. To address the severe multi-view inconsistency commonly observed in diffusion-based novel view synthesis, we introduce a geometry-guided pipeline. Specifically, we leverage the point cloud-estimated from sparse input images-as 3D priors to guide the video diffusion process, enabling both geometrically plausible and frame-consistent novel view synthesis. Furthermore, to achieve geometrically consistent depth, we introduce a Local Depth Alignment (LDA) strategy that adjusts monocular depth estimates into a scale-aware representation. This strategy is performed relative to the sparse input point cloud prior, resolving the inherent scale ambiguity in monocular depth prediction. We then propose a consistency evaluation module that computes 2D reprojection error and 3D depth discrepancy across views, yielding confidence scores. These views with high confidence scores are fused into a regularized 3D Gaussian Splatting (3DGS) pipeline, where parameters of Gaussians are optimized under confidenceweighted RGB and depth constraints. By enforcing 3D consistency through point-cloud-guided video diffusion model and a comprehensive confidence-weighted 3DGS optimization-which integrates depth alignment, confidence prediction, incremental fusion, soft constraint optimization, and real-time visual refinement-Consistent3D achieves high-fidelity and visually smooth 3DGS reconstruction from sparse observations without requiring model training. Our framework effectively bridges the generative power of diffusion models with the representational efficiency of 3DGS, delivering geometrically consistent and visually compelling 3D scenes.
Diabetic Foot Ulcers (DFUs) present a growing challenge in healthcare, often progressing silently until severe complications arise. Accurate classification of DFUs is vital, but challenging due to the subjectivity of manual assessments. This research proposes a novel deep learning-based hybrid architecture, ResViT with iPPM, designed to enhance the classification performance on the DFUC2021 dataset. The proposed model integrates ResNet-50 and Vision Transformer (ViT) as dual-path feature extractors to leverage both local texture patterns and global contextual information. These complementary features are then fused and passed through an improved Pyramid Pooling Module (iPPM), which captures multi-scale spatial information and enhances semantic refinement. The final classification is achieved using a fully connected layer. Extensive experiments demonstrate that our model achieves superior performance compared to existing baseline recall of 88.55%, an F1score of 89.93% and an accuracy of 88.43%. Additionally, GradCAM visualizations validate that the model effectively focuses on clinically relevant regions of the input images.
Timely and accurate detection of brain tumors from magnetic resonance imaging (MRI) is essential for effective clinical diagnosis and treatment planning and remains an active area of research in medical image analysis. In this study, we address two key challenges in this domain: the underrepresentation of data from low- and middle-income countries in existing research, and the need for efficient models suitable for deployment in real-world clinical settings. Specifically, we present: (1) the development and evaluation of a large-scale brain MRI dataset comprising 10,234 images collected from hospitals in Bangladesh and (2) the design of a lightweight convolutional neural network (CNN) for multiclass brain tumor classification. We performed 5-fold cross-validation to evaluate model performance. The proposed method achieved 95.28% accuracy, outperforming ResNet50 (83.31%), VGG16 (89.23%), VGG19 (90.36%), and MobileNetV2 (91.90%), demonstrating that our model achieves superior or comparable accuracy with substantially reduced computational complexity. The lightweight architecture enhances the model's suitability for use in environments with limited computing resources, such as pointof-care systems or mobile diagnostic tools. Grad-CAM-based visualizations were employed to highlight salient regions within MRI scans that contributed most to the model's predictions, providing interpretability and aiding clinical validation. Overall, the proposed framework represents a practical, scalable, and trustworthy solution for brain tumor diagnosis.
Answering questions about 3D environments is a frontier challenge in AI, demanding a nuanced grasp of spatial reasoning that distinguishes it from 2D VQA. The dominant paradigm for this task relies on pre-constructed scene graphs to explicitly model inter-object relationships. This approach, however, introduces architectural complexity and creates a dependency on intermediate representations that lack robustness. In this paper, we challenge this convention by introducing PointQA, a streamlined architecture that learns spatial and semantic context directly from raw point cloud data. Our model employs guided cross-modal attention to dynamically fuse visual and linguistic features, bypassing the need for explicit graph construction. Evaluated on the CLEVR3D-Real dataset, PointQA achieves state-of-the-art performance, outperforming prior methods that require more complex structural inputs. Our work demonstrates that judicious use of attention is sufficient for high-level 3D reasoning, paving the way for simpler and more scalable VQA models.
Few-shot cultivar classification (FS-CC) is a challenging task due to the need to distinguish subtle fine-grained (FG) variations using only a few labelled samples. This challenge is compounded by the fact that cultivar images often exhibit both shared visual patterns across categories and unique class-specific characteristics, making it difficult to learn discriminative representations, particularly under data scarcity. Existing methods struggle to capture these FG distinctions, often relying on globally extracted features that fail to distinguish subtle intra-class variations. To address this, we propose FDR-CultivarNet, a novel framework comprising two key modules: (1) the Feature Decomposition Network (FDN), which separates high-level features into shared (class-common) and unique (class-specific) components and (2) the Feature Reconstruction Network (FRN), which refines their separation and alignment and integrates them into a unified representation. Extensive experiments on four benchmark datasets— SoyCultivar200 for cultivar classification and three other FG datasets (Stanford Dogs, Stanford Cars, CUB200)—demonstrate that FDR-CultivarNet outperforms state-of-the-art methods.
Osteoporosis is a systemic disease characterized by low Bone Mineral Density (BMD) caused by an imbalance between bone formation and resorption. Gold standard BMD assessments rely on dual energy X-ray absorptiometry (DXA) scans of the hip, however uptake among eligible patients remains low, highlighting the need for alternative assessment methods. In this work, we propose a simple fusion strategy that combines information from distal radius $X$-rays with basic patient information (i.e. sex), and imaging acquisition parameters (i.e. machine settings) to predict femoral neck BMD, the primary metric for osteoporosis diagnosis. Rather than relying on complex or hard-toimplement methods, our approach is intentionally straightforward. A Vision Transformer (ViT) model processes the Xray image, and its features are directly fused with metadata using a lightweight design. This minimalist fusion strategy makes our solution easy to integrate into clinical workflows and robust against overfitting or unnecessary complexity. Among five tested models (DenseNet121, ResNet50, VGG16, ViT, and Swin), our simple ViT-based fusion model achieved the best results, with an MAE of 0.813, accuracy of 0.733, precision of 0.829, recall of 0.600, F1 score of 0.689, specificity of 0.865, NPV of 0.687, and AUC of 0.820. Our findings also show that including even basic metadata can boost performance. This study demonstrates that a simple fusion design, pairing image features with essential patient and acquisition data, can achieve promising performance and is well-suited for routine osteoporosis screening.
Large Multimodal Models (LMMs) have emerged as powerful general-purpose vision systems, yet their effectiveness compared to specialized tools remains unclear. We present a comprehensive evaluation of state-of-the-art LMMs across three paradigms: end-to-end inference, toolassisted prompting, and autonomous agent-based tool selection. Through our benchmark suite MMToolSet, we assessed leading LMMs on diverse vision tasks ranging from high-level reasoning (image-caption matching) to precise low-level operations (object counting, spatial grounding). Our findings reveal a nuanced landscape: while LMMs excel at semantic understanding tasks without external assistance, they struggle with spatial precision and systematic enumeration. Tool augmentation yields substantial improvements for these challenging tasks, reducing counting errors by up to 50% and enabling accurate object localization. However, success depends critically on model sophistication: advanced models like ChatGPT-4o demonstrate selective tool usage, invoking external assistance only when beneficial, while open-source alternatives often misapply tools, degrading performance. Notably, no model exhibits dynamic tool configuration capabilities, relying instead on default parameters even in suboptimal scenarios. These results suggest that the future of multimodal AI lies not in replacing specialized tools but in developing hybrid systems that seamlessly combine the generalization of LMMs with the precision of task-specific models. Data and code are available here https://github.com/TLink666/MMToolSet.
Renal conditions like chronic kidney disease (CKD) and renal cell carcinoma (RCC) pose major clinical challenges, particularly in early detection and precise anatomical analysis. While CT imaging is key for RCC diagnosis, manual interpretation is time-consuming and can vary between experts. This study introduces a novel Hybrid Segmentation Model that integrates Swin Transformer and VGGNet for multi-scale feature fusion, Graph Attention Networks (GAT) for spatial relationship modeling, and a transposed convolution decoder for full-resolution map reconstruction. Encoder features are divided into spatial patches and connected using a grid-like graph, enabling the GAT to learn anatomical dependencies effectively. A hybrid loss function combination of cross-entropy and Dice loss addresses class imbalance and enhances boundary precision. The model was evaluated on the KiTS23 dataset, achieving a competitive average Dice score of 89.86%, with 88.79% for masses and 87.90% for tumor outperforming existing works. These results demonstrate model's strong potential for delivering accurate, consistent, and anatomically meaningful segmentation outcomes.
Although AI promises to revolutionise manufacturing through the use of cameras, robots, and deep learning, numerous obstacles remain that prevent the industry from moving forward. These obstacles include the lack of understanding of sensor noise in high-precision metrology, of how to optimally process multiple sources of error, especially for multi-camera calibration, of real-time AI processes, and of the flexibility of robotic automation to be integrated into existing environments. To address these gaps, we introduce a new open toolkit called ROSELLA, comprising a hardware interface module, a 3D vision module, and a shape analysis module. These components enable us to rapidly develop robotic and computer vision applications based on established best practices. It offers extendable, opensourced, and commercial solutions to meet different needs. It promises to speed up our progress to realise the potential of Industry 4.0, and increase the productivity and resilience of the manufacturing industry.
Pixel-wise classification of hyperspectral images (HSI) remains challenging due to high spectral dimensionality, spatial complexity, and the presence of noisy labels. In this work, we address the pixel-wise classification problem in hyperspectral images, wherein each pixel is assigned a semantic class based on its spectral and spatial context. Although contrastive learning and attention-based models have shown promise, their reliance on clean labels limits their real-world applicability. To address these challenges, we propose a novel comprehensive feature learning with a triple contrastive learning framework that enhances classification robustness under weak supervision. Our approach integrates a Low-Rank Dimensionality Reduction (LoRDR) module to mitigate spectral redundancy while preserving discriminative features. Additionally, Noise-Adaptive Hybrid Attention (NAHA) refines spectral-spatial representations, ensuring model adaptability to label inconsistencies. A transformer encoder further captures long-range dependencies, enhancing feature generalisation. Extensive experiments on Kennedy Space Centre (KSC), ZY1-02D and GF-5 datasets demonstrate that our method consistently outperforms state-of-the-art models, achieving superior classification accuracy and stability under both symmetric and asymmetric noise conditions.
Recent advances in neural representations, such as Neural Radiance Fields (NeRF) and Neural Implicit Surfaces (NeuS), enable accurate 3D reconstruction of objects and scenes. While these methods achieve good results, they face significant challenges when dealing with small objects, such as insects, which contain intricate geometric details, including delicate wing patterns, antennae, and translucent surfaces, of high biological significance. In this paper, we propose Dynamic Neural Adaptive Sampling and Computation (DNASC), a transformative enhancement to Neural Implicit Surfaces (NeuS) and High-Frequency NeuS (HF-NeuS), optimized for complex insect morphologies. DNASC leverages thin structure scores, derived from edge detection and multi-scale variance analysis, to highlight detail-rich regions. We introduce a novel hybrid ray sampling strategy that ensures comprehensive scene coverage while focusing on critical areas, complemented by a score-weighted color loss that prioritizes high-detail zones. Also, we propose an adaptive score refinement mechanism that dynamically adjusts sampling based on reconstruction errors, driving iterative improvement of the geometry reconstruction. Evaluated on a custom insects datasets of three spieces, By building upon existing methods such as NeuS and HF-NeuS, the proposed DNASC achieves superior performance in various quantitative metrics (PSNR, SSIM, LPIPS, EPI) and qualitative mesh quality. This work elevates neural rendering for entomological applications, with profound implications for taxonomic studies and digital archiving. Code available at: https://github.com/a93088428/3d-insect-reconstruction
This research proposes a pipeline for intelligent medical prescription digitization. The solution is lightweight, efficient and integrates enhanced image pre-processing, hybrid OCR engines (PaliGemma and Tesseract) with domain- specific fine-tuning, and post-processing via Natural Language Processing (NLP) methods including spell checking and fuzzy matching against a curated medical dictionary. The system connects to an AI validation layer and a vector medical database to ensure output reliability and mitigate AI hallucinations. The proposed pipeline was evaluated on a dataset comprising 129 handwritten medical prescription images sourced from Kaggle, which included diverse hand-writing styles, noise levels, and irregular layouts to simu-late real-world scenarios. The evaluation results showed that the system achieved near-perfect extraction rates for printed prescriptions. For handwritten prescriptions, the hybrid OCR approach demonstrated a character-level accuracy of up to 90 % for block-style writing and an average of 52 % for cursive writing. The resulting structured text is ready for seamless integration with Electronic Health Record (EHR) systems and pharmacy platforms. This approach delivers accurate, scalable digitization of handwritten prescriptions while emphasizing generalizability and low computational requirements.
We propose a framework that extends explainable AI (XAI) from per-instance interpretation to dataset-wide knowledge discovery in dental morphology. A deep convolutional neural network was trained on 5,132 panoramic radiographs from 2,778 individuals to predict sex. Using OPG-SHAP, a domain-specific XAI method, we identified influential oral parts and validated them with statistical analysis. The upper canine was the most influential region, with females showing a significantly higher width-to-height ratio (0.391) than males (0.347), aligning with existing literature. Additionally, the upper third molar emerged as a novel sexually dimorphic feature, with males showing a higher ratio (1.064) than females (1.036). Both differences were statistically significant ($p<0.001$). Our results demonstrate how interpretable AI can rediscover known anatomical patterns and reveal new insights, enabling clinically meaningful knowledge extraction from neural networks.
Brain cancer is a serious problem causing a direct threat to human health, and efficient and early diagnosis is required for the improvement of the patient's prognosis. Magnetic Resonance Imaging (MRI) has been demonstrated as a star non-invasive modality for neurological studies (such as cancers). Recent progress in deep learning, specifically in convolutional and transformer-based architectures, has further advanced the ability to classify medical images with high accuracy. In this paper, we introduce and evaluate a new hybrid model called MedAttnNet, which combines both CNN and attention mechanisms for brain cancer classification. MedAttnNet is also compared with two state-of-the-art (SOTA) approaches, VGG-16 and baseline CNN; the former is a powerful visual feature extraction model based on deep CNN, and the latter is a new work focusing on hierarchical attention for a better understanding of context. All three models were trained and validated on a publicly available T1-weighted contrast-enhanced brain cancer MRI database containing three types of cancers. Data augmentation was used to make the models more robust. The experiments demonstrate that our proposed hybrid MedAttnNet model outperforms VGG-16 and a simple CNN, achieving an accuracy of 98.27% and exhibiting superior generalization ability. The strong performance of MedAttnNet underscores its potential for enabling earlier and more accurate brain cancer diagnosis, which is critical for timely intervention and improved patient outcomes. These results suggest that the proposed hybrid deep learning architecture offers a robust, effective, and scalable solution for medical image classification, particularly in the context of brain cancer detection.
Target Re-Identification (Re-ID) aims to identify the same target across multiple image or video scenes. In crossmodal target Re-ID, the challenge is to bridge the modality gap between observations from different sensors. We propose a new Hierarchical Cross-Modal Attention Network (HCMAN) for use in the cross-modal target Re-ID problem. The proposed HCMAN module uses convolutional self-attention to combine information across feature maps of different resolution. It is lightweight and can be easily integrated into the feature extractor of many available cross-modal target Re-ID architectures with negligible overhead. HCMAN requires no special handling, being optimized by end-to-end training of the Re-ID pipeline. We demonstrate improved performance of multiple state-of-the-art ReID methods through the integration of HCMAN. This improvement is observed across a variety of cross-modal target Re-ID datasets, including SYSU-MM01, RegDB, RGBNT201, MSVR310, and HITSZ-VCM.
Despite significant advancements in automatic Xray report generation, clinical adoption remains limited. A key barrier is the lack of structured clinical metrics-such as anatomical measurements-within generated reports, which hinders their practical utility for diagnosis. Current language-and vision-language-based approaches typically overlook these quantitative components due to their inherent learning constraints. In this work, we focus on cardiomegaly, a common thoracic condition where the cardiothoracic ratio (CTR)-a measurement derived from the width and height of the heart relative to the thorax-is an essential diagnostic metric. Unlike large language models (LLMs) or vision-language models (VLMs), which struggle to extract such precise geometric features, our approach explicitly incorporates CTR estimation into the report generation process. Our contributions are threefold: (1) we introduce a novel cardiomegaly-specific annotation set based on the MIMIC-CXR dataset, comprising 300 manually annotated bounding boxes of the heart and thoracic cavity; (2) we develop a YOLOv11-based cardiomegaly detector that achieves over 99% AP@50 performance; and (3) we integrate the CTR computation into a Vision Encoder Decoder (VED)-based report generation model, enabling more refined and clinically aligned radiology reports. Although ground truth CTR values are not available in the dataset for quantitative evaluation, our qualitative results demonstrate that incorporating CTR enables more accurate and interpretable severity descriptions of cardiomegaly. This highlights the effectiveness of incorporating explicit anatomical measurements in enhancing clinical usability.
