Object detectors often perform well in-distribution, yet degrade sharply on a different benchmark. We study cross-dataset object detection (CD-OD) through a lens of setting specificity. We group benchmarks into setting-agnostic datasets with diverse everyday scenes and setting-specific datasets tied to a narrow environment, and evaluate a standard detector family across all train--test pairs. This reveals a clear structure in CD-OD: transfer within the same setting type is relatively stable, while transfer across setting types drops substantially and is often asymmetric. The most severe breakdowns occur when transferring from specific sources to agnostic targets, and persist after open-label alignment, indicating that domain shift dominates in the hardest regimes. To disentangle domain shift from label mismatch, we compare closed-label transfer with an open-label protocol that maps predicted classes to the nearest target label using CLIP similarity. Open-label evaluation yields consistent but bounded gains, and many corrected cases correspond to semantic near-misses supported by the image evidence. Overall, we provide a principled characterization of CD-OD under setting specificity and practical guidance for evaluating detectors under distribution shift. Code will be released at \href{[https://github.com/Ritabrata04/cdod-icpr.git}{https://github.com/Ritabrata04/cdod-icpr}.
Few-shot Font Generation aims to generate stylistically consistent glyphs from a few reference glyphs. However, capturing complex font styles from a few exemplars remains challenging, and the existing methods often struggle to retain discernible local characteristics in generated samples. This paper introduces DRG-Font, a contrastive font generation strategy that learns complex glyph attributes by decomposing style and content embedding spaces. For optimal style supervision, the proposed architecture incorporates a Reference Selection (RS) Module to dynamically select the best style reference from an available pool of candidates. The network learns to decompose glyph attributes into style and shape priors through a Multi-scale Style Head Block (MSHB) and a Multi-scale Content Head Block (MCHB). For style adaptation, a Multi-Fusion Upsampling Block (MFUB) produces the target glyph by combining the reference style prior and target content prior. The proposed method demonstrates significant improvements over state-of-the-art approaches across multiple visual and analytical benchmarks.
Generating SVGs from text is a challenging vision task, requiring diverse yet realistic depictions of the seen as well as unseen entities. Existing research has been mostly limited to generating single-object rather than comprehensive scenes comprising multiple elements. In response, CraftSVG introduces an end-to-end framework for creating SVGs depicting entire scenes from a textual description. Utilizing a pre-trained LLM for layout generation from text via iterative in-context learning, CraftSVG introduces a per-box mask latent mechanism for accurate object placement. A fusion mechanism is developed to integrate the attention maps, employing a diffusion U-Net for coherent composition, which accelerates stroke initialization. Recognizing the importance of abstract SVGs in communication, we incorporated an MLP-based mechanism to simplify SVGs, with alignment and perceptual loss via differential rendering and opacity modulation to improve aesthetics. CraftSVG outperforms previous methods in abstraction, recognizability, and detail, as depicted by its CLIP-T: 0.5013, Aesthetic: 7.0779, score. The code is available at github.com/CraftSVG.
Text detection in natural scene images is gaining prominence due to real-world applications such as player identification in sports and runner identification in marathons. In these applications, the same scene is captured by multiple cameras, resulting in multiple views of the scene. For accurate text detection, the correct sequence (order) of views (Front, Left, Right, and Top) is important. Therefore, this work presents a new approach for detecting the proper sequence of views. We believe that changes in camera focus and angle are reflected in the edges across the views. To extract regions that provide important cues for identifying views, the proposed model detects patches based on edges and density to remove unwanted regions in the views. The patch vectors are used to classify them into focused and defocused regions using k-means clustering. Statistical features are extracted from the focused and defocused patches to identify the views and determine the correct sequence. Experimental evaluations demonstrate a sequence accuracy of 98.5
Continual Relation Extraction (CRE) has achieved significant success due to its ability to adapt to new relations without frequent retraining. However, existing methods still face challenges such as overfitting and representation bias. Inspired by the wake-sleep memory consolidation process of the human brain, this paper proposes a Wake-Sleep Memory Consolidation (WSMC) framework to address these issues systematically. During the wake phase, the model simulates the brain’s information processing mechanism, quickly encoding new relations and storing them in short-term memory. We also introduce the Experience Iterative Learning (EIL) approach, which dynamically adjusts the distribution of relation samples. This approach corrects the model’s representation bias and enhances memory stability through experience replay. During the sleep phase, the model consolidates existing knowledge by replaying long-term memory. Moreover, the framework generates diverse dream data from existing memory sets, thereby increasing the diversity of the training data and improving the model’s generalization capability. Experimental results show that WSMC significantly outperforms other CRE baseline methods on FewRel and TACRED datasets, demonstrating its superior performance compared to baseline methods. Our source code is available at https://github.com/Gyanis9/WSMC.git.
Non-contact estimation of peripheral blood oxygen saturation (SpO _2 ) from facial videos offers a promising alternative to traditional pulse oximetry, but current rPPG-based methods remain sensitive to motion, illumination changes, and inter-subject variability, and often rely on computationally heavy models. We propose a method, called FRETS, which is a lightweight Frequency-Enhanced Residual Transformer System that jointly models spatial, temporal, and spectral dynamics for robust SpO _2 estimation. FRETS employs an adaptive frequency separation module to decompose rPPG signals into DC and AC components, followed by an FFT-based Transformer enabling global spectral–temporal attention. A dual-stream DC/AC convolutional backbone captures physiological coupling between static appearance and pulsatile dynamics, while residual spectral fusion improves robustness to noise, motion artifacts, and lighting variation. From the experimental results (shown in Table 1) it can be seen that the proposed FRETS model outperforms most of the state-of-the-art methods in MAE, RMSE, and correlation, with particularly strong gains under challenging conditions on all three datasets (PURE, BH-rPPG, and VIPL-HR). From the experimental results, it can also be seen that FRETS is the most efficient and low-latency among all of the state-of-the-art methods, making it well-suited for real-time and edge deployment. Code Availability. The source code for this work is publicly available at: https://github.com/Surajit-M-00/FRETS .
Feature selection (FS) has gained considerable attention for mining discriminative features from high-dimensional data. Existing FS algorithms rank features with all training instances. However, the representative instances (RIs) that describe the common characteristics of most instances within the class, can properly rank the discriminative ability of features on dataset than the outliers. In this paper, we propose a novel strategy, namely RI selection for FS methods (RIS-FS). In order to reasonably identify the RIs, quantitative formulations based on the modified 3a principle are proposed. Theoretical analyses are also given to support RIS-FS. Extensive experiments with different groups of state-of-the-art FS methods on multi-type benchmark datasets and the ablation study show the effectiveness and rationality of RIS-FS.
In Document Understanding, the challenge of reconstructing damaged, occluded, or incomplete text remains a critical yet unexplored problem. Subsequent document understanding tasks can benefit from a document reconstruction process. In response, this paper presents a novel unified pipeline combining state-of-the-art Optical Character Recognition (OCR), advanced image analysis, masked language modeling, and diffusion-based models to restore and reconstruct text while preserving visual integrity. We create a synthetic dataset of 30{,}078 degraded document images that simulates diverse document degradation scenarios, setting a benchmark for restoration tasks. Our pipeline detects and recognizes text, identifies degradation with an occlusion detector, and uses an inpainting model for semantically coherent reconstruction. A diffusion-based module seamlessly reintegrates text, matching font, size, and alignment. To evaluate restoration quality, we propose a Unified Context Similarity Metric (UCSM), incorporating edit, semantic, and length similarities with a contextual predictability measure that penalizes deviations when the correct text is contextually obvious. Our work advances document restoration, benefiting archival research and digital preservation while setting a new standard for text reconstruction. The OPRB dataset and code are available at \href{https://huggingface.co/datasets/kpurkayastha/OPRB}{Hugging Face} and \href{https://github.com/kunalpurkayastha/DocRevive}{Github} respectively.
Vision-language models must associate visual entities with textual attributes. Whether these associations or concept bindings remain stable when the language of the input changes is unexplored. We introduce M^2BIND, a benchmark varying the language of the context and query across multiple languages. We evaluate binding both extrinsically through task performance metrics and intrinsically through causal interventions. We find that binding is not language-invariant: cross-family and cross-script settings trigger significant binding collapse, with the model's internal binding computation shifting to later layers and losing causal strength. Closely related languages preserve associations comparatively better. In a broader sense, our findings indicate how VLMs deployed globally in multilingual settings cannot be assumed to maintain the same association quality observed in monolingual evaluation.
Scene text detection in real-world environments often suffers from view-dependent failures, where text instances are visible in one view and are missed in other views due to perspective distortion, occlusion, and appearance variation. Although multi-view imagery provides complementary information, most existing methods process each view independently and lack explicit mechanisms to utilise cues in individual views for detecting text in multi-view scenarios accurately. In this paper, we propose a diffusion-based method for robust multi-view scene text detection in real-world environments. Given multiple views of the same scene, we first identify text instances that are detected in some views but absent in others through cross-view association. We then propose a diffusion-based recovery framework that reconstructs missing text bounding boxes by leveraging geometric consistency across supporting views. Missing detections are modelled as latent variables conditioned on multi-view context, enabling principled recovery rather than heuristic fusion. We design a detector-agnostic, multi-stage pipeline consisting of frozen single-view text detection, cross-view instance association, base-box generation from detected views, and residual diffusion-based refinement. The diffusion model learns to denoise residual offsets between recovered base boxes and ground-truth boxes, allowing accurate localization without retraining the underlying detector. Experiments on multi-view street-level scenes demonstrate that the proposed approach recovers a large fraction of text instances missed by single-view detectors, yielding substantial improvements in missing-view recall. These results highlight the effectiveness of diffusion models for robust multi-view scene text detection in real-world environments. The proposed method is also tested on benchmark datasets, namely, ICDAR 2015, Total-Text and CTW1500, to demonstrate the effectiveness of the method. The dataset is available at MVSD.
Remote photoplethysmography (rPPG) allows for the contactless estimation of physiological signals from facial videos by analyzing subtle skin color changes. However, rPPG signals are extremely susceptible to illumination changes, motion, shadows, and specular reflections, resulting in low-quality signals in unconstrained environments. To overcome these issues, we present a Reliability-Aware Weighted Multi-Scale Spatio-Temporal (WMST) map that models pixel reliability through the suppression of environmental noises. These noises are modeled using different weighting strategies to focus on more physiologically valid areas. Leveraging the WMST map, we develop an SSL contrastive learning approach based on Swin-Unet, where positive pairs are generated from conventional rPPG signals and temporally expanded WMST maps. Moreover, we introduce a new High-High-High (HHH) wavelet map as a negative example that maintains motion and structural details while filtering out physiological information. Here, our aim is to estimate heart rate (HR), and the experiments on public rPPG benchmarks show that our approach enhances motion and illumination robustness with lower HR estimation error and higher Pearson correlation than existing Self-Supervised Learning (SSL) based rPPG methods.
Text spotting in natural scene images has achieved remarkable progress. However, it remains a significant challenge due to the vast diversity of text in the wild, including irregular shapes, mixed styles, occlusions, etc. This paper presents a novel model called Contextual-Based Pattern Aware Graph Neural Network (CPA-GNN) for text spotting in diverse situations. In contrast to existing models, which often employ a single, one-size-fits-all pipeline, the proposed model introduces contextual reasoning for text detection and pattern discovery for recognition. For text detection, the proposed model introduces contextual reasoning by constructing a graph neural network for text, object, and background proposals, unlike existing methods, which mostly use textual features. This step refines detection confidence and boundaries and discovers missing text. For recognition, the proposed model introduces a pattern-aware approach implemented through a GNN as a “team of specialists” architecture. It operates on a segmentation-free graph of feature patches and dynamically identifies text style (printed, curved, handwritten). This identification then weights the outputs of multiple specialist GNN branches, each trained to excel at a specific text pattern. Our method achieves the best accuracy on challenging benchmark datasets, including Total-Text, CDW, and ICDAR 2015, proving that CPA-GNN is an effective and robust approach compared to the existing models. Source Code: https://github.com/Gamer0198/CPA-GNN-Model .
Handwritten text recognition continues to be a dynamic area of research, with practical uses such as processing bank checks, digitizing historical manuscripts, enabling handwriting-based user interfaces, and other OCR-related applications. Despite its potential, the task remains difficult because of a wide variety of handwriting styles, stroke patterns, and visual structures found across different writing systems. This research provides a comprehensive comparative study of deep convolutional neural network (CNN) architectures for handwritten character and word recognition of world popular scripts such as Roman (English), Devanagari, Bengali, Tamil, Telugu, Hiragana, and Arabic. Most recent and popular works on CNNs are considered. We conduct comprehensive benchmarking of widely used CNN architectures—such as VGG, ResNet, and Inception—on handwritten datasets spanning multiple scripts. Our experimental findings yield important information regarding comparative performances under diverse conditions, in addition to insights regarding the impact due to architectural extensions, i.e., attention mechanisms and regularization schemes, to recognition performance. Ensemble schemes, i.e., majority voting and stacking to obtain additional boost in performance, yield measurable increments in prediction faithfulness. Our investigation encompasses all training, validation, and testing stages and discovers key patterns such as overfitting tendencies, specifically for scripts with high visual complexity. These observations emphasize careful model selection and provide practical suggestions regarding designing robust, script-aware CNNs for multilingual handwritten text recognition. Received: 30 April 2024 | Revised: 26 March 2025 | Accepted: 25 June 2025 Conflicts of Interest Umapada Pal is an Advisory Board Member for Artificial Intelligence and Applications, and he was not involved in the editorial review or the decision to publish this article. The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in Ekush (Bengali) at https://doi.org/10.1007/978-981-13-9187-3_14; in the Devanagari handwritten character Database at https://archive.ics.uci.edu/dataset/389; in Kuzushiji-49 at https://github.com/rois-codh/kmnist; in the Telugu handwritten character database at https://www.pseudoaj.com/2016/05/pseudoajdataset0-telugu-handwritten.html; in EMNIST at https://www.nist.gov/itl/products-and-services/emnist-dataset; in the Arabic handwritten character dataset (AHCD) at https://github.com/AmrHendy/Arabic-Handwritten-Images-Recognition; in the Tamil handwritten word database at https://doi.org/10.1109/ICDAR.2013.162; in the English, Hindi, and Bengali handwritten word dataset at http://dx.doi.org/10.1109/ICFHR.2012.238; and in the Arabic handwritten word dataset at http://www.ifnenit.com/. Author Contribution Statement Sabyasachi Mazumder: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Sayan Neogy: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization. Sahana Das: Conceptualization, Methodology, Formal analysis, Writing – review & editing, Supervision. Kaushik Roy: Conceptualization, Methodology, Formal analysis, Resources, Writing – review & editing, Visualization, Supervision, Project administration. Umapada Pal: Conceptualization, Writing – review & editing, Supervision, Project administration.
Evaluating descriptive answers is complex and challenging due to the multiple answers provided by students. Although there are approaches for solving the problem of descriptive answer grading, the focus of such methods is on short answers rather than long answers. This work proposes a new model based on estimating similarity scores between question-answers and schema and answers. The proposed work asserts that the schema provides the right words and sentences, considered references for grading descriptive answers. Similarly, the question words also provide vital information about the right answers. We explore the SentenceTransformer model and cosine distance measure to estimate the similarity score, which results in feature vectors. The SentenceTransformer is proposed for estimating semantic similarity between two sentences, while cosine similarity is proposed for estimating the degree of similarity between two words. Inspired by the success of the 1D convolutional neural network in classification, we adopted the 1D-CNN with fully connected dense layers for grading answers by feeding the feature vectors. This CNN model classifies descriptive answers into three categories: A (right answer), B (partially correct answer), and C (wrong answer). To demonstrate the significance of the proposed method, we conducted experiments on our dataset and compared its performance with state-of-the-art methods. The results show that the proposed model outperforms the existing methods in terms of the confusion matrix and average classification rate.
Personality traits prediction plays a significant role in several real-world applications, such as improving the education system, improving production in manufacturing, monitoring social media content, sentiment analysis of crowds for opinion mining, judging abnormalities in personal behavior, etc. The demand for personality traits prediction has increased drastically after COVID-19. Therefore, numerous methods have emerged to predict personality traits from a variety of sources, including handwriting, interviews, social media, text, images, and audio. This review focuses on methods developed between the years 2020 and 2024, categorizing them into Handwriting (Graphology), Vision (images and videos), Audio (speech and acoustic signals), Textual (status updates, descriptions), and Multimodal (combinations of the aforementioned) approaches. We critically analyze the methods proposed, the datasets, the scope of the work, the results obtained, and noteworthy remarks. Based on our critical analysis, we notice that increasingly methods tend to use deep learning over handcrafted features. Additionally, personality traits prediction methods are trending more toward multimodal methods because they consistently achieve the highest accuracy among the input modalities. Detailed discussions, tabular presentations, and figures facilitate easy comprehension and future reference. Then, we shed light on the challenges in this field. Many key applications are detailed. Additionally, we highlight significant limitations and offer insights into potential future directions.
This paper presents a summary of the 2025 Sclera Segmentation Benchmarking Competition (SSBC), which focused on the development of privacy-preserving sclera-segmentation models trained using synthetically generated ocular images. The goal of the competition was to evaluate how well models trained on synthetic data perform in comparison to those trained on real-world datasets. The competition featured two tracks: (i) one relying solely on synthetic data for model development, and (ii) one combining/mixing synthetic with (a limited amount of) real-world data. A total of nine research groups submitted diverse segmentation models, employing a variety of architectural designs, including transformer-based solutions, lightweight models, and segmentation networks guided by generative frameworks. Experiments were conducted across three evaluation datasets containing both synthetic and real-world images, collected under diverse conditions. Results show that models trained entirely on synthetic data can achieve competitive performance, particularly when dedicated training strategies are employed, as evidenced by the top performing models that achieved F_1 scores of over 0.8 in the synthetic data track. Moreover, performance gains in the mixed track were often driven more by methodological choices rather than by the inclusion of real data, highlighting the promise of synthetic data for privacy-aware biometric development. The code and data for the competition is available at: https://github.com/dariant/SSBC_2025.
Deep learning based diagnostic AI systems based on medical images are starting to provide similar performance as human experts. However, these data-hungry complex systems are inherently black boxes and therefore slow to be adopted for high-risk applications like healthcare. This problem of lack of transparency is exacerbated in the case of recent large foundation models, which are trained in a self-supervised manner on millions of data points to provide robust generalisation across a range of downstream tasks. The embeddings generated from them happen through a process that is not interpretable, and hence not easily trustable for clinical applications. To address this timely issue, we deploy conformal analysis to quantify the predictive uncertainty of a vision transformer (ViT)-based foundation model across patient demographics with respect to sex, age, and ethnicity for the task of skin lesion classification using several public benchmark datasets. The significant advantage of this method is that conformal analysis is method independent, and it not only provides a coverage guarantee at the population level but also provides an uncertainty score for each individual. This is used to demonstrate the effectiveness of utilizing these embeddings for specialized tasks like diagnostic classification, meanwhile reducing computational costs. Secondly, the public benchmark datasets we used had severe class imbalance in terms of the number of samples in different classes. We used a model-agnostic dynamic F1-score-based sampling during model training, which helped to stabilize the class imbalance. We investigate the effects on uncertainty quantification (UQ) with or without this bias mitigation step. Thus, our results show how this can be used as a fairness metric to evaluate the robustness of the feature embeddings of the foundation model (Google DermFoundation), advancing the trustworthiness and fairness of clinical AI.
Script identification is challenging because of the unpredictable nature of the scene text. This paper presents a new model for achieving accurate script identification irrespective of intra and inter-class variations. The distinct features that represent the scene text of different scripts uniquely are extracted by fusing inception, which captures multi-scale features, and dense network, which captures fine-grained features. To strengthen the feature extraction, the proposed work uses wavelet decomposition, which enhances the fine details like edges in the images. Furthermore, for extracting text style, we propose a soft style attention module, which captures the unique style of scene text. The above modules are integrated as a hybrid model for accurate script identification. To evaluate the proposed model, we conducted comprehensive experiments on benchmark datasets, namely CVSI2015, SIW-13, and MLe2e, and combined datasets (combining distinct classes of all three benchmark datasets). The results of the proposed model on different datasets show that the performance is superior to the state-of-the-art methods in terms of accuracy Received: 6 June 2024 | Revised: 12 November 2024 | Accepted: 31 December 2024 Conflicts of Interest Palaiahnakote Shivakumara is the Editor-in-Chief and Umapada Pal is an Advisory Board Member for Artificial Intelligence and Applications, and were not involved in the editorial review or the decision to publish this article. The authors declare that they have no conflicts of interest to this work. Data Availability Statement The data that support the findings of this study are openly available in CVSI 2025 at https://www.ict.griffith.edu.au/cvsi2015/Dataset.php, in GitHub at https://github.com/lluisgomez/scri pt_identification, and in Kaggle at https://www.kaggle.com/datase ts/ayush02102001/cvsi-script-identification-dataset. Author Contribution Statement Shivakumara Palaiahankote: Methodology, Writing – original draft. Umapada Pal: Writing – review & editing, Visualization, Supervision. Taha Mansouri: Validation, Resources.
Visual Question Answering (VQA) is one of the attractive topics in the field of multimedia, affective, and empathic computing to garner user interest. Unlike existing models which aim at addressing challenges of VQA for the scene images, this work aims at developing a new model for Personality Trait Question Answering (PQA). It uses Twitter account information, which includes shared images, profile pictures, banners, text in the images, and descriptions of the images. Motivated by the accomplishments of the transformer, for encoding visual features of the images, a new InfoGain Multi-Axial Wavelet Vision Transformer (IgMaWaViT) is explored here. For encoding textual features in the images and descriptions, a new Information Gain BERT (InfoBert) method is introduced, which can handle the variable length encoding of text by choosing the optimal discriminator. Furthermore, the model fuses encodings of images and text according to the questions on different personality traits for question answering. The model is called InfoGain Multi-Axial Wavelet Vision Transformer for Personality Traits Question Answering (IgMaWaViT-PQA). To validate the efficacy of the proposed model, a dataset has been constructed, and it is used along with standard datasets for experimentation. Comprehensive experiments show that the proposed model is better than the state-of-the-art models. The code is available at the link: https://github.com/biswaskunal29/InfoGain_MultiAxial_PQA .
Josep Lladós合作论文数Computer Vision Center - Computer Science Department50
Apostolos Antonacopoulos合作论文数University of Salford9