Recently, online education and video-conferencing platforms have experienced substantial growth. Owing to the limited scope of virtual classrooms, it is difficult for instructors to assess learners’ attention and comprehension in real time during instruction. In digital learning environments, an automated feedback mechanism that informs instructors about learners’ attentiveness at any given time would be beneficial. To this end, we present a novel computer vision-based approach for analyzing learners’ affective states and quantifying their attentiveness. The foundation of our approach is a multioutput classification model trained on the publicly available DAiSEE dataset, designed to accurately identify learning-centered affective states such as boredom, engagement, confusion, and frustration. A key contribution of this work is the novel attentiveness index, which is a composite metric derived from the four affective states to quantify perceived attentiveness. Furthermore, we present a complete end-to-end pipeline that integrates this index into a practical, real-time system, processing learners’ live video feeds to provide instructors with detailed, actionable analytics. The robustness of this approach is validated by experimental outcomes in which our underlying models demonstrate state-of-the-art performance in affective state classification, thereby supporting the reliability of the entire system. The result is a comprehensive and practical real-time solution that is both deployable and easy to use, offering instructors an effective tool for gauging learner attentiveness in virtual classrooms.
Gaze estimation is essential for human–computer interaction applications. However, real-world conditions—including occlusions, illumination changes, and head pose variations—make robust estimation challenging, and most existing methods lack reliable uncertainty quantification. This paper presents AttentiveGaze, a multi-modal framework for robust gaze estimation that addresses critical challenges in real-world deployment, including adaptive feature fusion across heterogeneous visual cues and reliable uncertainty quantification. Our approach focuses on a task-specific integration of attention, multi-modal fusion, and uncertainty modeling tailored for gaze estimation. Specifically, the framework combines an attention-enhanced feature extractor to capture fine-grained spatial-channel dependencies in eye regions, and a multi-modal fusion mechanism with learnable gating and cross-modal attention to adaptively combine information from the face, eyes, and head pose. The fused representation is further refined through a multi-head embedding, enabling joint prediction of gaze coordinates along with associated uncertainty estimates. An uncertainty-aware regression head predicts gaze direction together with sample-wise confidence, improving robustness to ambiguous inputs. Evaluated on three public benchmark datasets (MPIIFaceGaze, EyeDiap, and GazeCapture), AttentiveGaze achieves competitive performance with a compact, real-time architecture. Furthermore, the proposed uncertainty modeling improves reliability in detecting out-of-distribution samples, highlighting its potential for safety-critical applications such as driver monitoring and clinical assessment.
As the need for efficiently storing and transmitting high-resolution digital images continues to increase, the development of compression algorithms that achieve substantial size reduction while preserving image quality becomes critical. This work presents two novel hybrid compression schemes that intelligently combine lossy and lossless techniques to optimize efficiency while preserving perceptual image quality. The first scheme, designated HLL1, merges Rounding Intensity Followed by Dividing (RIFD) with the Lossless Method of Decomposing Huffman Trees (LM-DH). The second, termed HLL2, pairs RIFD with the Lempel–Ziv-Welch (LZW) coding algorithm. Both approaches first reduce bit-depth via RIFD and then exploit the resulting statistical redundancies to generate compact encoded files. Experiments were carried out on a dataset of 20 standard test images encompassing 8-bit grayscale, 16-bit grayscale, and 24-bit RGB formats, with all processing performed in MATLAB R2021b. The results indicate that HLL1 yields an average improvement of compression ratio (ICR) of 0.20 over the RIFD + Huffman baseline, while HLL2 produces an outstanding 73
Knowledge-Guided Machine Learning (KGML) has emerged as an important paradigm for addressing the limitations of purely data-driven models by incorporating domain knowledge into the learning process. Although previous surveys have discussed KGML in broad terms, a comprehensive review dedicated specifically to computer vision remains lacking. To address this gap, this paper presents a systematic survey of KGML in computer vision based on a PRISMA-guided review of studies published between 2014 and 2025. From an initial pool of 2,788 publications, 286 high-quality studies were selected for detailed analysis. Based on this review, we propose a taxonomy that distinguishes between scientific knowledge and non-scientific knowledge, and further categorizes knowledge integration strategies into knowledge-guided learning, knowledge-guided architectures, and knowledge-guided pre-training. The survey shows that KGML has been applied across a wide range of computer vision tasks, including classification, object detection, segmentation, and multimodal understanding, with reported benefits in generalization, interpretability, and robustness. In addition, we identify key open challenges related to knowledge representation, fusion complexity, evaluation, and bias, and outline future research directions for advancing KGML in computer vision.
This paper proposes ShuffleNet-MSAA, a lightweight approach for sonar defect detection in underground drainage pipelines under turbid, high-water conditions. The proposed method addresses severe noise, small target-to-background ratios, and resource constraints. Built on the lightweight ShuffleNet backbone, ShuffleNet-MSAA integrates multi-scale feature extraction with pixel-wise adaptive spatial attention to improve defect detection and classification. The multi-scale feature extractor enhances performance on small targets and complex noise, while the adaptive attention module, implemented via median filtering, suppresses noise by dynamically weighting pixel contributions. To mitigate class imbalance and long-tailed data distributions in sonar image datasets, this paper introduces a dual-weighted focal loss (DW-FL), improving performance on imbalanced data. Experiments showed that ShuffleNet-MSAA outperformed state-of-the-art CNN- and Transformer-based baselines in both detection and classification performance. Results on both a custom dataset and a public sonar defect benchmark demonstrate high accuracy, robustness, and generalization, indicating strong potential for real-world applications.
Improving digital teacher training is a global imperative across educational fields. As Information and Communication Technologies (ICT) continue to reshape the educational landscape, it becomes essential for educators to embrace new pedagogical approaches. This research aims to investigate the application of virtual media and a non-traditional flipped learning method in a Compulsory Secondary Education (ESO) classroom. The study explores the advantages and drawbacks of this pedagogical approach, as well as the utilization of virtual reality (VR) resources within the educational domain. Specifically, the research combines the innovative teaching-learning method known as the Flipped Classroom with captivating digital resources, employing VR technology in the context of Geography and History subjects. The proposed methodology is developed in the second year of secondary education in the Fernando III high school (Priego de Córdoba, Spain). The research includes a comparative analysis of academic performance between traditional educational methodologies, the proposed Flipped Classroom (FC) and Virtual Reality resources approach. Compared with the first-quarter unit taught with the conventional approach, students obtained higher scores in the second-quarter unit implemented with the FC + VR methodology; given the single-group quasi-experimental design, these findings should be interpreted as observed differences in this sample rather than as definitive causal proof. This observation is supported by statistical evidence: the Wilcoxon signed-rank test revealed a significant difference between the two conditions: traditional and FC + VR evaluations (Z = − 3.039, p = 0.002), with a large effect size (r = 0.62), and excellence grades increased from 37.5
Breast cancer remains one of the leading causes of mortality among women worldwide, making accurate tumor segmentation across medical imaging modalities essential for early diagnosis and treatment planning. Although Swin Transformer–based U-Net architectures have shown promise, existing methods often lack effective mechanisms to exploit encoder–decoder interactions and multi-scale tumor characteristics. In this study, we propose a novel Swin Transformer–based U-Net architecture whose primary contribution lies in a Swin-Enhanced Cross Attention (SECA)-driven decoding strategy, rather than the use of a Swin encoder alone. The proposed framework employs a Swin Transformer encoder to capture long-range dependencies and global contextual information, while introducing SECA modules in the decoder to explicitly model cross-attention between encoder and decoder features, enabling precise alignment of global semantics with fine-grained spatial details. To further address tumor size and shape variability, a transformer-guided multi-scale decoder based on Inception modules is integrated. In addition, a hybrid loss function combining Dice, Focal, and Boundary losses is adopted to mitigate class imbalance, enhance learning from hard samples, and improve boundary delineation. The proposed model is extensively evaluated on four benchmark datasets, CBIS-DDSM, BreastDM, BCSS, and the Breast Ultrasound Images dataset, spanning mammography, DCE-MRI, histopathology, and ultrasound modalities. Experimental results demonstrate consistent performance gains over U-Net, U-Net++, and existing Swin U-Net variants, achieving improvements of up to +2.1
The integration of deep learning in healthcare and human-centered systems has enabled significant advancements in predictive analytics. However, concerns regarding interpretability, reliability, and safety remain critical barriers to adoption. This paper proposes SCLAF-FV (A SHAP-Guided CNN–LSTM–Attention Framework with Formal Verification for Reliable Injury Detection), a human-centric framework for injury risk prediction that combines deep learning, explainable artificial intelligence (XAI), and formal verification. SCLAF-FV integrates convolutional and recurrent neural networks with an attention mechanism to capture both spatial and temporal patterns in physiological signals. To enhance transparency, SHAP (SHapley Additive exPlanations) is employed to interpret model predictions and quantify feature importance. Furthermore, the Marabou formal verification framework is utilized to ensure that SCLAF-FV satisfies predefined safety and robustness constraints. Experiments conducted on the Ninapro dataset demonstrate that the proposed approach achieves superior predictive performance compared to baseline models, while also providing interpretable and verifiable outputs. This work contributes toward bridging the gap between high-performance machine learning models and their safe deployment in real-world scenarios.
H.265/HEVC(High Efficiency Video Coding) involves a lot of computational complexity, the main factor being recursive block partitioning and PU mode decision. In this paper, a novel online threshold model based square PU partition mode early detection scheme has been proposed, all 2 N square PU modes, including both the merging and AMVP modes, as well as the skip mode. Based on the statistical analysis of the RD cost of the best PU partition mode of the previously encoded CUs, we established a threshold model for early detection of CUs where the 2 N square PU partition mode is optimal. If less than a threshold, the evaluation of the non-square PU partition modes is entirely skipped and the square PU partition mode is determined optimally. Through experiments, the proposed method proved to be superior to previous methods in terms of negligible coding efficiency and quality degradation compared to HM, especially for video sequences with different motion characteristics.
Transcripts of lecture videos may have topic transitions that are subtle and gradual, making segmentation difficult for computers. Current segmentation methods face limitations: lexical cohesion methods rely on overlap between words on the surface level without capturing deep meaning, transformer methods break a single long-form transcript into chunks to impair coherence, and methods based on structure cannot handle the casual nature of a transcript from a lecture video. This work introduces the Thematic Segmentation of Lecture Video Transcripts, a four-component hybrid technique involving deep semantic encoding in a long-context semantics model, neural segmentation on multiple scales using contrastive learning, graph refinement, and reinforcement learning using quality-driven rewards. Our experiments on the Synthetic Lecture Transcript Benchmark (210 transcripts) as well as on several real-world lecture datasets demonstrate that this proposed method achieves F1 = 0.78 ± 0.01, Pk = 0.27 ± 0.01, WD = 0.25 ± 0.01, outperforming the evaluated baseline methods by an average of + 10.4
This study develops a multimedia training system for cardiopulmonary resuscitation (CPR) that couples HRNet-based pose estimation with ST-GCN + + skeleton action recognition to deliver real-time audio-visual feedback. The system segments streaming video into short clips, classifies CPR actions on the fly, and triggers process-aware prompts when the execution order or posture deviates from the standard procedure. This study is framed as a controlled feasibility study. The model was evaluated using randomized segment-level splits. The system achieves an action classification accuracy of 0.93, recall of 0.93, a balanced accuracy of 0.91, and a macro-averaged F1-score of 0.91. A quasi-experimental study was conducted with 60 participants divided into two groups of 30. The entire experiment was completed over approximately eight hours, including participant briefing, pre-test, instructor-led instruction, group-based practice, and post-test assessment. Simulated operation performance was evaluated by a blinded CPR instructor based on action sequence correctness and hesitation duration. The intelligent system group significantly outperforms the traditional training group in skill execution (mean score: 89 vs. 72, p < 0.001, rank-biserial r = 0.606), reports lower cognitive load, achieves a System Usability Scale score of 78.6, and expresses greater satisfaction with their training experience. The results suggest that skeleton-based action recognition with real-time multisensory feedback can support practical skill acquisition and learner confidence in a controlled feasibility setting. The proposed lightweight architecture is implemented on commodity hardware, suggesting its potential applicability to remote or resource-constrained settings and possible extension to other skill-based domains after further validation and runtime profiling.
Human Activity Recognition (HAR) using sensor-based multimodal time-series data from wearable devices has gained significant attention in recent years, primarily due to privacy concerns associated with vision-based approaches. Modern wearables such as smartphones, smartwatches, and smart glasses are equipped with inertial measurement unit (IMU) sensors that generate rich time-series signals widely used for HAR applications. Developing accurate and reliable HAR systems for recognizing Activities of Daily Living (ADLs) requires access to large-scale, diverse, and high-quality datasets. However, existing datasets often suffer from key limitations, including limited demographic coverage, low sampling rates, and small participant pools. This study makes two major contributions. First, we introduce HumCareADL, a novel and diverse multimodal HAR dataset specifically designed to address these limitations by capturing data from participants across varied age groups and activity contexts. Second, we propose a hybrid multi-branch CNN-LSTM architecture, termed HyMCL-Net, which leverages the complementary strengths of Convolutional Neural Networks (CNNs) for spatial feature extraction and Long Short-Term Memory (LSTM) networks for temporal dependency modeling. Extensive experiments demonstrate that HyMCL-Net achieves an average accuracy of 85.27
Content-based image retrieval (CBIR) relies on feature extraction techniques such as color, shape, and texture to retrieve visually similar images. Traditional CBIR methods often struggle to extract these different features, leading to unsatisfactory retrieval results. In this regard, we introduce a multimodal CBIR framework that combines handcrafted color histograms with deep texture and shape features obtained using a customized variant of the VGG16 network. To generate compact and comparable representations, the network is adapted by removing the fully connected layers and applying Global Average Pooling to the convolutional outputs. These descriptors are then grouped via unsupervised k-means clustering to facilitate more discriminative image grouping within the feature space. Our system is optimised using Nash game theory transforming our problem into a multi-criteria optimization process. Each feature extraction method is considered an independent player in a non-cooperative game, with each one proposing the cluster that contains images that are visually similar to the query image. The game follows an iterative process at each step, the intersection of all the similar images becomes the new dataset, and the same process is reapplied until a Nash equilibrium is reached, where no player can further improve the results.
The explosive growth of large-scale video archives from surveillance networks, online platforms, and personal devices has made efficient and semantically rich video retrieval a critical challenge. Existing approaches based on deep multimodal embeddings have significantly improved retrieval accuracy. However, they often lack scalability, modularity, and system-level integration with indexing and metadata management. In this work, we present a modular and scalable pipeline for semantic video indexing and retrieval, tailored to person-centric search. The proposed architecture decouples a web-based front-end from a back-end organized into two pipelines. The indexing pipeline performs video chunking, person detection and tracking, crop selection, metadata enrichment, and semantic vectorization. The retrieval pipeline supports visual, textual, and hybrid queries, including face-based matching. The system leverages YOLO11 and BoT-SORT for real-time person detection and tracking, SigLIP2 for multilingual vision-language embeddings, and InsightFace for face recognition, storing all representations in a vector database with rich, traceable metadata. We further fine-tune the SigLIP2-SO400M-Patch14-384 checkpoint on a curated mixture of person-centric image-text datasets and evaluate the resulting model on the RSTPReid benchmark. Experimental results show that our approach achieves state-of-the-art performance on the RSTPReid benchmark under the considered setting. In particular, it achieves competitive Recall@k performance with respect to recent text-based person search methods and significantly improves mean Average Precision, reaching 0.68 against a best competing value of 0.54.
Offline signature verification is a traditional method used to authenticate a person since ancient times. Though it has been studied for over 40 years, it remains a challenge for the research community due to two major issues: intra-person variation and inter-person similarity. To address these issues, it is crucial to determine stability-driven, consistent features that are stable, repeatable, and influential discriminatory features across genuine signature samples of the same signer. In this paper, a Multi-Phase Fusion Architecture is proposed, where fusion at multiple stages is employed to develop a reliable signature verification system. The primary aim of this work is to identify a compact set of consistent and stable discriminatory features from offline signature images that repeatedly appear across genuine signature samples of the same individual. The first phase of the proposed method combines global texture features with fine-grained texture features to effectively capture the discriminative characteristics of handwritten signatures. Proceeding with the second phase to determine the consistent features, multiple feature selection techniques encompassing both filter-based and wrapper-based methods were explored. More emphasis is placed on stability analysis, which evaluates the consistency and robustness of the selected features through stability metrics across varying datasets. Evaluating the consistency of features ensures that the model generalises well on unseen and challenging signature samples in real-world scenarios. In the third phase, an ensemble learning technique is used to achieve enhanced reliability in final decision-making. The proposed method is evaluated on multilingual signature datasets: CEDAR, BHSig260 (Hindi, Bengali), MCYT-75, and UTSig, achieving accuracies of 100
The classification of tomato leaf diseases is critical for advancing smart agriculture; however, deep learning model performance is often limited by low-data regimes. This study introduces and evaluates a Combined Data Augmentation (CDA) strategy that integrates traditional augmentation techniques (TDA) with data generated via a Stable Diffusion model (GDA) to improve classification performance under severely limited data conditions, specifically with only 20 training images per class. Experiments were conducted on subsets of the PlantVillage ( D^PV ) and PDR2018 ( D^PDR ) datasets, utilizing EfficientNet-B0 as the backbone across three training configurations: transfer learning with partial freezing, training from scratch, and fine-tuning all layers. The statistical significance of the results was assessed using the Wilcoxon signed-rank test with Holm correction and Cohen’s d_z effect sizes across 15 cross-validation folds. Findings indicate that Combined Data Augmentation (CDA) provides its most decisive benefit under the from-scratch configuration, rescuing the model from training collapse on both datasets ( D^PV : +21.95 percentage points, p_holm = 0.009; D^PDR : +7.91 percentage points). Under pretrained configurations, the improvement is modest on D^PV and absent or negative on D^PDR , where augmentation strategies including GDA were found to significantly reduce accuracy. Further analysis demonstrates that the effectiveness of generative augmentation is highly dependent on the Strength parameter; Strength = 0.35 is the only regime preserving performance parity with the Baseline, while higher values introduce semantic drift and statistically significant performance degradation. A learning rate sensitivity analysis at η _0 = 10^-3 confirmed that these conclusions are stable across hyperparameter choices. This study provides empirical and statistical evidence characterizing the conditions under which combined data enhancement strategies are beneficial, neutral, or detrimental in limited data contexts, and informs the development of robust plant disease diagnostic systems.
Skin cancer is a common and possibly deadly illness if it goes undetected in its early stages. Recently, progress in deep learning techniques has significantly improved the effectiveness of models used for classifying skin cancer. This work presents SkinEnsemNet, a transfer-learning-based heterogeneous ensemble framework that integrates VGG16, DenseNet201, and EfficientNetB0 for skin lesion classification using the HAM10000 and ISIC-2019 datasets. By combining feature representations from architecturally diverse CNN backbones through feature-level fusion and dataset-specific classification layers, the framework aims to exploit complementary spatial, hierarchical, and computationally efficient representations of dermoscopic images. The ensemble leverages the complementary strengths of these models: VGG16 captures localized spatial hierarchical features, DenseNet201 provides feature re-usage and hierarchical feature extraction, and EfficientNetB0 balances accuracy and computing cost. The preprocessing stage of the pipeline comprises normalization and class imbalance handling by random oversampling. Following feature extraction from the base models, global average pooling, feature concatenation, and classification using a stack of dense layers with dropout regularization are performed. On the held-out image-level test partitions, SkinEnsemNet achieved accuracies of 98.85
With the rapid expansion of digital media, ensuring authenticity and security has become fundamental. Audio Watermarking (AW) provides an effective solution to embed the information while maintaining imperceptibility and robustness. However, the prevailing studies overlooked the overlapping segments during watermark embedding, thus resulting in high computational overhead. Therefore, this article proposes a Robust Audio Watermarking System with Spatial-and-cross scaled gaussian error linear unit-based deep convolutional neural network and Blancmange curve cryptography for Security (RAWS-SBS) for attaining high security and robustness. Primarily, the FlatTop window (FT) technique segments the audio signal while ensuring non-overlapping segments. Afterward, by utilizing Polar Cartesian-based Discrete Wavelet Transform (PC-DWT), the segmented frames are decomposed; also, matrix formation is done. Then, the features are extracted. The Gamma Functional Energy Valley Optimization (GFEVO) determines the optimal location to embed the watermarks. Next, the matrix is transformed into a 2-D format. Thereafter, the watermark image is encrypted using the Blancmange Curve Cryptography (BCC) algorithm, and the encrypted bit is converted into a vector bit. Lastly, the audio watermarked signal is obtained. The Spatial-and-Cross with Scaled Gaussian Error Linear Unit-based Deep Convolutional Neural Network (SC-SGELU-DCNN) is employed to extract the watermarks. The experimental outcomes stated that the proposed method obtained a minimum Bit Error Rate (BER) of 0.03232. The GFEVO had 4.256 Perceptual Evaluation of Speech Quality (PESQ). Further, the proposed approach had a maximum accuracy of 97
Given the widespread impact of earthquakes, there is a pressing need for more effective methods capable of leveraging multimodal disaster data. This research proposes a multimodal prediction framework for classifying earthquake-related tweets. The proposed multimodal architecture first captures contextual dependencies in tweet text and fuses them with high-level semantic features learned by deep convolutional networks, resulting in a more discriminative joint representation that enhances the classification performance. The effectiveness of the proposed framework is tested using the CrisisMMD multimodal dataset. The dataset contains tweets collected during real-world natural disaster events. In thisstudy, we analyze earthquake related data and CrisisMMD which includes 2,743 text and image samples from two earthquakes data: the Mexico earthquake and the Iraq-Iran earthquake. The evaluation is based on performance metric, including accuracy, precision, recall, F1-score, and statistical significance scores. Comparison with deep learning models used in previous studies shows that the proposed framework is one of the high-performing investigations. The experimental results demonstrate that the proposed framework achieves an accuracy of approximately 88
Generative artificial intelligence (AI) provides a potential way to reduce the time and cost required for creating three-dimensional (3D) assets for cultural heritage virtual reality (VR) applications. However, its practical value depends not only on generation speed but also on whether the generated assets can be optimized, integrated, and deployed on standalone VR hardware while maintaining acceptable user experience. This study proposes a low-cost AI-to-VR content generation pipeline for a Taiwanese three-courtyard (Sanheyuan) cultural heritage tour. The pipeline combines open-source Trellis3D-based asset generation, Blender-based geometric and texture optimization, Unity OpenXR integration, and deployment on Meta Quest 3S. A total of 43 supplementary cultural objects were generated and optimized for static VR presentation. The measured object-generation time ranged from 25 to 135 s (M = 59.2 s, SD = 26.7 s), with kitchen and dining utensils showing the shortest average generation time (M = 46.6 s). The deployed system maintained a stable target frame rate of at least 72 FPS with interaction latency below 20 ms on Meta Quest 3S. A user evaluation with 50 valid questionnaire responses was conducted using a five-point Likert scale. The revised constructs showed acceptable to excellent internal consistency, with Cronbach’s alpha values ranging from 0.847 to 0.970. The construct means ranged from 3.72 to 3.87, indicating moderately positive perceptions of immersion, usability, comfort, usefulness, model realism, and system performance. The results suggest that open-source generative AI can support low-cost asset creation for cultural heritage VR, although post-processing, model-detail consistency, and broader participant validation remain necessary.
