Sign Language Generation (SLG) has received increasing attention in recent years, with various models aiming to produce natural and temporally coherent sign gestures from spoken or written language. However, SLG remains a challenging task due to its inherently one-to-many nature, where a single sentence can correspond to multiple valid gesture sequences, and the requirement for smooth, synchronized motion across multiple articulators. Although diffusion-based models capture diversity, their stochastic denoising often introduces temporal misalignment and motion artifacts. In this work, we propose SignFlow, a novel architecture for SLG based on conditional flow matching with optimal transport. By modeling deterministic flow paths guided by optimal transport and supervised via velocity fields, SignFlow generates gestures that are both semantically coherent and visually smooth. Experiments on the CSL-Daily dataset demonstrate that SignFlow achieves superior BLEU scores and DTW-based motion accuracy compared to both diffusion and autoregressive baselines.
A framework named perceptual functional spectrum analysis (pFSA) for analyzing how people perceive the multifunctional nonverbal behaviors that emerge in conversations is proposed. The goal is to elucidate the intrinsic nonverbal properties, called functional multiplicity and interpretational ambiguity, in a separable way. The former property is that a single behavior could imply multiple meanings, and the latter is that different observers could interpret the same behaviors differently. In the pFSA framework, the labels of multiple raters across multiple functions over time are represented as a third-order tensor. This study then formulated a semiorthogonal nonnegative tensor factorization (SO-NTF) that approximates the input tensor as a linear combination of the functional basis matrix, perceptual basis matrices, and perceptual coefficient matrices. The functional basis matrix consists of functional spectra that represent fundamental functionalities in conversations. The perceptual basis matrices represent the perceptual tendencies, which consist of the sensitivities of the raters to the fundamental functionalities. The perceptual coefficient matrices represent the temporal activations of the perceptual tendencies. The pFSA framework constructs the perceptual basis matrices to characterize both label reliability and diversity. This study targeted 32 head movement functions labeled by ten raters. The experimental results confirmed that pFSA could successfully analyze the levels of ambiguity for multiple functionalities, such as low ambiguity for addressing and backchannel functions and high ambiguity for thinking functions.
Disinformation poses serious challenges as it involves deliberate falsehoods that mislead the public, erode trust, and hinder informed decision-making, issues exacerbated by AI technologies like ChatGPT, which facilitate realistic generation while complicating detection. Existing methods often rely on large-scale multimodal datasets like NewsCLIPpings, but their limited language coverage and reliance on naive, easily detectable examples from social media or synthetic sources reduce their effectiveness, especially on out-of-domain (unseen) data. To address these limitations, we develop a novel multilingual, multimodal dataset curated from fact-checking websites, presenting a greater challenge for existing detection models. Notably, we find that multilingual training not only facilitates cross-lingual generalization but also improves robustness on monolingual data, highlighting the value of linguistic diversity in building generalizable OOC detectors. Our dataset uniquely incorporates supporting information, and we propose a simple training strategy that utilizes supportive information to enhance both learning efficiency (effective use of small-scale data) and detection accuracy, even on unseen data. The dataset and code will be publicly released.
Identifying affective responses of users engaged with digital information can provide valuable information on user experience and can be used for user modelling, retrieval, and content recommendation. Methods for recognising affective states often rely on supervised learning from a single modality, such as decoding affective information from content via computer vision, human physiology, or brain responses. These approaches assume the availability of paired data that contain the modality from which the prediction is made and the corresponding affective state label that is used to supervise the model. Unlike previous research, we introduce an approach to decode affective states via bimodal contrastive learning without using any externally provided affective state labels. Our method, entitled BALE, uses only bimodal data (paired brain recordings and images) to self-learn latent representations by contrasting image representations and the representations of the brain responses evoked when humans perceive the visual stimuli. We demonstrate the effectiveness of our approach using two publicly available datasets: fNIRS recordings of images showing various levels of valence (positivity/negativity) and fMRI recordings of individuals viewing human faces showing different emotions (joy/anger/contempt/pride/neutral). We evaluate our model on two tasks: classification of affective states and ranking of visual stimuli based on the affective responses they evoke when perceived. We report the first successful results for bimodal learning of affective states without labels and report performance approaching models trained with labelled supervision data. The results provide evidence that subjective affective states can be captured from multimodal interactions. Our source code is openly released at https://github.com/VadymV/BALE/.
Accurate pain assessment remains a major challenge in clinical practice, particularly in non-communicative patients where self-report is unfeasible. This study presents a hybrid, explainable approach for automatic pain classification from physiological signals, developed in the context of the AI4PAIN 2025 Challenge. We address key limitations of the dataset (limited size, signal discontinuity, and quality degradation) by combining deep learning (DL) for pattern discovery with classical machine learning (ML) for robust classification. In Stage 1, a CNN-LSTM model with attention and explainability modules, Gradient-weighted Class Activation Maps (GradCAM) and feature ablation, identifies temporally and modality relevant signal patterns. Guided by these insights, Stage 2 extracts a compact set of physiologically grounded features from photoplethysmography (PPG), electrodermal activity (EDA), and chest respiratory effort (RESP) signals. Stage 3 employs an ensemble of optimized ML models (CatBoost, XGBoost, LightGBM) trained on these features, achieving 0.84 F1-macro on validation data. Explainability analysis confirms that the model captures clinically relevant pain markers, such as sympathetic arousal (EDA), disrupted pulse variability, in terms of amplitude and width, and altered respiratory patterns. Despite a validation-to-test performance drop (0.14), our XAI-guided methodology demonstrates strong interpretability, highlighting its potential for trustworthy pain assessment in real-world clinical settings.
