Single-cell RNA sequencing (scRNA-seq) data analysis faces multiple challenges, including high dimensionality, significant noise, and data loss. To effectively address these issues, we introduce AIGS, a robust and transparent single-cell analysis framework. AIGS utilizes an intelligent gene selection method that systematically identifies the most informative genes for clustering based on the normalized mutual information between pre-learned pseudo-labels and quantified genes. Additionally, AIGS incorporates a scale-invariant distance metric to assess cell-to-cell similarity, enhancing connections between homogeneous cells and ensuring more accurate and robust results. Through comprehensive comparisons with state-of-the-art techniques, AIGS demonstrates superior performance in both clustering accuracy and multi-resolution visualization quality. Our in-depth analysis of clustering and visualization results further reveals that AIGS can uncover complex, stage-specific gene expression patterns during the same developmental cell stage.

Continuous dimensional speech emotion recognition captures affective variation along valence, arousal, and dominance, providing finer-grained representations than categorical approaches. Yet most multimodal methods rely solely on global transcripts, leading to two limitations: (1) all words are treated equally, overlooking that emphasis on different parts of a sentence can shift emotional meaning; (2) only surface lexical content is represented, lacking higher-level interpretive cues. To overcome these issues, we propose MSF-SER (Multi-granularity Semantic Fusion for Speech Emotion Recognition), which augments acoustic features with three complementary levels of textual semantics--Local Emphasized Semantics (LES), Global Semantics (GS), and Extended Semantics (ES). These are integrated via an intra-modal gated fusion and a cross-modal FiLM-modulated lightweight Mixture-of-Experts (FM-MOE). Experiments on MSP-Podcast and IEMOCAP show that MSF-SER consistently improves dimensional prediction, demonstrating the effectiveness of enriched semantic fusion for SER.

Recent LLM-based TTS systems achieve strong quality and zero-shot ability, but lack fine-grained emotional control due to their reliance on discrete speech tokens. Existing approaches either limit emotions to categorical labels or cannot generalize to LLM-based architectures. We propose EMORL-TTS (Fine-grained Emotion-controllable TTS with Reinforcement Learning), a framework that unifies global intensity control in the VAD space with local emphasis regulation. Our method combines supervised fine-tuning with reinforcement learning guided by task-specific rewards for emotion category, intensity, and emphasis. Moreover, we further investigate how emphasis placement modulates fine-grained emotion intensity. Experiments show that EMORL-TTS improves emotion accuracy, intensity differentiation, and emphasis clarity, while preserving synthesis quality comparable to strong LLM-based baselines. Synthesized samples are available on-line1.

Dynamic Facial Expression Recognition (DFER) aims to identify human emotions from temporally evolving facial movements and plays a critical role in affective computing. While recent vision-language approaches have introduced semantic textual descriptions to guide expression recognition, existing methods still face two key limitations: they often underutilize the subtle emotional cues embedded in generated text, and they have yet to incorporate sufficiently effective mechanisms for filtering out facial dynamics that are irrelevant to emotional expression. To address these gaps, We propose GRACE, Granular Representation Alignment for Cross-modal Emotion recognition that integrates dynamic motion modeling, semantic text refinement, and token-level cross-modal alignment to facilitate the precise localization of emotionally salient spatiotemporal features. Our method constructs emotion-aware textual descriptions via a Coarse-to-fine Affective Text Enhancement (CATE) module and highlights expression-relevant facial motion through a motion-difference weighting mechanism. These refined semantic and visual signals are aligned at the token level using entropy-regularized optimal transport. Experiments on three benchmark datasets demonstrate that our method significantly improves recognition performance, particularly in challenging settings with ambiguous or imbalanced emotion classes, establishing new state-of-the-art (SOTA) results in terms of both UAR and WAR.

Emotional talking-head generation has emerged as a pivotal research area at the intersection of computer vision and multimodal artificial intelligence. Its core value lies in enhancing human-computer interaction through immersive and empathetic engagement. With the advancement of multimodal large language models, the driving signals for emotional talking-head generation has shifted from audio and video to more flexible text. However, current text-driven methods rely on predefined discrete emotion labels. This oversimplifies the dynamic complexity of real facial muscle movements, limiting natural emotional expressiveness. This study proposes the Think-Before-Draw framework to address two key challenges. First, we tackle in-depth semantic parsing of emotions. By innovatively introducing Chain-of-Thought, we transform abstract emotion labels into physiologically grounded facial muscle movement descriptions. This enables the mapping from high-level semantics to actionable motion features. Second, we address fine-grained expressiveness optimization. Inspired by artists’ portrait painting process, we propose a progressive guidance denoising strategy. It employs a ”global emotion localization—local muscle control” mechanism to refine fine-grained facial expression dynamics in generated videos. Our experiments demonstrate that our approach achieves competitive performance on the MEAD and HDTF datasets. Additionally, we collected a set of portrait images to evaluate our model’s zero-shot generation capability.

The brevity and noise inherent in microblog texts make interpreting their sentiment accurately challenging. However, social context provides crucial complementary information. Existing methods extract social context features separately using graph neural networks and integrate social and content features via simple concatenation, which fails to capture the crucial interaction between social and semantic information. Moreover, these methods are primarily based on Euclidean space and suffer from structural distortion when representing the underlying hierarchical and scale-free graph. To overcome this limitation, we propose a framework that jointly leverages social and semantic components of microblogs to contextualize their interpretation in this work. Specifically, this is achieved by constructing a heterogeneous multilayer graph that incorporates both social and linguistic information, and building a model that combines large-scale pretraining with transductive learning for sentiment classification. Furthermore, we use hyperbolic graph convolutional networks to learn better microblog representations that account for the scale-free and hierarchical nature of social networks. Our experiments on two real public datasets demonstrate that our proposed method outperforms existing baselines, highlighting the advantages of our multilayer graph and hyperbolic embeddings.

Open-Vocabulary Multimodal Emotion Recognition (OV-MER) is inherently challenging due to the ambiguity of equivocal multimodal cues, which often stem from distinct unobserved situational dynamics. While Multimodal Large Language Models (MLLMs) offer extensive semantic coverage, their performance is often bottlenecked by premature commitment to dominant data priors, resulting in suboptimal heuristics that overlook crucial, complementary affective cues across modalities. We argue that effective affective reasoning requires more than surface-level association; it necessitates reconstructing nuanced emotional states by synthesizing multiple evidence-grounded rationales that reconcile these observations from diverse latent perspectives. We introduce HyDRA, a Hybrid-evidential Deductive Reasoning Architecture that formalizes inference as a Propose-Verify-Decide protocol. To internalize this abductive process, we employ reinforcement learning with hierarchical reward shaping, aligning the reasoning trajectories with final task performance to ensure they best reconcile the observed multimodal cues. Systematic evaluations validate our design choices, with HyDRA consistently outperforming strong baselines–especially in ambiguous or conflicting scenarios–while providing interpretable, diagnostic evidence traces.

Depression detection remains challenged by generalized modeling approaches that fail to account for individual heterogeneity. To address this, the Multimodal Personality-aware Depression Detection (MPDD) Challenge introduced personalized features into the modeling process, aiming to better capture individual variability. However, the baseline models still exhibit two critical limitations: the neglect of textual semantics embedded in audio, and inconsistent predictions for the same subject across tasks and samples. Motivated by these limitations, we introduce HOPE (Hierarchical fusion for Optimized and Personality-aware Estimation of Depression), a unified framework for consistent, subject-level depression estimation. HOPE first employs a Latent Semantic Projection (LSP) module to reconstruct textual semantics from audio features when transcripts are unavailable. It then introduces a consistency-aware integration mechanism that hierarchically fuses multi-branch predictions to resolve inter-task and inter-sample contradictions. HOPE achieved first place in the MPDD Challenge Young Track, demonstrating strong cross-modal learning capabilities and consistent, subject-level depression prediction.

The advent of large language models, enabling flexibility through instruction-driven approaches, has revolutionized many traditional generative tasks, but large models for 3D data, particularly in comprehensively handling 3D shapes with other modalities, are still under-explored. By achieving instruction-based shape generations, versatile multimodal generative shape models can significantly benefit various fields like 3D virtual construction and network-aided design. In this work, we present ShapeGPT, a shape-included multi-modal framework to leverage strong pre-trained language models to address multiple shape-relevant tasks. Specifically, ShapeGPT employs a word-sentence-paragraph framework to discretize continuous shapes into shape words, further assembles these words for shape sentences, as well as integrates shape with instructional text for multi-modal paragraphs. To learn this shape-language model, we use a three-stage training scheme, including shape representation, multimodal alignment, and instruction-based generation, to align shape-language codebooks and learn the intricate correlations among these modalities. Extensive experiments demonstrate that ShapeGPT achieves comparable performance across shape-relevant tasks, including text-to-shape, shape-to-text, shape completion, and shape editing.

Prosody plays a fundamental role in human speech and communication, facilitating intelligibility and conveying emotional and cognitive states. Extracting accurate prosodic information from speech is vital for building assistive technology, such as controllable speech synthesis, speaking style transfer, and speech emotion recognition (SER). However, it is challenging to disentangle speaker-independent prosody representations since prosodic attributes, such as intonation, excessively entangle with speaker-specific attributes, e.g., pitch. In this article, we propose a novel model, called Diffsody, to disentangle and refine prosody representations: 1) to disentangle prosody representations, we leverage the expressive generative ability of a diffusion model by conditioning it on quantified semantic information and pretrained speaker embeddings. Additionally, a prosody encoder automatically learns prosody representations used for spectrogram reconstruction in an unsupervised fashion; and 2) to refine and learn speaker-invariant prosody representations, a scheduled gradient reversal layer (sGRL) is proposed and integrated into the prosody encoder of Diffsody. We extensively evaluate Diffsody through qualitative and quantitative means. t-SNE visualization and speaker verification experiments demonstrate the efficacy of the sGRL method in preventing speaker-specific information leakage. Experimental results on speaker-independent SER and automatic depression detection (ADD) tasks demonstrate that Diffsody can efficiently factorize speaker-independent prosody representations, resulting in a significant boost in SER and ADD. In addition, Diffsody synergistically integrates with the semantic representation model WavLM, which leads to a discernibly elevated performance, outperforming contemporary methods in both SER and ADD tasks. Furthermore, the Diffsody model exhibits promising potential for various practical applications, such as voice or style conversion. Some audio samples can be found on our https://leyuanqu.github.io/Diffsody/demo website.

We present a robust feature scaling method designed to handle imbalanced data in both machine learning and deep learning contexts. This method aims to scale features effectively when the underlying training data are imbalanced, a common scenario in real-world applications. While conventional z-score standardization performs well with balanced datasets, its effectiveness is less clear when dealing with imbalanced data. Although techniques such as under-sampling and over-sampling can address this issue in many cases, they risk introducing user bias and altering the statistical characteristics of the data. Therefore, we propose a method that does not rely on data augmentation or rebalancing. Our proposed method employs the tanh function as a scaling mechanism to map two imbalanced classes into the range of-1 to 1. The process involves calculating the means of the two classes and using the average of these means as the midpoint for mapping onto the center of the tanh curve. We then determine a scaling factor to map the majority of the data to the sloped section of the curve. Compared to z-score standardization, our method offers several numerical advantages: the scaled data are confined between-1 and 1, and their standard deviations are upper-bounded. This results in more stable numerical behavior during model training, mitigating issues like gradient explosion, which can occur with z-score standardization. Experiments and comparisons conducted on both machine learning and deep learning tasks demonstrate that our new scaling method generally outperforms z-score standardization. Thus, it provides a valuable alternative for scaling imbalanced data.

Inspired by biological processes, feature learning techniques, such as deep learning, have achieved great success in various fields. However, since biological organs may operate differently from semiconductor devices, deep models usually require dedicated hardware and are computation-complex. High energy consumption has made deep model growth unsustainable. We present an approach that directly implements feature learning using semiconductor physics to minimize disparity between model and hardware. Following this approach, a feature learning technique based on memristor drift-diffusion kinetics is proposed by leveraging the dynamic response of a single memristor to learn features. The model parameters and computational operations of the kinetics-based network are reduced by up to 2 and 4 orders of magnitude, respectively, compared with deep models. We experimentally implement the proposed network on 180 nm memristor chips for various dimensional pattern classification tasks. Compared with memristor-based deep learning hardware, the memristor kinetics-based hardware can further reduce energy and area consumption significantly. We propose that innovations in hardware physics could create an intriguing solution for intelligent models by balancing model complexity and performance.

The great variety of human emotional expression as well as the differences in the ways they perceive and annotate them make Speech Emotion Recognition (SER) an ambiguous and challenging task. With the development of deep learning, long-term progress has been made in supervised SER systems. However, the existing convolutional neural networks present certain limitations, such as their inability to well capture global features, which contain important emotional information. In addition, due to the subjective nature and continuity of emotion, the instance segments in which emotional speech is typically segmented do not fully reflect the true labels and cannot describe dynamic temporal changes. Thus, accurate emotional representation cannot be learnt in the process of feature extraction. In order to overtake these limitations, we propose an end-to-end network only for speech that maps sequences of different lengths to a fixed number of chunks and strictly preserves the order of chunks by adaptively adjusting their overlap. Subsequently, it extracts log-mel spectrogram features from chunk-level segments and feeds them into the Residual Multi-Scale Convolutional Neutral Networks with Transformer(RMSCTx) model framework. Finally, by keeping the order of the chunk-level segments, a temporal domain mean layer is used to further extract utterance-level feature representations. With this method, we perform multidimensional SER, i. e., the prediction of arousal, valence, and dominance. The experimental results on three popular corpora demonstrate not only the superiority of our approach, but also the robustness of the model for SER, showing an improvement of the recognition accuracy in the newest version of the public dataset MSP-Podcast (1.9).

Emotion Recognition in Conversations (ERC) is hard because discriminative evidence is sparse, localized, and often asynchronous across modalities. We center ERC on emotion hotspots and present a unified model that detects per-utterance hotspots in text, audio, and video, fuses them with global features via Hotspot-Gated Fusion, and aligns modalities using a routed Mixture-of-Aligners; a cross-modal graph encodes conversational structure. This design focuses modeling on salient spans, mitigates misalignment, and preserves context. Experiments on standard ERC benchmarks show consistent gains over strong baselines, with ablations confirming the contributions of HGF and MoA. Our results point to a hotspot-centric view that can inform future multimodal learning, offering a new perspective on modality fusion in ERC.

Speech Emotion Recognition (SER) is crucial for human-computer interaction applications. However, SER remains a challenging task due to limited datasets and ambiguous emotion boundaries. While Self-Supervised Learning (SSL) models have demonstrated considerable success in speech processing tasks, existing approaches still struggle to distinguish subtle emotional variations. In this paper, we propose a novel Label Semantic-driven Contrastive Learning framework (LaSCL) that integrates emotion label semantic embeddings into speech representation learning. Our method uses label embeddings as semantic anchors to explicitly model relationships between emotions and employ a label divergence loss to better establish clearer emotion boundaries. Experiments on the widely used IEMOCAP benchmark indicate that LaSCL achieves state-of-the-art performance compared with previous methods.

In recent years, emotional Text-to-Speech (TTS) synthesis and emphasis-controllable speech synthesis have advanced significantly. However, their interaction remains underexplored. We propose Emphasis Meets Emotion TTS (EME-TTS), a novel framework designed to address two key research questions: (1) how to effectively utilize emphasis to enhance the expressiveness of emotional speech, and (2) how to maintain the perceptual clarity and stability of target emphasis across different emotions. EME-TTS employs weakly supervised learning with emphasis pseudo-labels and variance-based emphasis features. Additionally, the proposed Emphasis Perception Enhancement (EPE) block enhances the interaction between emotional signals and emphasis positions. Experimental results show that EME-TTS, when combined with large language models for emphasis position prediction, enables more natural emotional speech synthesis while preserving stable and distinguishable target emphasis across emotions. Synthesized samples are available on-line.

Multimodal emotion recognition (MER) is important for building humanoid chatbots and has gained increasing attention in recent years. Existing studies have proven that extracting better modality-specific representations, which keep both commonality and individuality information of different modalities, is important for the MER task. However, all these works are restricted in making final predictions based on fusing modality-specific representations, and the effectiveness of the modality-specific decisions has not been studied. In this paper, we propose for the first time to fuse both the modality-specific representations and decisions for the MER task and design a bi-channel fusing network (BCFN). Specifically, a BCFN model first extracts and mixes the modality-specific representations and decisions in two convolutional blocks respectively, and then fuses the two joint multimodal features for the final decision. Extensive experiments are conducted on two MER benchmark datasets with state-of-the-art models employed as baselines for comparison. The empirical results demonstrate the superiority of our proposed BCFN model and confirm the effectiveness of incorporating modality-specific decisions for the MER task.

Humans can effortlessly modify various prosodic attributes, such as the placement of stress and the intensity of sentiment, to convey a specific emotion while maintaining consistent linguistic content. Motivated by this capability, we propose EmoAug, a novel style transfer model designed to enhance emotional expression and tackle the data scarcity issue in speech emotion recognition tasks. EmoAug consists of a semantic encoder and a paralinguistic encoder that represent verbal and non-verbal information respectively. Additionally, a decoder reconstructs speech signals by conditioning on the aforementioned two information flows in an unsupervised fashion. Once training is completed, EmoAug enriches expressions of emotional speech with different prosodic attributes, such as stress, rhythm and intensity, by feeding different styles into the paralinguistic encoder. EmoAug enables us to generate similar numbers of samples for each class to tackle the data imbalance issue as well. Experimental results on the IEMOCAP dataset demonstrate that EmoAug can successfully transfer different speaking styles while retaining the speaker identity and semantic content. Furthermore, we train a SER model with data augmented by EmoAug and show that the augmented model not only surpasses the state-of-the-art supervised and self-supervised methods but also overcomes overfitting problems caused by data imbalance. Some audio samples can be found on our demo website(1).

We present a new segmentation method based on minimizing the summation of total variations of the foreground and background of image. In this method, we define a non-parametric cost function of normalized total variation (NTV) and search over the histogram of an image for a threshold that minimizes the cost function. By its design, the method does not involve any preset or user-specified parameters, making the implementation and usage of the method consistent across sites and users. Contribution of the method is the adoption of the concept of total variation to irregular shapes that result from image segmentation. Novelty of the method is the use of total variation as a metric for assessing the appropriateness of segmentation. We applied the method to different kinds of images for evaluation and compared the method to some widely used existing methods. Visual inspection and quantitative evaluation show that the new method can achieve superior performance.

Advanced emotion monitoring and intervention systems are crucial for human - machine collaboration to accomplish specific tasks. One key challenge is accurately identifying mixed emotions and strategically implementing measures to enhance individual cognitive control abilities through facilitating emotional state regulation. To achieve these objectives, this research consists of two parts: the development of mixed emotion recognition methods based on EEG signals and the study of the neural mechanisms underlying the impact of mixed emotions on cognitive control. The results reveal that the proposed DBN-GBDT (Deep Belief NetworkGradient Boosting Decision Tree) method outperformed the other two methods, attaining a high classification accuracy of 96.7 %. Furthermore, mixed emotions of different valences exert differential effects on cognitive control, which can be captured by a two -stage model involving the P200 (early attention engagement) and N300 (late conflict resolution) components. The preparatory mechanisms induced by positive mixed emotions and the adaptive mechanisms induced by heterogeneously mixed emotions contribute to conflict resolution and enhance cognitive control abilities. The primary contributions of this study lie in providing feasible solutions for machine understanding of mixed emotions and enhancing individual cognitive control abilities, which holds immense potential for applications in the field of human - machine collaboration.