The psychological profile that structurally documents the case of a depression patient is essential for psychotherapy. Large language models can be applied to summarize the profiles from counseling speech, however, it may suffer from long-context forgetting and produce unverifiable hallucinations, due to overlong length of speech, multi-party interactions and unstructured chatting. Hereby, we propose a StreamProfile, a streaming framework that processes counseling speech incrementally, extracts evidences grounded from ASR transcriptions by storing it in a Hierarchical Evidence Memory, and then performs a Chain-of-Thought pipeline according to PM+ psychological intervention for clinical reasoning. The final profile is synthesized strictly from those evidences, making every claim traceable. Experiments on real-world teenager counseling speech have shown that the proposed StreamProfile system can accurately generate the profiles and prevent hallucination.

Speech to Ultrasound Tongue Imaging (UTI) provides a non-invasive, clinically accessible modality for modeling tongue movement, critical for speech assessment and rehabilitation in dysarthria. However, existing approaches face two major challenges: the scarcity of paired dysarthric patient data limits model robustness and generalization, and the inherent articulatory and acoustic divergences between healthy and pathological speakers create domain shift problems during model training, resulting in generalization failures when processing unseen speech patterns. To address these issues, we propose Uni-UTIDiff, a unified diffusion model framework that adaptively generates UTI sequences from both normal and dysarthric speech. Furthermore, Our framework introduces a contrastive clustering-based unsupervised articulatory pattern extractor that disentangles pathological and normal speech features without manual labels, and an adaptive conditional fusion module that dynamically integrates acoustic embeddings with learned speech patterns to preserve articulatory specificity during UTI generation. Extensive experiments demonstrate that Uni-UTIDiff achieves state-of-the-art performance in both objective metrics and subjective evaluations. The results highlight its potential for robust, data-efficient, and clinically viable articulatory generation from speech.

Speech emotion recognition (SER) is limited by the lack of authentic conversational datasets and reliable annotation methods. We introduce CCSEMO, a naturalistic Chinese speech emotion dataset collected from real-world psychological counseling sessions, and propose a unified annotation framework grounded in emotional psychology to label subtle and dynamic emotions in conversational speech. The framework standardizes label definition, speech segmentation, annotator screening and training, and the overall annotation procedure, and is used to generate valence, arousal, and emotion-type labels for CCSEMO. We further construct CCSEMO-mini, a balanced subset for benchmarking, and report comparative baselines on the CCSEMO-mini, IEMOCAP, and MSP-Podcast datasets. Results show that CCSEMO-mini provides a realistic and challenging benchmark for conversational SER, while the proposed framework supports a standardized and practical annotation procedure for naturalistic speech emotions.

Autism spectrum disorder (ASD) is a prevalent neurodevelopmental condition lacking objective biomarkers for early diagnosis. Electroencephalography (EEG), with high temporal resolution, offers a way to capture atypical oscillations of ASD. We propose a Multi-Frequency Contrastive Learning Model (MF-CLM), a self-supervised framework integrating multi-band EEG to learn robust representations without extensive labels. Eighty children (40 ASD, 40 typical) underwent high-density EEG while viewing emotional stimuli. MF-CLM uses cross-frequency fusion, graph augmentation, and contrastive learning to improve robustness. Results showed superior performance, with 84.0% accuracy, 81.1% AUC, 83.2% sensitivity, and 78.0% specificity, exceeding SimCLR and baselines by up to 10.4%. Network analyses revealed atypical patterns: delta–theta hyperactivation in the sensorimotor network, beta–gamma suppression in the frontoparietal network, and theta–beta mixed activation in the visuomotor network. Causality showed hierarchical coupling, delta–theta driving beta–gamma and theta–beta. MF-CLM provides an interpretable, powerful framework for ASD diagnosis and mechanistic insights.

Automatic depression detection using speech signals with acoustic and textual modalities is a promising approach for early diagnosis. Depression-related patterns exhibit sparsity in speech: diagnostically relevant features occur in specific segments rather than being uniformly distributed. However, most existing methods treat all frames equally, assuming depression-related information is uniformly distributed and thus overlooking this sparsity. To address this issue, we proposes a depression detection network based on Adaptive Cross-Modal Gating (ACMG) that adaptively reassigns frame-level weights across both modalities, enabling selective attention to depression-related segments. Experimental results show that the depression detection system with ACMG outperforms baselines without it. Visualization analyses further confirm that ACMG automatically attends to clinically meaningful patterns, including low-energy acoustic segments and textual segments containing negative sentiments.

Non-invasive EEG-based speech decoding typically treats the neural-to-text mapping as a black box, offering limited interpretability and few explicit links to the brain’s language networks. Here we present a framework that reconstructs language-network dynamics in source space from scalp EEG by integrating neural perturbational inference with geometric constraints derived from cortical eigenmodes. A meta-learning diffusion decoder then translates these dynamics into text with few-shot adaptation. Evaluated on Chinese and English reading EEG, the framework achieves state-of-the-art decoding performance, reducing the character error rate by up to 11% relative to existing methods. The decoded semantic fields localize to temporal regions that match the ventral language pathway, and cross-modal experiments reveal shared semantic representations between reading and listening. Moreover, orthographic, omission, and semantic errors map onto distinct electrophysiological signatures—the N170, P200, and N400 components, respectively—and error-informed correction using these signatures further reduces the character error rate by 8.3%. By establishing an interpretable signal-to-language-network-to-text pathway, this work advances non-invasive brain–computer interfaces toward physiologically grounded, cross-modally generalizable, and diagnostically meaningful communication

Autism Spectrum Disorder (ASD) is a neuro-developmental disorder that adversely affects the healthy development of children. The current clinical diagnosis of ASD relies on demographics and descriptions of clinical symptoms and assessments based on rating scales, which are both time-consuming and subjective. Previous research reported atypical visual patterns exhibited by ASD children, implying that eye-tracking could be a potential tool to assist in distinguishing ASD from others. However, most studies only examined a singular dimension of visual trajectory, neglecting the possible use of comprehensive information on eye scan paths. Given this, the present study introduced the FF-ASDNET - a framework designed for the automated screening of ASD individuals leveraging eye-tracking technologies in conjunction with deep representation learning and multi-scale feature fusion. Eye movement trajectory was first objectively quantified from multi-scale perspectives, including Gaze Movements (GM), Spatial Attention Distribution (SAD), and Temporal Visual Information (TVI). The residual network (ResNet) and the rectangular residual convolutional temporal network (RecResTCNN) were then proposed to learn the features of the three representations. Finally, the features of the three scales were fused using a convolutional neural network. The proposed design achieved an AUC of 85.96\% and an F1 score of 0.85 on an independent dataset, showing that fusing multi-scale representations of eye scan trajectories could significantly improve the model's performance, rendering the development of an automated ASD screening plausible. In summary, FF-ASDNET is a promising tool for screening ASD individuals, with potential applications in clinical diagnosis and intervention.

Spontaneous speech has emerged as a promising biomarker for the non-invasive detection of Alzheimer’s disease (AD). Existing approaches rely on theory-based acoustic features, which are interpretable but may incompletely capture AD-related speech patterns, and on high-dimensional deep learning representations, which are expressive less interpretable. To address these limitations, this study proposes an interpretable AD detection framework based on multi-scale fusion of disentangled speech representations. A neural audio codec is employed to decompose the speech signal into three interpretable attributes: content, prosody, and timbre. These representations are integrated with sentence-level and global linguistic embeddings through a Graph Attention Network (GAT), enabling the modeling of complex interdependencies across multiple scales. Experimental results demonstrate that our proposed method achieves performance comparable to state-of-the-art models on cross-cultural and cross-linguistic datasets, attaining accuracies of 89.6%, 85.9%, and 95.4% on the ADReSS, ADReSSo, and a Chinese dataset, respectively. Feature analysis further indicates that timbre provides a discriminative signal comparable to linguistic features, suggesting its potential as a biomarker. The results highlight that fusing disentangled, multi-scale speech representations can improve both the performance and interpretability of automated AD detection systems.

BACKGROUND:Somatic symptom disorder (SSD) exhibits substantial clinical heterogeneity that limits treatment efficacy, with over 40% of patients failing to respond to standard interventions. Here, we developed a framework that integrates multi-frequency electroencephalography (EEG) connectomics with contrastive learning to identify distinct subtypes of SSD. METHODS:A contrastive variational autoencoder with Gaussian mixture modeling (CVAE-GM) was developed using resting-state EEG connectomics from a discovery cohort of 1,419 patients with SSD. The derived subtypes were clinically correlated with symptom dimensions and validated for reproducibility in an independent external cohort (n=530). RESULTS:We identified three robust subtypes, characterized by dominant connectivity in somatomotor, central executive, and limbic networks. Cross-validated canonical correlation analysis revealed distinct associations between subtype-related neural features and Neuro-11 clinical dimensions: the SMN-dominant subtype was associated with greater somatic symptom burden (cross-validated rcv = 0.42, fold-wise SD = 0.021, permutation p < 0.001), the CEN-dominant subtype with lower negative event reactivity (rcv = -0.38, SD = 0.017, p < 0.001), and the LN-dominant subtype with greater emotional symptoms (rcv = 0.36, SD = 0.014, p = 0.002). Notably, the insula emerged as a convergent hub across subtypes, whereas subtype differentiation was characterized by preferential insula coupling with the anterior cingulate cortex, dorsolateral prefrontal cortex, and thalamus, respectively. Independent validation in an external cohort confirmed subtype reproducibility with superior classification performance (accuracy=0.85, AUC=0.87). CONCLUSIONS:These findings support an EEG-based connectomic framework for investigating neurobiological heterogeneity in SSD and highlight insula-centered network features as promising candidates for future mechanistic stratification studies.

Speech therapy is essential for rehabilitating speech disorders caused by neurological impairments such as stroke. However, traditional manual and computer-assisted systems are limited in real-time accessibility and articulatory motion feedback. Recent advances in multimodal large language models (MLLMs) have demonstrated significant potential in healthcare, especially through their adaptive assessment and therapeutic feedback capabilities. Nevertheless, challenges including insufficient acquisition and fusion of articulatory information, inadequate parsing of articulatory organ motion trajectories, and the scarcity of domain-specific datasets hinder the application of MLLMs in speech therapy. To address these limitations, we propose an MLLM-based speech rehabilitation assistance system that leverages ultrasound tongue imaging and speech signals to deliver precise, interactive articulatory feedback. We construct a high-quality domain-specific dataset comprising ultrasound-speech dialogue pairs. This dataset facilitates fine-tuning to enhance the model’s clinical adaptability. Furthermore, our method develops spatiotemporal fusion training strategy of ultrasound videos and speech signals, enabling fine-grained articulatory impairment analysis and ultimately generating actionable feedback. Experimental results demonstrate the effectiveness of our model in articulatory analysis and clinical assessment.

The development of Brain-Computer-Interface (BCI) technology requires accurate decoding of brain activities measured by EEG. However, due to the non-stationary characteristics of EEG signals and intra- and inter-individual variability, it is not easy to construct a reliable and universal evaluation model for different subjects. In practical applications, most of the target domains are invisible, yet the current transfer learning models based on EEG signals mostly are target-domains-visable. To address this problem, this paper proposes a deep migration learning framework that reduces individual differences through intra-subject alignment and inter-subject alignment, extracts stable features after superposition averaging using a multi-scale spatio-temporal graph neural network, and employs a multi-source domain distribution normalization method, which enables the model to be effectively generalized to the target domain. The adaptive subject normalization layer introduced in the model gradually realizes the alignment of different source domain distributions during the training process, and executes the Test-Time-Adaptation (TTA) strategy in the testing phase to achieve dynamic adaptation to the target domain data. The experimental results show that the model outperforms traditional deep learning models on TMS-EEG data, while the ablation experiments verify the effectiveness of the subject-level normalization module in improving the model generalization ability.

The Structured Dialogue System, referred to as SuDoSys, is an innovative Large Language Model (LLM)-based chatbot designed to provide psychological counseling. SuDoSys leverages the World Health Organization (WHO)'s Problem Management Plus (PM+) guidelines to deliver stage-aware multi-turn dialogues. Existing methods for employing an LLM in multi-turn psychological counseling typically involve direct fine-tuning using generated dialogues, often neglecting the dynamic stage shifts of counseling sessions. Unlike previous approaches, SuDoSys considers the different stages of counseling and stores essential information throughout the counseling process, ensuring coherent and directed conversations. The system employs an LLM, a stage-aware instruction generator, a response unpacker, a topic database, and a stage controller to maintain dialogue flow. In addition, we propose a novel technique that simulates counseling clients to interact with the evaluated system and evaluate its performance automatically. When assessed using both objective and subjective evaluations, SuDoSys demonstrates its effectiveness in generating logically coherent responses. The system's code and program scripts for evaluation are open-sourced.

IntroductionDysarthria is a motor speech disorder frequently associated with subcortical damage. However, the precise roles of the subcortical nuclei, particularly the basal ganglia and thalamus, in the speech production process remain poorly understood.MethodsThe present study aimed to better understand their roles by mapping neuroimaging, behavioral, and speech data obtained from subacute stroke patients with subcortical lesions. Multivariate lesion-symptom mapping and voxel-based morphometry methods were employed to correlate lesions in the basal ganglia and thalamus with speech production, with emphases on linguistic processing and articulation.ResultsThe present findings revealed that the left thalamus and putamen are significantly correlated with concept preparation (r = 0.64, p < 0.01) and word retrieval (r = 0.56, p < 0.01). As the difficulty of the behavioral tasks increased, the influence of cognitive factors on early linguistic processing gradually intensified. The globus pallidus and caudate nucleus were found to significantly impact the movements of the larynx (r = 0.63, p < 0.01) and tongue (r = 0.59, p = 0.01). These insights underscore the complex and interconnected roles of the basal ganglia and thalamus in the intricate processes of speech production. The lateralization and hierarchical organization of each nucleus are crucial to their contributions to these speech functions.DiscussionThe present study provides a nuanced understanding of how lesions in the basal ganglia and thalamus impact various stages of speech production, thereby enhancing our understanding of the subcortical neuromechanisms underlying dysarthria. The findings could also contribute to the identification of multimodal assessment indicators, which could aid in the precise evaluation and personalized treatment of speech impairments.

Depression affects emotional expression and perception. As a non-invasive and privacy-preserving method, speech is widely used for automatic depression detection. However, existing models often focus only on depressive features in speech, ignoring the differential emotion expression patterns across different emotion-inducing tasks. To address this, we propose an emotion-guided graph attention network (emoGAT) for depression detection. By collecting speech-text data from depressed individuals and healthy controls during emotion-inducing tasks, we construct graph embeddings using sentiment cues from both speech and text. Experimental results show our method reduces the standard deviation by 1.8% and improves accuracy by 4.36%. Graph attention visualization also reveals depression-specific characteristics, such as flattened prosody in neutral picture description tasks and cognitive biases toward negative information, offering deeper insights into emotional relational expressions.

Dysarthria detection is crucial for clinical diagnosis and treatment. However, existing methods predominantly rely on supervised learning, which requires extensive annotated data, resulting in high costs and inconsistent data quality. To address this issue, this paper proposes a feature extraction method for dysarthria detection based on contrastive learning, which does not require annotated data. This method investigates how to extract features from patients and normal individuals using different pre-trained acoustic models. By maximizing the differences in their acoustic feature spaces, this method enhances detection accuracy. Finally, multiple classification methods are employed to detect dysarthria using the extracted features, achieving significant improvements across various evaluation metrics.

Challenges in the ability of older adults to comprehend vocal emotion have been documented. However, limited research has investigated the combined effects of aging and age-related hearing loss. The present study aimed to bridge this research gap by comparing the performance of three participant groups (younger adults with normal hearing, older adults with hearing loss, and older adults without hearing loss) in identification of "happy" and "sad" emotions via prosodic and semantic channels. We conducted regression models to investigate the relationship between age, hearing threshold, cognitive abilities and overall emotion perception performance. Results of emotion identification accuracy demonstrated that older adults with hearing loss performed worse than other two normal hearing groups in both channels. Additionally, only older adults with hearing loss showed lower accuracy in the emotional prosody than semantics, indicating only this group is influenced by channel. As for response time, both older listener groups responded more slowly than younger listeners in both channels. They also exhibited faster responses to "happy" compared to "sad", supporting the positivity effect on emotion perception in older participants. Moreover, the regression models indicated that age, hearing threshold and working memory (measured by Digit Span test) were predictive of participants' overall identification accuracy, and selective attention (measured by Stroop test) was predictive of participants' overall reaction time. These findings suggest that the degraded emotion perception among older adults is attributed to complex underlying mechanisms, which can be reasonably explained by not only aging but also the decline in hearing sensitivity and cognitive functions.

Ultrasound imaging has been widely adopted in speech research to visualize dynamic tongue movements during speech production. These images are universally used as visual feedback in interventions for articulation disorders or visual cues in speech recognition. Nevertheless, the availability of high-quality audio-ultrasound datasets remains scarce. The present study, therefore, aims to construct a multimodal database designed for Mandarin speech. The dataset integrates synchronized ultrasound images of lingual movement, and the corresponding audio recordings and text annotations elicited from 43 healthy speakers and 11 patients with dysarthria through speech tasks (including vowels, monosyllables, and sentences), with a total duration of 22.31 hours. In addition, a customized helmet structure was employed to stabilize the ultrasound probe, precisely controlling for head movement and minimizing displacement interference, The proposed database carries apparent values in automatic speech recognition, silent interface development, and research in speech pathology and linguistics.

Objective Adolescent depression has become an urgent global health issue, which has a considerable increase in mental health problems among young individuals. Traditional diagnostic methods for depression often rely on self-reported symptoms and subjective evaluations, which can lead to underdiagnosis or misdiagnosis, especially in teenagers who may conceal their symptoms due to stigma. This study aims to fill this gap by developing a multimodal physiological signal database designed specifically for adolescent depression. This database incorporates various physiological signals, including speech and heart rate data, to enhance the objectivity of depression diagnosis. The goal of this work is to provide a tool that improves diagnostic accuracy and offers insights into the autonomic nervous system (ANS) dysfunctions associated with depression, thus paving the way for effective therapeutic interventions. Methods This study recruited 86 adolescents aged between 12.00 and 20.00 years who were native Mandarin speakers. Data collection focused on multiple physiological modalities, including speech audio recordings, electrocardiogram (ECG) signals, and blood pressure readings. The participants were asked to engage in emotion elicitation tasks based on cognitive psychology principles. These tasks were designed to trigger a range of emotional responses, from neutral to positive and negative states, and allowed for real-time collection of vocal and physiological data under varying emotional conditions. ECG signals were analyzed to assess heart rate variability (HRV), a key marker of ANS function. Statistical methods, along with machine learning algorithms, were employed to analyze the relationship between vocal characteristics (pitch and speech energy) and physiological markers (HRV) to uncover potential patterns that differentiate depressed individuals from healthy controls. Results Analysis revealed significant differences between the depressed and control groups in speech patterns and physiological responses. Adolescents with depression showed reduced variability in pitch and low speech energy, which were indicative of emotional blunting, a common symptom of depression. These vocal changes were strongly correlated with anomalies in HRV, specifically, a reduction in HRV, which signals impaired ANS function. The integration of multimodal data types (speech and physiological signals) not only confirmed the presence of ANS dysregulation in depressed adolescents but also provided a new framework for identifying vocal biomarkers as reliable indicators of depression severity. Additionally, the present study demonstrated that using multimodal data improved the overall precision of depression diagnosis because the combination of physiological and vocal features yielded better discriminatory power than the use of either modality alone. Conclusions The creation of the multimodal physiological database presented herein represents an important step forward in the objective diagnosis of adolescent depression. By combining speech analysis with physiological markers, such as HRV, this study offers a comprehensive tool that can be used to diagnose depression with increased accuracy. This database not only provides a valuable resource for clinicians and researchers but also opens new avenues for personalized treatment approaches based on objective physiological data. Furthermore, this work highlights the critical role of the ANS in the pathology of depression and underscores the importance of integrating multimodal data in future psychiatric diagnostics. In conclusion, this database has the potential to revolutionize how adolescent depression is diagnosed and treated, providing a nuanced understanding of the neurophysiological mechanisms underlying this disorder.

In recent years, significant progress has been made in automated depression detection methods using speech and text data combined with deep learning. However, few studies have explored the connection between depression and speech emotions. To address this issue, this paper proposes a novel Transformer-based network leveraging Speech Emotion Information (SEI) for depression detection. The proposed network consists of a primary network used for depression classification and an auxiliary network is employed for speech emotion classification. In the primary network, the pre-trained Hubert and RoBERTa are used to obtain the short-term acoustic and textual features, respectively. And then the long-term audio and text features are aggregated from the shot-term features by using Transformer-based approaches with average pooling. The SEI extracted from the auxiliary network serves as supplementary auxiliary features aimed at augmenting the precision of depression recognition. Based on the proposed method, our best experimental results achieved an accuracy of 76.10