The psychological profile that structurally documents the case of a depression patient is essential for psychotherapy. Large language models can be applied to summarize the profiles from counseling speech, however, it may suffer from long-context forgetting and produce unverifiable hallucinations, due to overlong length of speech, multi-party interactions and unstructured chatting. Hereby, we propose a StreamProfile, a streaming framework that processes counseling speech incrementally, extracts evidences grounded from ASR transcriptions by storing it in a Hierarchical Evidence Memory, and then performs a Chain-of-Thought pipeline according to PM+ psychological intervention for clinical reasoning. The final profile is synthesized strictly from those evidences, making every claim traceable. Experiments on real-world teenager counseling speech have shown that the proposed StreamProfile system can accurately generate the profiles and prevent hallucination.

Speech to Ultrasound Tongue Imaging (UTI) provides a non-invasive, clinically accessible modality for modeling tongue movement, critical for speech assessment and rehabilitation in dysarthria. However, existing approaches face two major challenges: the scarcity of paired dysarthric patient data limits model robustness and generalization, and the inherent articulatory and acoustic divergences between healthy and pathological speakers create domain shift problems during model training, resulting in generalization failures when processing unseen speech patterns. To address these issues, we propose Uni-UTIDiff, a unified diffusion model framework that adaptively generates UTI sequences from both normal and dysarthric speech. Furthermore, Our framework introduces a contrastive clustering-based unsupervised articulatory pattern extractor that disentangles pathological and normal speech features without manual labels, and an adaptive conditional fusion module that dynamically integrates acoustic embeddings with learned speech patterns to preserve articulatory specificity during UTI generation. Extensive experiments demonstrate that Uni-UTIDiff achieves state-of-the-art performance in both objective metrics and subjective evaluations. The results highlight its potential for robust, data-efficient, and clinically viable articulatory generation from speech.

Speech emotion recognition (SER) is limited by the lack of authentic conversational datasets and reliable annotation methods. We introduce CCSEMO, a naturalistic Chinese speech emotion dataset collected from real-world psychological counseling sessions, and propose a unified annotation framework grounded in emotional psychology to label subtle and dynamic emotions in conversational speech. The framework standardizes label definition, speech segmentation, annotator screening and training, and the overall annotation procedure, and is used to generate valence, arousal, and emotion-type labels for CCSEMO. We further construct CCSEMO-mini, a balanced subset for benchmarking, and report comparative baselines on the CCSEMO-mini, IEMOCAP, and MSP-Podcast datasets. Results show that CCSEMO-mini provides a realistic and challenging benchmark for conversational SER, while the proposed framework supports a standardized and practical annotation procedure for naturalistic speech emotions.

OBJECTIVES:This study aimed to develop and validate machine learning (ML) models for automated prediction of perceptual dysphonia severity, as indexed by the Grade (G) parameter of the GRBAS scale, using acoustic analyses of sustained vowels. The overarching goal was to enhance objectivity, reproducibility, and efficiency in clinical voice assessment. METHODS:A total of 524 sustained/a/samples were collected from three databases. The evaluations of all recordings by ten raters using the GRBAS scale, that achieved excellent interrater reliability (Krippendorff's α = 0.96), were modelled. Forty-seven acoustic features spanning spectral, cepstral, perturbation, and noise-based indices were extracted using Parselmouth (Praat). Five ML classifiers-Decision Tree (DT), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), and Categorical Boosting (CatBoost)-were trained using 5-fold cross-validation (80/20 split) and evaluated by accuracy, F1-score, and quadratic weighted kappa (QWK). RESULTS:Gradient boosting algorithms outperformed traditional tree-based models. LightGBM achieved the highest QWK (0.945), followed by CatBoost (QWK = 0.941) and XGBoost (QWK = 0.935). Feature-importance analyses identified cepstral measures-particularly Smoothed Cepstral Peak Prominence (CPPS), Cepstral Spectral Index of Dysphonia (CSID), Acoustic Voice Quality Index (AVQI), Harmonics-to-Noise Ratio (HNR) as the most influential predictors of perceptual Grade (G), while jitter and shimmer parameters contributed minimally. Correlation analyses confirmed strong associations between Grade and AVQI (r = 0.854), HNR (r = -0.853), and cepstral indices (r = -0.835 to -0.832). CONCLUSIONS:Gradient boosting methods, particularly LightGBM, produced near-expert agreement with perceptual ratings, supporting their potential as objective, interpretable tools for clinical dysphonia assessment.

Autism spectrum disorder (ASD) is a prevalent neurodevelopmental condition lacking objective biomarkers for early diagnosis. Electroencephalography (EEG), with high temporal resolution, offers a way to capture atypical oscillations of ASD. We propose a Multi-Frequency Contrastive Learning Model (MF-CLM), a self-supervised framework integrating multi-band EEG to learn robust representations without extensive labels. Eighty children (40 ASD, 40 typical) underwent high-density EEG while viewing emotional stimuli. MF-CLM uses cross-frequency fusion, graph augmentation, and contrastive learning to improve robustness. Results showed superior performance, with 84.0% accuracy, 81.1% AUC, 83.2% sensitivity, and 78.0% specificity, exceeding SimCLR and baselines by up to 10.4%. Network analyses revealed atypical patterns: delta–theta hyperactivation in the sensorimotor network, beta–gamma suppression in the frontoparietal network, and theta–beta mixed activation in the visuomotor network. Causality showed hierarchical coupling, delta–theta driving beta–gamma and theta–beta. MF-CLM provides an interpretable, powerful framework for ASD diagnosis and mechanistic insights.

Previous studies have demonstrated that emotional features from a single acoustic sentiment label can enhance depression diagnosis accuracy. Additionally, according to the Emotion Context-Insensitivity theory and our pilot study, individuals with depression might convey negative emotional content in an unexpectedly calm manner, showing a high degree of inconsistency in emotional expressions during natural conversations. So far, few studies have recognized and leveraged the emotional expression inconsistency for depression detection. In this paper, a multimodal cross-attention method is presented to capture the Acoustic-Textual Emotional Inconsistency (ATEI) information. This is achieved by analyzing the intricate local and long-term dependencies of emotional expressions across acoustic and textual domains, as well as the mismatch between the emotional content within both domains. A Transformer-based model is then proposed to integrate this ATEI information with various fusion strategies for detecting depression. Furthermore, a scaling technique is employed to adjust the ATEI feature degree during the fusion process, thereby enhancing the model's ability to discern patients with depression across varying levels of severity. To best of our knowledge, this work is the first to incorporate emotional expression inconsistency information into depression detection. Experimental results on a counseling conversational dataset illustrate the effectiveness of our method.

Automatic depression detection using speech signals with acoustic and textual modalities is a promising approach for early diagnosis. Depression-related patterns exhibit sparsity in speech: diagnostically relevant features occur in specific segments rather than being uniformly distributed. However, most existing methods treat all frames equally, assuming depression-related information is uniformly distributed and thus overlooking this sparsity. To address this issue, we proposes a depression detection network based on Adaptive Cross-Modal Gating (ACMG) that adaptively reassigns frame-level weights across both modalities, enabling selective attention to depression-related segments. Experimental results show that the depression detection system with ACMG outperforms baselines without it. Visualization analyses further confirm that ACMG automatically attends to clinically meaningful patterns, including low-energy acoustic segments and textual segments containing negative sentiments.

Chronic subjective tinnitus (CST) has been increasingly associated with emotion as a potential physiological correlate. We investigated neural responses to negative emotion using EEG in 40 CST patients and 31 healthy controls (HC). Both groups showed N170 and P300 components, but CST exhibited consistently larger amplitudes, indicating hyper-reactive cortical processing. Time–frequency decoding revealed robust gamma-band discrimination in HC, while CST failed to sustain such differentiation. Moreover, gamma activity significantly compensated from decompensated patients, suggesting its potential as a biomarker of tinnitus severity. These results highlight a dual mechanism in CST, characterized by amplified ERP responses but inefficient gamma-band organization, which supports the integration of emotion dynamics into objective tinnitus assessment.

Non-invasive EEG-based speech decoding typically treats the neural-to-text mapping as a black box, offering limited interpretability and few explicit links to the brain’s language networks. Here we present a framework that reconstructs language-network dynamics in source space from scalp EEG by integrating neural perturbational inference with geometric constraints derived from cortical eigenmodes. A meta-learning diffusion decoder then translates these dynamics into text with few-shot adaptation. Evaluated on Chinese and English reading EEG, the framework achieves state-of-the-art decoding performance, reducing the character error rate by up to 11% relative to existing methods. The decoded semantic fields localize to temporal regions that match the ventral language pathway, and cross-modal experiments reveal shared semantic representations between reading and listening. Moreover, orthographic, omission, and semantic errors map onto distinct electrophysiological signatures—the N170, P200, and N400 components, respectively—and error-informed correction using these signatures further reduces the character error rate by 8.3%. By establishing an interpretable signal-to-language-network-to-text pathway, this work advances non-invasive brain–computer interfaces toward physiologically grounded, cross-modally generalizable, and diagnostically meaningful communication

Autism Spectrum Disorder (ASD) is a neuro-developmental disorder that adversely affects the healthy development of children. The current clinical diagnosis of ASD relies on demographics and descriptions of clinical symptoms and assessments based on rating scales, which are both time-consuming and subjective. Previous research reported atypical visual patterns exhibited by ASD children, implying that eye-tracking could be a potential tool to assist in distinguishing ASD from others. However, most studies only examined a singular dimension of visual trajectory, neglecting the possible use of comprehensive information on eye scan paths. Given this, the present study introduced the FF-ASDNET - a framework designed for the automated screening of ASD individuals leveraging eye-tracking technologies in conjunction with deep representation learning and multi-scale feature fusion. Eye movement trajectory was first objectively quantified from multi-scale perspectives, including Gaze Movements (GM), Spatial Attention Distribution (SAD), and Temporal Visual Information (TVI). The residual network (ResNet) and the rectangular residual convolutional temporal network (RecResTCNN) were then proposed to learn the features of the three representations. Finally, the features of the three scales were fused using a convolutional neural network. The proposed design achieved an AUC of 85.96\% and an F1 score of 0.85 on an independent dataset, showing that fusing multi-scale representations of eye scan trajectories could significantly improve the model's performance, rendering the development of an automated ASD screening plausible. In summary, FF-ASDNET is a promising tool for screening ASD individuals, with potential applications in clinical diagnosis and intervention.

Spontaneous speech has emerged as a promising biomarker for the non-invasive detection of Alzheimer’s disease (AD). Existing approaches rely on theory-based acoustic features, which are interpretable but may incompletely capture AD-related speech patterns, and on high-dimensional deep learning representations, which are expressive less interpretable. To address these limitations, this study proposes an interpretable AD detection framework based on multi-scale fusion of disentangled speech representations. A neural audio codec is employed to decompose the speech signal into three interpretable attributes: content, prosody, and timbre. These representations are integrated with sentence-level and global linguistic embeddings through a Graph Attention Network (GAT), enabling the modeling of complex interdependencies across multiple scales. Experimental results demonstrate that our proposed method achieves performance comparable to state-of-the-art models on cross-cultural and cross-linguistic datasets, attaining accuracies of 89.6%, 85.9%, and 95.4% on the ADReSS, ADReSSo, and a Chinese dataset, respectively. Feature analysis further indicates that timbre provides a discriminative signal comparable to linguistic features, suggesting its potential as a biomarker. The results highlight that fusing disentangled, multi-scale speech representations can improve both the performance and interpretability of automated AD detection systems.

BACKGROUND:Somatic symptom disorder (SSD) exhibits substantial clinical heterogeneity that limits treatment efficacy, with over 40% of patients failing to respond to standard interventions. Here, we developed a framework that integrates multi-frequency electroencephalography (EEG) connectomics with contrastive learning to identify distinct subtypes of SSD. METHODS:A contrastive variational autoencoder with Gaussian mixture modeling (CVAE-GM) was developed using resting-state EEG connectomics from a discovery cohort of 1,419 patients with SSD. The derived subtypes were clinically correlated with symptom dimensions and validated for reproducibility in an independent external cohort (n=530). RESULTS:We identified three robust subtypes, characterized by dominant connectivity in somatomotor, central executive, and limbic networks. Cross-validated canonical correlation analysis revealed distinct associations between subtype-related neural features and Neuro-11 clinical dimensions: the SMN-dominant subtype was associated with greater somatic symptom burden (cross-validated rcv = 0.42, fold-wise SD = 0.021, permutation p < 0.001), the CEN-dominant subtype with lower negative event reactivity (rcv = -0.38, SD = 0.017, p < 0.001), and the LN-dominant subtype with greater emotional symptoms (rcv = 0.36, SD = 0.014, p = 0.002). Notably, the insula emerged as a convergent hub across subtypes, whereas subtype differentiation was characterized by preferential insula coupling with the anterior cingulate cortex, dorsolateral prefrontal cortex, and thalamus, respectively. Independent validation in an external cohort confirmed subtype reproducibility with superior classification performance (accuracy=0.85, AUC=0.87). CONCLUSIONS:These findings support an EEG-based connectomic framework for investigating neurobiological heterogeneity in SSD and highlight insula-centered network features as promising candidates for future mechanistic stratification studies.

Speech therapy is essential for rehabilitating speech disorders caused by neurological impairments such as stroke. However, traditional manual and computer-assisted systems are limited in real-time accessibility and articulatory motion feedback. Recent advances in multimodal large language models (MLLMs) have demonstrated significant potential in healthcare, especially through their adaptive assessment and therapeutic feedback capabilities. Nevertheless, challenges including insufficient acquisition and fusion of articulatory information, inadequate parsing of articulatory organ motion trajectories, and the scarcity of domain-specific datasets hinder the application of MLLMs in speech therapy. To address these limitations, we propose an MLLM-based speech rehabilitation assistance system that leverages ultrasound tongue imaging and speech signals to deliver precise, interactive articulatory feedback. We construct a high-quality domain-specific dataset comprising ultrasound-speech dialogue pairs. This dataset facilitates fine-tuning to enhance the model’s clinical adaptability. Furthermore, our method develops spatiotemporal fusion training strategy of ultrasound videos and speech signals, enabling fine-grained articulatory impairment analysis and ultimately generating actionable feedback. Experimental results demonstrate the effectiveness of our model in articulatory analysis and clinical assessment.

The Structured Dialogue System, referred to as SuDoSys, is an innovative Large Language Model (LLM)-based chatbot designed to provide psychological counseling. SuDoSys leverages the World Health Organization (WHO)'s Problem Management Plus (PM+) guidelines to deliver stage-aware multi-turn dialogues. Existing methods for employing an LLM in multi-turn psychological counseling typically involve direct fine-tuning using generated dialogues, often neglecting the dynamic stage shifts of counseling sessions. Unlike previous approaches, SuDoSys considers the different stages of counseling and stores essential information throughout the counseling process, ensuring coherent and directed conversations. The system employs an LLM, a stage-aware instruction generator, a response unpacker, a topic database, and a stage controller to maintain dialogue flow. In addition, we propose a novel technique that simulates counseling clients to interact with the evaluated system and evaluate its performance automatically. When assessed using both objective and subjective evaluations, SuDoSys demonstrates its effectiveness in generating logically coherent responses. The system's code and program scripts for evaluation are open-sourced.

The development of Brain-Computer-Interface (BCI) technology requires accurate decoding of brain activities measured by EEG. However, due to the non-stationary characteristics of EEG signals and intra- and inter-individual variability, it is not easy to construct a reliable and universal evaluation model for different subjects. In practical applications, most of the target domains are invisible, yet the current transfer learning models based on EEG signals mostly are target-domains-visable. To address this problem, this paper proposes a deep migration learning framework that reduces individual differences through intra-subject alignment and inter-subject alignment, extracts stable features after superposition averaging using a multi-scale spatio-temporal graph neural network, and employs a multi-source domain distribution normalization method, which enables the model to be effectively generalized to the target domain. The adaptive subject normalization layer introduced in the model gradually realizes the alignment of different source domain distributions during the training process, and executes the Test-Time-Adaptation (TTA) strategy in the testing phase to achieve dynamic adaptation to the target domain data. The experimental results show that the model outperforms traditional deep learning models on TMS-EEG data, while the ablation experiments verify the effectiveness of the subject-level normalization module in improving the model generalization ability.

IntroductionDysarthria is a motor speech disorder frequently associated with subcortical damage. However, the precise roles of the subcortical nuclei, particularly the basal ganglia and thalamus, in the speech production process remain poorly understood.MethodsThe present study aimed to better understand their roles by mapping neuroimaging, behavioral, and speech data obtained from subacute stroke patients with subcortical lesions. Multivariate lesion-symptom mapping and voxel-based morphometry methods were employed to correlate lesions in the basal ganglia and thalamus with speech production, with emphases on linguistic processing and articulation.ResultsThe present findings revealed that the left thalamus and putamen are significantly correlated with concept preparation (r = 0.64, p < 0.01) and word retrieval (r = 0.56, p < 0.01). As the difficulty of the behavioral tasks increased, the influence of cognitive factors on early linguistic processing gradually intensified. The globus pallidus and caudate nucleus were found to significantly impact the movements of the larynx (r = 0.63, p < 0.01) and tongue (r = 0.59, p = 0.01). These insights underscore the complex and interconnected roles of the basal ganglia and thalamus in the intricate processes of speech production. The lateralization and hierarchical organization of each nucleus are crucial to their contributions to these speech functions.DiscussionThe present study provides a nuanced understanding of how lesions in the basal ganglia and thalamus impact various stages of speech production, thereby enhancing our understanding of the subcortical neuromechanisms underlying dysarthria. The findings could also contribute to the identification of multimodal assessment indicators, which could aid in the precise evaluation and personalized treatment of speech impairments.

Depression affects emotional expression and perception. As a non-invasive and privacy-preserving method, speech is widely used for automatic depression detection. However, existing models often focus only on depressive features in speech, ignoring the differential emotion expression patterns across different emotion-inducing tasks. To address this, we propose an emotion-guided graph attention network (emoGAT) for depression detection. By collecting speech-text data from depressed individuals and healthy controls during emotion-inducing tasks, we construct graph embeddings using sentiment cues from both speech and text. Experimental results show our method reduces the standard deviation by 1.8% and improves accuracy by 4.36%. Graph attention visualization also reveals depression-specific characteristics, such as flattened prosody in neutral picture description tasks and cognitive biases toward negative information, offering deeper insights into emotional relational expressions.

Dysarthria detection is crucial for clinical diagnosis and treatment. However, existing methods predominantly rely on supervised learning, which requires extensive annotated data, resulting in high costs and inconsistent data quality. To address this issue, this paper proposes a feature extraction method for dysarthria detection based on contrastive learning, which does not require annotated data. This method investigates how to extract features from patients and normal individuals using different pre-trained acoustic models. By maximizing the differences in their acoustic feature spaces, this method enhances detection accuracy. Finally, multiple classification methods are employed to detect dysarthria using the extracted features, achieving significant improvements across various evaluation metrics.

Ultrasound imaging has been widely adopted in speech research to visualize dynamic tongue movements during speech production. These images are universally used as visual feedback in interventions for articulation disorders or visual cues in speech recognition. Nevertheless, the availability of high-quality audio-ultrasound datasets remains scarce. The present study, therefore, aims to construct a multimodal database designed for Mandarin speech. The dataset integrates synchronized ultrasound images of lingual movement, and the corresponding audio recordings and text annotations elicited from 43 healthy speakers and 11 patients with dysarthria through speech tasks (including vowels, monosyllables, and sentences), with a total duration of 22.31 hours. In addition, a customized helmet structure was employed to stabilize the ultrasound probe, precisely controlling for head movement and minimizing displacement interference, The proposed database carries apparent values in automatic speech recognition, silent interface development, and research in speech pathology and linguistics.