End-to-end Speech-to-Speech Translation (S2ST) enables direct transformation from source speech in one language to target speech in another. Still, it typically requires large-scale parallel data, which is unavailable for many low-resource languages. In this work, we investigate transfer learning strategies to improve S2ST performance in low-resource settings. Our approach involves pretraining a multi-task S2ST model on high-resource language pairs (e.g., French-English, German-English, Spanish-English) and transferring parameters to models for low-resource pairs such as Italian-English. We evaluate multiple transfer configurations, including full S2UT transfer, auxiliary ASR module sharing, and partial encoder/decoder reuse. Experimental results demonstrate that transfer from French-English consistently yields the most significant gains, owing to its larger data volume. BLEU scores improve from 10.41 to 13.77 (ES-EN), 9.70 to 12.15 (DE-EN), and more notably from 2.89 to 10.01 and 2.10 to 9.10 for the low-resource IT-EN and RU-EN pairs. Ablation studies show that encoder transfer contributes more to performance than decoder transfer, and that auxiliary ASR modules provide limited benefits. These findings suggest that carefully designed structural transfer, particularly of the encoder, is crucial for effective low-resource S2ST.
As machine learning models have advanced, the risk of attacks on such models has also increased. Typical examples include a poisoning attack ($\text{P A}$), which is an attack of injecting some “poisoned” samples into a training dataset of a target machine learning model to degrade its performance. In particular, PA using poisoned samples that seem clean for human eyes is called a clean-label poisoning attack (CLPA). A conventional approach for defending against PA and CLPA is to remove the poisoned samples from a dataset before training. However, this prophylactic defense approach is difficult to implement in federated learning (FL) environments because training samples dispersedly owned by multiple clients are not shared in a single place in FL. To address this problem, this paper proposes a novel therapeutic defense approach: a “detoxification” method for recovering the performance of a “poisoned” model that has suffered CLPA. To achieve detoxification, the proposed method fine-tunes only the network head of a poisoned model using a small set of labeled clean samples and a large set of unlabeled out-of-distribution (OOD) samples. Poisoned models tend to output uneven confidence scores near the ideal decision boundary, which is undesirable and therefore is calibrated by the proposed method using OOD samples with almost uniform soft labels. In the results of our experiments conducted on an object image recognition model, its recognition accuracy was recovered by up to $\text{1 1. 9 8} \%$ compared to the original poisoned model. This demonstrates the effectiveness of the proposed method.
Phase estimation is a classical problem in speech processing. Recently, diffusion-based phase estimation methods have achieved promising performance due to the powerful capability of generative models. Nonetheless, they usually suffer from inherent drawbacks like excessive computational complexity and relatively slow inference speed. To this end, this paper proposes SinDiff Phase, a novel single-step diffusion-based phase estimation model. Through unifying the setting of discrete-time steps to 1, and substituting the original score-matching loss function with direct optimization of the phase and its differential forms along the time and frequency dimensions, the proposed model achieves phase generation in a single step for the first time. Extensive experiments on the VCTK corpus show that while achieving around $35.1 \times$ faster inference speed than DiffPhase-small, a state-of-the-art diffusion-based phase estimation model, the proposed model achieves competitive performance and notably outperforms existing traditional and DNN-based baselines in various objective metrics, fully demonstrating the potential of the proposed method in real-time applications.
Spherical images capture visual information from all directions in a single frame, making them an essential tool for recording and analyzing landscapes. However, these images differ significantly from conventional photographs taken by cameras or from human visual perception, often resulting in a sense of visual discomfort. To address this issue, this study constructs the theory of a mathematical model for geometrically describing and analyzing landscape representations in spherical images. In particular, the model captures angular deviations that arise from differences in depth to the observed objects as the viewpoint moves. The proposed model is defined by a geometric structure that includes points at infinity in three-dimensional directions, distinguishing it from the conventional Riemann sphere.
Rain removal remains a challenging task in computer vision due to the complex interference of rain streaks with background structures. While Transformer-based methods have shown promising performances in modeling long-range dependencies, they have drawbacks such as a lack of prior guidance, confusion between rain textures and scene semantics, as well as insufficient cross-scale consistency. To address these limitations, we propose a novel image de-raining framework that integrates a Variational Autoencoder (VAE) with a Transformer backbone via a Latent-Guided Attention (LGA) mechanism. The VAE learns a global latent representation that captures the statistical structure of rain patterns, which is then injected into the Transformer at multiple levels through cross-attention modules. This enables the network to focus more effectively on essential scene features while suppressing rain artifacts. Furthermore, a hierarchical feature integration strategy is adopted to fuse low-level textures and high-level semantics across scales. Experiments demonstrate that our method achieves superior PSNR and SSIM scores on standard benchmark datasets, while maintaining robust performance under complex rain conditions. Our framework offers a new perspective on leveraging latent priors for enhancing Transformer-based image restoration, paving the way for future applications in image de-weathering and denoising.
Emotion recognition from EEG signals is essential for advancing human-computer interaction, mental health monitoring, and adaptive systems. While deep learning has shown promise in decoding complex EEG patterns, existing models introduce significant computational overhead, limiting their suitability for real-time and resource-constrained applications. This paper addresses the research gap in developing lightweight yet accurate models for direct EEG signal processing by proposing Tiny-VRN, a compact neural architecture tailored for EEG-based emotion recognition. The objective is to design an efficient model that maintains high classification performance while significantly reducing complexity. Tiny-VRN integrates residual learning with a variational bottleneck in a fully connected framework, using stacked dense residual blocks to capture temporal dependencies. Compared to a strong baseline, ResNet1D, Tiny-VRN reduces parameter count by over $\mathbf{7 5 \%}$ ($\mathbf{4 7 3, 8 6 5}$ vs. $\mathbf{1 0 7, 5 4 3}$ parameters) and cuts training time by more than half across both the GAMEEMO and LUMED datasets. Despite its compact design, Tiny-VRN achieves superior accuracy: 93.65% on GAMEEMO and $\mathbf{9 4. 4 2 \%}$ on LUMED, outperforming ResNet1D on all metrics. These findings underscore Tiny-VRN's effectiveness in balancing accuracy and efficiency, offering a practical solution for deploying EEG-based emotion recognition systems in real-time and edge-computing environments. The proposed model sets a new benchmark for compact EEG classifiers, with implications for scalable and responsive affective computing applications.
Accurate secondary path modeling is critical for the stability and effectiveness of active noise control systems. While offline secondary path modeling yields precise estimates before system control, it fails to accommodate time-varying environments. Online methods utilizing auxiliary noise enhance adaptability but inevitably increase residual noise. Neural network-based approaches offer data-driven alternatives; however, they demand substantial computational resources and prior training. Building upon prior single-channel work, this paper proposes a frequency-domain online modeling algorithm for multiple secondary paths without auxiliary noise. Theoretical analysis confirms the uniqueness of the modeling paths and control filters, and the frequencydomain processing substantially reduces computational complexity compared to the existing online modeling algorithm. Simulations in a multichannel ANC system demonstrate that the proposed algorithm achieves fast convergence and superior tracking performance under acoustic path variations, demonstrating its suitability for real-time applications in dynamic scenarios.
This paper proposes an organ detection method called OGTransVG based on vision-language model. The proposed network developed based on TransVG comprises several components: an image encoder, a text encoder, an imagetext feature fusion module, and two predictors. The image encoder and text encoder extract critical features from the image and text, respectively. The image-text feature fusion module integrates multi-model features to achieve useful feature representation. One predictor is used to detect objects and the other indicates whether objects exist or not. The proposed OGTransVG model is trained by some medical images. Here we perform the subjective and objective evaluation for performance analysis. Experimental results demonstrate that the proposed OGTransVG network can not only detect organs well but also successfully deal with the situation: no-target expressions.
Transformer models, particularly Large Language Models (LLMs), are revolutionizing numerous fields but impose significant computational demands, necessitating specialized hardware acceleration. This paper introduces NOCTUA, a novel Network-On-Chip (NoC)-based Transformer Universal Accelerator, designed and presented as a High-Efficiency Reconfigurable NoC-based Transformer Universal Accelerator. NOCTUA leverages a scalable Network-on-Chip, renowned for its flexibility and high-bandwidth communication, to interconnect a multitude of Processing Elements (PEs). These PEs are designed to execute complex Transformer operations both independently and in coordinated concert with other PEs, enabling fine-grained parallelism and collaborative computation. This distributed and reconfigurable design, underpinned by the adaptable NoC backbone, allows NOCTUA to dynamically adapt its hardware resources and dataflow pathways, achieving both high operational efficiency and universal applicability across a wide spectrum of Transformer models and workloads. We present the architectural details of NOCTUA and demonstrate its significant potential to enhance processing throughput and energy efficiency compared to conventional approaches. Through experiments with existing attention-based NLP models, including BERT and GPT-2 on various language tasks, NOCTUA achieves a performance of $\mathbf{1. 0 5}$ TOPS/W and outperforms these models in area efficiency with 0.71 TOPS $/ \text{mm}^{2}$.
To enable early diagnosis of Parkinson's disease (PD), gait analysis has gained attention, and smart insoles provide a practical solution for real-time gait monitoring. To address limitations like short battery life and reduced durability caused by numerous sensors in a smart insole, we developed a novel deep convolutional neural network (CNN). This CNN independently analyzes each foot's gait pattern, capturing spatial and temporal features. Tested on a public dataset, our model achieved $\mathbf{8 4. 7 4 \%}$ accuracy using only 8 of 32 sensors, outperforming traditional CNNs by approximately 8 %. This demonstrates the potential of our CNN for efficient and practical PD gait detection with fewer sensors.
Vision language model (VLM)-aided visual place recognition (VPR) has recently emerged as a promising paradigm, where pre-trained vision encoders provide coarse retrieval and VLM reasoning refines candidate rankings. Despite recent progress, existing approaches employ giant trillion-parameter VLMs to deliver qualitative same-place judgments, without explicit reasoning, which may overlook informative visual cues. To address this, we propose a hierarchical two-stage zero-shot VPR framework that incorporates quantitative vision-language reasoning while leveraging a compact vision-language model. In the first stage, dense local features from vision foundation models are aggregated to form compact global descriptors for coarse retrieval. In the second stage, the VLM is prompted to generate discriminative local context descriptions together with importance scores that explicitly highlight location-identifiable cues. These descriptions enable quantitative re-ranking of candidates through weighted similarity computation. The framework remains fully zero-shot, requiring no task-specific fine-tuning and relying on a compact off-the-shelf 38B-parameter VLM. Experiments on the SF-XL Occlusion, SF-XL Night, and AmsterTime benchmarks demonstrate that our method achieves performance comparable to supervised approaches.
Emotion recognition from multimodal data has become increasingly vital for affective computing applications such as virtual assistants, social robots, and mental health monitoring. Among various modalities, audio and text offer complementary emotional signals, but effectively integrating them remains a challenge due to asynchronous signals, modality imbalance, and static fusion strategies. This paper presents Emot-CM-BERT, a cross-modal framework designed to recognize emotion from audio and text inputs. The model integrates acoustic features—such as intonation, pitch, and rhythm—with semantic features that capture the contextual meaning of language, using a masked multimodal attention mechanism that dynamically fuses information across modalities. To further optimize learning, curriculum learning is employed to progressively introduce training complexity, while class-aware sampling addresses imbalances in emotion label distributions. We evaluate Emot-CM-BERT on three benchmark datasets—CREMA-D, IEMOCAP, and CMU-MOSEI—encompassing both acted and naturalistic emotional expressions. Experimental results demonstrate that the proposed model achieves competitive or superior performance compared to baselines, particularly in real-world, conversational settings. The analysis further confirms its effectiveness in aligning heterogeneous features and generalizing across diverse emotions.
This paper presents the Tiered Learning Framework (TLF) for course and assessment design in Digital Signal Processing (DSP) education. TLF divides the learning curve within a course into five levels, encourages students to reflect on which level they are currently at in their learning, and guides them on what they need to do to progress to the next level. Within this framework, students have more control and choice over how much they want to learn and how deeply they engage with the material. A key driver of student adherence to the framework and a mechanism for appropriate feedback is the use of suitable tiered assessments. The TLF was specifically trialled in class settings of various sizes over the past three years, for DSP courses and design projects. Feedback from students indicated that the TLF-based assessment inspired and motivated them to self-assess their understanding of the topic or course they studied and guided their improvement to the next level.
Text-to-Speech (TTS) technology has significantly progressed with deep learning, especially through models like Variational Autoencoder with Adversarial Learning for End-toEnd Text-to-Speech (VITS). However, improving audio quality particularly in duration diversity remains a challenge, especially for languages like Indonesian due to limited datasets and research. This study compares the performance of VITS using Stochastic Duration Predictor (SDP) and Deterministic Duration Predictor (DDP), while also exploring the impact of adversarial training on duration prediction. Evaluation employed subjective Mean Opinion Score (MOS) and objective Cosine Similarity using Resemblyzer. Two datasets were used: 343 formal audio samples and $\mathbf{1 2 5 0}$ mixed (formal and informal) samples. The more diverse dataset achieved better results, with a cosine similarity of 0.91124 and a MOS of 4.54. Findings indicate that SDP produces more natural durations, and adversarial learning enhances audio quality through better duration modeling.
Efficient utilization of computational resources remains a central challenge in reinforcement learning (RL)-based locomotion control, especially for high-dimensional bipedal robots. The robot learning process typically comprises two stages: data collection through environment interaction and policy optimization. Among these, the data collection phase is considered the primary bottleneck limiting overall learning speed. To address these challenges, we present HasRL Robot, a heterogeneous asynchronous RL-based learning system for bipedal locomotion. In this framework, trajectories are collected in parallel across CPUs, while the policy and value networks are updated asynchronously on the GPUs. This design enables effective use of computational resources and improves overall training throughput. Furthermore, our framework is extensible to other RL-driven tasks on high-performance machines, thereby maximizing computational efficiency. Experimental results demonstrate that HasRL Robot achieves superior training and testing performance, enabling the emergence of stable and human-like gait behaviors in simulation.
The advancements in voice conversion (VC) technology, specifically one-shot VC, have raised significant concerns regarding privacy and identity verification. Researchers use adversarial examples to protect voice data, as they can mislead algorithms. However, existing methods for protection via adversarial examples lack recoverability and reversibility, rendering them ineffective as robust protection mechanisms. To address this issue, we propose a recoverable audio adversarial example generation approach. It not only effectively protects speech data but also allows authorized users to restore the original speaker characteristics when needed. Our approach is tested using one-shot VC, with both subjective and objective evaluations conducted under white-box and black-box scenarios. The results demonstrate that the output of the VC model differs significantly from the protected speaker's voice, and our approach allows authorized users to restore the utterance for normal use.
Covariance matrix estimation in high dimensions is a central problem in data science, signal processing, and machine learning, yet it remains challenging due to the need to ensure both statistical accuracy and computational efficiency. In this paper, we revisit the positive definite covariance estimation framework of [1] and develop an efficient primal-dual algorithm that alternately updates primal and dual variables. We demonstrate that the proposed method can be interpreted equivalently as a proximal gradient scheme in the dual domain. The algorithm inherently preserves positive definiteness throughout the iterations and is provably globally linearly convergent. Numerical experiments establish the superiority of the proposed algorithm over state-of-the-art methods, highlighting its effectiveness for high-dimensional covariance estimation.
This paper proposes a novel semi-supervised end-to-end speech-to-text translation (S2TT) approach that leverages pseudo-labeling (PL) through joint decoding with both a text-totext translation (T2TT) model and an S2TT model. While pseudolabeling has proven effective in mitigating the data scarcity issue in S2TT, conventional PL approaches typically rely either on a standalone S2TT model or a cascade system combining automatic speech recognition (ASR) with a T2TT model. However, cascade systems trained on out-of-domain data often perform poorly on target domain inputs, and S2TT models fine-tuned on limited in-domain data tend to struggle with low-occurrence tokens. To address these limitations, our method employs both the S2TT model and the cascade system for PL, using the T2TT model to complement the S2TT model by improving accuracy on rare tokens. Experimental results demonstrate that our proposed method outperforms conventional PL strategies that utilize only a T2TT or S2TT model.
Few-shot acoustic signal classification remains a challenging problem due to the high diversity and variability of acoustic data and limited availability of labeled samples. While pretrained audio classification models have proven effective for various acoustic signal classification tasks, fine-tuning them can still lead to overfitting in low-resource settings. In this work, we proposes an attention-based meta-learning framework that operates on the hidden states of a pretrained audio classification model. Specifically, we introduce a trainable hierarchical additive attention module to extract meaningful features from the hidden states of a large-scale pre-trained Audio Spectrogram Transformer (AST). The attention mechanism is trained with a simple meta-learning paradigm, enabling effective adaptation to one-shot learning tasks. We evaluates the proposed model on multiple acoustic signal classification tasks, including acoustic scene classification, sound event recognition and underwater vessel noise classification. Experimental results demonstrate that our proposed framework substantially outperforms the existing methods such as CNN-based prototypical networks in terms of one-shot classification accuracy. This research not only provides an efficient solution for low data resource acoustic pattern recognition tasks but also demonstrate the strong potential of pre-trained audio classification models when combined with metalearning framework for few-shot learning.
