LLM-based universal information extraction (UIE) methods often rely on additional information beyond the original training data, which increases training complexity yet often yields limited gains. To address this, we propose ProUIE, a Macro-to-Micro progressive learning approach that improves UIE without introducing any external information. ProUIE consists of three stages: (i) macro-level Complete Modeling (CM), which learns NER, RE, and EE along their intrinsic difficulty order on the full training data to build a unified extraction foundation, (ii) meso-level Streamlined Alignment (SA), which operates on sampled data with simplified target formats, streamlining and regularizing structured outputs to make them more concise and controllable, and (iii) micro-level Deep Exploration (DE), which applies GRPO with stepwise fine-grained rewards (SFR) over structural units to guide exploration and improve performance. Experiments on 36 public datasets show that ProUIE consistently improves unified extraction, outperforming strong instruction-tuned baselines on average for NER and RE while using a smaller backbone, and it further demonstrates clear gains in large-scale production-oriented information extraction.

Recent progress in Multimodal Large Language Models (MLLMs) has enabled mobile GUI agents capable of visual perception, cross-modal reasoning, and interactive control. However, existing benchmarks are largely English-centric and fail to capture the linguistic and interaction characteristics of the Chinese mobile ecosystem. They also focus on isolated skills such as GUI grounding or offline agent, lacking a unified and fine-grained framework to assess the full capability chain from perception to execution. To address this gap, we introduce \textbf{GUI-CEval}, the first comprehensive benchmark for Chinese mobile GUI agents, built entirely on \textbf{real-device} environments. GUI-CEval spans 201 mainstream apps across four device types and adopts a two-level structure that evaluates both atomic abilities and end-to-end performance along five dimensions: perception, planning, reflection, execution, and evaluation. All data are collected and verified through multi-stage manual processes to ensure authenticity and reproducibility. Extensive experiments on 20 representative MLLMs and multi-agent systems show that while models such as Qwen2.5-VL and UI-TARS perform competitively, most MLLMs still exhibit clear weaknesses in reflective decision-making and post-action self-evaluation, limiting their reliability in real-world interactions. We hope GUI-CEval provides a reproducible and interpretable benchmark to guide capability diagnosis and advance the development of Chinese mobile GUI agents.

Multimodal retrieval systems typically employ Vision Language Models (VLMs) that encode images and text independently into vectors within a shared embedding space. Despite incorporating text encoders, VLMs consistently underperform specialized text models on text-only retrieval tasks. Moreover, introducing additional text encoders increases storage, inference overhead, and exacerbates retrieval inefficiencies, especially in multilingual settings. To address these limitations, we propose a multi-task learning framework that unifies the feature representation across images, long and short texts, and intent-rich queries. To our knowledge, this is the first work to jointly optimize multilingual image retrieval, text retrieval, and natural language understanding (NLU) tasks within a single framework. Our approach integrates image and text retrieval with a shared text encoder that is enhanced by NLU features for intent understanding and retrieval accuracy.

Reinforcement learning has emerged as a paradigm for post-training large language models, boosting their reasoning capabilities. Such approaches compute an advantage value for each sample, reflecting better or worse performance than expected, thereby yielding both positive and negative signals for training. However, existing approaches often mix them indiscriminately, especially in the early stages, leading to ambiguous guidance and limited gains. To address this issue, we propose CAPO (Curriculum Advantage Policy Optimization), an adaptive curriculum mechanism based on advantage signals. The proposed mechanism bootstraps imitation learning with positive-only advantage samples to establish robust foundations, and subsequently introduces negative signals to cultivate discriminative capabilities, thereby improving generalization across complex scenarios. Compatible with diverse optimization methods including GRPO, PPO, RLOO, and Reinforce++, our method consistently achieves stable and significant improvements in mathematical reasoning tasks, and further generalizes effectively to multimodal Graphical User Interface (GUI) reasoning scenarios, establishing itself as a versatile and robust optimization paradigm.

Multimodal large language models (MLLMs) have made significant progress in mobile agent development, yet their capabilities are predominantly confined to a reactive paradigm, where they merely execute explicit user commands.The emerging paradigm of \textbf{proactive intelligence}, where agents autonomously anticipate needs and initiate actions, represents the next frontier for mobile agents. However, its development is critically bottlenecked by the lack of benchmarks that can address real-world complexity and enable objective, executable evaluation. To overcome these challenges, we introduce \textbf{ProactiveMobile}, a comprehensive benchmark designed to systematically advance research in this domain. ProactiveMobile formalizes the proactive task as inferring latent user intent across four dimensions of on-device contextual signals and generating an executable function sequence from a comprehensive function pool of 63 APIs. The benchmark features over 3,660 instances of 14 scenarios that embrace real-world complexity through multi-answer annotations. To ensure quality, a team of 30 experts conducts a final audit of the benchmark, verifying factual accuracy, logical consistency, and action feasibility, and correcting any non-compliant entries.Extensive experiments demonstrate that our fine-tuned Qwen2.5-VL-7B-Instruct achieves a success rate of 19.15\%, outperforming o1 (15.71\%) and GPT-5 (7.39\%). This result indicates that proactivity is a critical competency widely lacking in current MLLMs, yet it is learnable, emphasizing the importance of the proposed benchmark for proactivity evaluation.

The direction of target speech is a crucial clue for multichannel speech enhancement. But its real-time estimation is usually very challenging, especially when the source is moving and in far-field reverb environments. Instead of directly estimating the direction, we propose a direction-guided spatial attention to automatically focus on the target source and incorporate it into masking neural beamformers for speech enhancement. Neural beamformers are trained to learn to preserve signals from the look direction areas and suppress signals from other direction areas as much as possible. Spatial attention is guided by an extra classification objective of the target direction to focus on the target direction area and select an appropriate neural beamformer to listen to the direction area of target speech. Systematic experiments demonstrate that the proposed approach improves the performances of both speech enhancement and far-field speech recognition against prior methods. 11Corresponding Author: Yaran Chen

Current multimodal large lanauge models possess strong perceptual and reasoning capabilities, however high computational and memory requirements make them difficult to deploy directly on on-device environments. While small-parameter models are progressively endowed with strong general capabilities, standard Vision Transformer (ViT) encoders remain a critical bottleneck, suffering from excessive latency and memory consumption when processing high-resolution inputs.To address these challenges, we introduce HyperVL, an efficient multimodal large language model tailored for on-device inference. HyperVL adopts an image-tiling strategy to cap peak memory usage and incorporates two novel techniques: (1) a Visual Resolution Compressor (VRC) that adaptively predicts optimal encoding resolutions to eliminate redundant computation, and (2) Dual Consistency Learning (DCL), which aligns multi-scale ViT encoders within a unified framework, enabling dynamic switching between visual branches under a shared LLM. Extensive experiments demonstrate that HyperVL achieves state-of-the-art performance among models of comparable size across multiple benchmarks. Furthermore, it significantly significantly reduces latency and power consumption on real mobile devices, demonstrating its practicality for on-device multimodal inference.

To solve the cocktail party problem in real multi-talker environments, this article proposed a multi-cue guided semi-supervised target speaker separation method (MuSS). Our MuSS integrates three target speaker-related cues, including spatial, visual, and voiceprint cues. Under the guidance of the cues, the target speaker is separated into a predefined output channel, and the interfering sources are separated into other output channels with the optimal permutation. Both synthetic mixtures and real mixtures are utilized for semi-supervised training. Specifically, for synthetic mixtures, the separated target source and other separated interfering sources are trained to reconstruct the ground-truth references, while for real mixtures, the mixture of two real mixtures is fed into our separation model, and the separated sources are remixed to reconstruct the two real mixtures. Besides, in order to facilitate finetuning and evaluating the estimated source on real mixtures, we introduce a real multi-modal speech separation dataset, RealMuSS, which is collected in real-world scenarios and is comprised of more than one hundred hours of multi-talker mixtures with high-quality pseudo references of the target speakers. Experimental results show that the pseudo references effectively improve the finetuning efficiency and enable the model to successfully learn and evaluate estimating speech on real mixtures, and various cue-driven separation models are greatly improved in signal-to-noise ratio and speech recognition accuracy under our semi-supervised learning framework.

Bamboo forests exhibit a unique efficient growth pattern that makes them invaluable in reducing atmospheric CO2 levels. Additionally, bamboo forests offer a diverse range of products, thus holding the potential to bolster local income. Despite these benefits, the comprehensive assessment of bamboo forests’ potential in both carbon abatement and improving local income enhancement has been hindered by the absence of a detailed bamboo biomass map. In this study, we address this gap by amalgamating a bamboo aboveground biomass (AGB) map covering three prominent producing provinces in southern China, utilizing multi-source remote sensing datasets. The results not only demonstrate a satisfactory consistency with China’s Ninth National Forest Inventory but also provide a more detailed spatial distribution. Based on this AGB estimation, we project an approximately threefold potential increase in annual bamboo culm harvest from existing bamboo forests. This represents a significant opportunity for expanding carbon abatement efforts, elevating local income levels, and facilitating the production of bamboo-derived biofuels. Furthermore, the adoption of an optimized management strategy has the potential to further enhance bamboo production. This study generates the first high-resolution bamboo AGB map and underscores the substantial potential of China’s bamboo forests in contributing to carbon sequestration and improving local income. The favorable income generated for local residents can serve as a compelling incentive for the implementation of sustainable forest management practices, offering a promising pathway toward achieving carbon-related objectives within the forestry sector and providing necessary support for forestry designation projects.

Deep learning has great potential on decoding EEG in brain-computer interface. While common deep learning algorithms cannot directly train models with data from multiple individuals because of the inter-individual differences in EEG. Collecting enough data for each subject to satisfy the training of deep learning would result in an increase in training cost. This study proposes a novel transfer learning, EEGNet-based multi-source domain filter for transfer learning (EEGNet-MDFTL), to reduce the amount of training data and improve the performance of BCI. The EEGNet-MDFTL uses bagging ensemble learning to learn domain-invariant features from the multi-source domain and utilizes model loss value to filter the multi-source domain. Compared with baseline methods, the accuracy of the EEGNet-MDFTL reaches 91.96%, higher than two state-of-the-art methods, which demonstrates source domain filter can select similar source domains to improve the accuracy of the model, and remains a high level even when the data amount is reduced to 1/8, proving that ensemble learning learns enough domain invariant features from the multi-source domain to make the model insensitive to data amount. The proposed EEGNet-MDFTL is effective in improving the decoding performance with a small amount of data, which is helpful to save the BCI training cost.

Forest biomass is an important indicator of forest ecosystem productivity, and it plays vital roles in the global carbon cycling, global climate change mitigating, and ecosystem researches. Multiscale, rapid, and accurate extraction of forest biomass information is always a research topic. In this study, comprehensive investigation of a larch (Larix olgensis) plantation was performed using remote sensing and field-based monitoring methods, in combination with LiDAR-based multisource data and machine learning methods. On this basis, a universal, multiscale (single tree, stand, management unit, and region), and unit-high-precision continuous monitoring method was proposed for forest biomass components. The results revealed the following. (1) Airborne LiDAR point cloud variables exhibited significant correlation with the aboveground components (except leaves) and the whole-plant biomass (Radj2 > 0.91), suitable for extraction or estimation of forest parameters such as biomass and stock volume. (2) In terms of biomass monitoring at forest stand and management unit scale, a random forest model performed well in fitting accuracy and generalization ability, whereas a multiple linear regression model produced clearer explanation regarding the biomass of each forest component. (3) Using seasonal phenological characteristics in the study area, larch distribution information was extracted effectively. The overall accuracy reached 90.0%, and the kappa coefficient reached 0.88. (4) A regional-scale forest biomass component estimation model was constructed using a long short-term memory model, which effectively reduced the probability of biomass underestimation while ensuring good estimation accuracy, with R2 exceeding 0.6 for the biomass of the aboveground and whole-plant components. This research provides theoretical support for rapid and accurate acquisition of large-scale forest biomass information.

Although the advantage of spike timing-based over rate-based network computation has been recognized, the underlying mechanism remains unclear. Using Tempotron and Perceptron as elementary neural models, we examined the intrinsic difference between spike timing-based and rate-based computations. For more direct comparison, we modified Tempotron computation into rate-based computation with the retention of some temporal information. Previous studies have shown that spike timing-based computation are computationally more powerful than rate-based computation in terms of the number of computational units required and the capability in classifying random patterns. Our study showed that spike timing-based and rate-based Tempotron computations provided similar capability in classifying random spike patterns, as well as in text sentiment classification and spam text detection. However, spike timing-based computation is superior in performing a task involving discriminating forward vs. reverse sequence of events, i.e., information mainly temporal in nature. Further studies revealed that this superiority required the asymmetry in the profile of the postsynaptic potential (PSP), and that temporal sequence information was converted to biased spatial distribution of synaptic weight modifications during learning. Thus, the intrinsic PSP asymmetry is a mechanistic basis for the high efficiency of spike timing-based computation for processing temporal information.

Multi-modal cues, including spatial information, facial expression and voiceprint, are introduced to the speech separation and speaker extraction tasks to serve as complementary information to achieve better performance. However, the introduction of these cues brings about an increasing number of parameters and model complexity, which makes it harder to deploy these models on resource-constrained devices. In this paper, we alleviate the aforementioned problem by proposing a Lightweight Multi-modal framework for Speaker Extraction (LiMuSE). We propose to use GC-equipped TCN, which incorporates Group Communication (GC) and Temporal Convolutional Network (TCN) in the Context Codec module, the audio block and the fusion block. The experiments on the MC GRID dataset demonstrate that LiMuSE achieves on par or better performance with a much smaller number of parameters and less model complexity. We further investigate the impacts of the quantization of LiMuSE. Our code and dataset are provided.

Brain-computer interface (BCI) is a useful device for people without relying on peripheral nerves and muscles. However, the performance of the event-related potential (ERP)-based BCI declines when applying it to real environments, especially in cross-state and cross-subject conditions. Here we employ temporal modeling and adversarial training to improve the visual ERP-based BCI under different mental workload states and to alleviate the problems above. The rationality of our method is that the ERP-based BCI is based on electroencephalography (EEG) signals recorded from the scalp's surface, continuously changing with time and somewhat stochastic. In this paper, we propose a hierarchical recurrent network to encode all ERP signals in each repetition at the same time and model them with a temporal manner to predict which visual event elicited an ERP. The hierarchical architecture is a simple yet effective method for organizing recurrent layers in a deep structure to model long sequence signals. Taking a cue from recent advances in adversarial training, we further applied dynamic adversarial perturbations to create adversarial examples to enhance the model performance. We conduct our experiments on one published visual ERP-based BCI task with 15 subjects and 3 different auditory workload states. The results indicate that our hierarchical method can effectively model the long sequence EEG raw data, outperform the baselines on most conditions, including cross-state and cross-subject conditions. Finally, we show how deep learning-based methods with limited EEG data can improve ERP-based BCI with adversarial training. Our code is available at https://github.com/aispeech-lab/VisBCI.

Recently, our proposed speaker extraction model, WASE (learning When to Attend for Speaker Extraction) yielded superior performance over the prior state-of-the-art methods by explicitly modeling onset clue and regarding it as important guidance in speaker extraction tasks. However, it still remains challenging when it comes to the deployments on the resource-constrained devices, where the model must be tiny and fast to perform inference with minimal budget in CPU and memory while keeping the speaker extraction performance. In this work, we utilize model compression techniques to alleviate the problem and propose a lightweight speaker extraction model, TinyWASE, which aims to run on resource-constrained devices. Specifically, we mainly investigate the grouping effects of quantization-aware training and knowledge distillation techniques in the speaker extraction task and propose Distillation-aware Quantization. Experiments on WSJ0-2mix dataset show that our proposed model can achieve comparable performance as the full-precision model while reducing the model size using ultra-low bits (e.g. 3 bits), obtaining 8.97x compression ratio and 2.15 MB model size. We further show that TinyWASE can combine with other model compression techniques, such as parameter sharing, to achieve compression ratio as high as 23.81 with limited performance degradation. Our code is available at https://github.com/aispeech-lab/TinyWASE .

Real-world sounds are often interrupted by various kinds of noise. The target signal of the mixture sounds is often degraded or lost. While the human auditory system can extract the target signal from the mixture and restore the degraded or lost parts simultaneously, current computational models often simplify the complex scenarios, which leads to two individual tasks, audio inpainting and speech enhancement. In this work, we take a pioneering step towards modeling auditory restoration, that is to restore the target speech signal, in which there are missing parts in the target signal and the target signal is interfered by background noise. Different from the speech enhancement task, we attempt to fill in the missing gaps with the existence of background noise. Different from the auditory inpainting task, there is some noise in our input signal and the positions of the missing gaps are unknown. In other words, we attempt to reduce interference and restore missing gaps simultaneously. We propose Hourglass-shaped Convolutional Recurrent Network (HCRN) trained with Spectro- Temporal loss to restore the target signal from the incomplete noisy mixture. Moreover, instead of restoring non-human sounds, we focus on speech restoration, which poses more challenges on reconstruction. Both the quantitative and qualitative performance show that our proposed method can suppress the background noise, identify and restore the missing gaps of the salient signal with the unreliable context information. Our code is available in https://github.com/aispeech-lab/HC $\text{RN}$.

Most speech separation methods, trying to separate all channel sources simultaneously, are still far from having enough general- ization capabilities for real scenarios where the number of input sounds is usually uncertain and even dynamic. In this work, we employ ideas from auditory attention with two ears and propose a speaker and direction inferred speech separation network (dubbed SDNet) to solve the cocktail party problem. Specifically, our SDNet first parses out the respective perceptual representations with their speaker and direction characteristics from the mixture of the scene in a sequential manner. Then, the perceptual representations are utilized to attend to each corresponding speech. Our model gener- ates more precise perceptual representations with the help of spatial features and successfully deals with the problem of the unknown number of sources and the selection of outputs. The experiments on standard fully-overlapped speech separation benchmarks, WSJ0- 2mix, WSJ0-3mix, and WSJ0-2&3mix, show the effectiveness, and our method achieves SDR improvements of 25.31 dB, 17.26 dB, and 21.56 dB under anechoic settings. Our codes will be released at https://github.com/aispeech-lab/SDNet.

In the speaker extraction problem, it is found that additional information from the target speaker contributes to the tracking and extraction of the target speaker, which includes voiceprint, lip movement, facial expression, and spatial information. However, no one cares for the cue of sound onset, which has been emphasized in the auditory scene analysis and psychology. Inspired by it, we explicitly modeled the onset cue and verified the effectiveness in the speaker extraction task. We further extended to the onset/offset cues and got performance improvement. From the perspective of tasks, our onset/offset-based model completes the composite task, a complementary combination of speaker extraction and speaker-dependent voice activity detection. We also combined voiceprint with onset/offset cues. Voiceprint models voice characteristics of the target while onset/offset models the start/end information of the speech. From the perspective of auditory scene analysis, the combination of two perception cues can promote the integrity of the auditory object. The experiment results are also close to state-of-the-art performance, using nearly half of the parameters. We hope that this work will inspire communities of speech processing and psychology, and contribute to communication between them. Our code will be available in https: //github.com/aispeech-lab/wase/.

Audio-visual speech separation (AVSS) refers to separating individual voice from an audio mixture of multiple simultaneous talkers by conditioning on visual features. For the AVSS task, visual features play an important role, based on which we manage to extract more effective visual features to improve the performance. In this paper, we propose a novel AVSS model that uses speech-related visual features for isolating the target speaker. Specifically, the method of extracting speech-related visual features has two steps. Firstly, we extract the visual features that contain speech-related information by learning joint audio-visual representation. Secondly, we use the adversarial training method to enhance speech-related information in visual features further. We adopt the time-domain approach and build audio-visual speech separation networks with temporal convolutional neural networks block. Experiments on four audio-visual datasets, including GRID, TCD-TIMIT, AVSpeech, and LRS2, show that our model significantly outperforms previous state-of-the-art AVSS models. We also demonstrate that our model can achieve excellent speech separation performance in noisy realworld scenarios. Moreover, in order to alleviate the performance degradation of AVSS models caused by the missing of some video frames, we propose a training strategy, which makes our model robust when video frames are partially missing. The demo, code, and supplementary materials can be available at https://github.com/aispeech-lab/advr-avss.

Crop evapotranspiration (ET) is an essential part of agricultural water consumption, and robust monitoring of remote sensing (RS)-based ET at the field scale improves agricultural water management against water shortages. In this study, we propose a high-resolution optical RS-driven daily ET estimation framework coupling water vaporization and carbon assimilation based on Sentinel-2 satellite data. To determine if the proposed framework is accurate compared with flux observations, three tower sites are chosen (Guantao and Huailai from the Haihe Basin; Daman from the Heihe Basin), with a total of four years of observations adopted for model validation. The correlation coefficient R ranges from 0.870 to 0.912, and the RMSE ranges from 0.89 to 1.21 mm/day. Sensitivity analyses indicate that ET is most sensitive to air temperature, followed by ambient CO2 concentration and absorbed shortwave radiation, which provides indications into potential future farming strategies to confront global climate change. Finally, we discuss the scale effects on the proposed model at the field scale. Results from three sites show that for a larger area of interest (AOI) the impact of scales increases. This research provides insights into ET calculations across several spatial scales and application potential in precision agricultural water management.