Reinforcement learning from human feedback (RLHF) has succeeded remarkably in large language models (LLMs) alignment. However, in the standard LLMs RLHF method, a distribution mismatch exists between the generation policy and reward model (RM) when the policy continuously improves, but the RM remains unchanged. Moreover, the static RM may restrict the performance ceiling of the optimized policy. Furthermore, gathering high-quality human preference datasets to train RM can be time-consuming and expensive. To solve the above limitations, we propose a reinforcement learning from human feedback with cooperative policy-reward optimization (CoRLHF) method to improve RLHF. CoRLHF optimizes the policy and RM iteratively and automatically. In CoRLHF, the preference datasets of training RM are collected by the consensus scores of policy and RM on policy experiences, and the policy is trained by the iteratively improved RM. The iterative and automatic cooperative policy-RM optimization loop can improve the performance ceiling of the optimized policy. Furthermore, we show that CoRLHF is a promising method for weak-to-strong RLHF, and CoRLHF can be considered a bridge between RLHF and reinforcement learning from artificial intelligence feedback. Experimental results show that CoRLHF outperforms RLHF in alignment with human preferences.

In this paper, we aim to combine the advantages of existing sequence parallelism paradigms and overcomes their drawbacks, the most serious of which is the incapability to correctly compute causal attention on the hybrid-context packed sequences, in a stronger sequence parallelism framework. The practical technique of packing sequences for efficiently pretraining and fine-tuning large language models causes cross-contamination problem in attention computation, which can be effectively solved when no parallelism in the sequence length dimension is taken. However, in sequence parallelism, existing approaches either ignore the scenario of hybrid-context sequences or conversely sacrifice and limit parallelism degree for supporting the scenario. To this end, we innovatively propose an efficient Sequence-Aware Parallelism algorithm to conquer the obstacles of intensive tensor transmission and partial attention computation across multiple device groups. Our algorithm utilizes JIT (Just-In-Time) compilation to optimize the communication strategy of all device groups in NCCL level. Further, we integrate existing sequence parallelism paradigms into a Hierarchical Sequence-Aware Parallelism framework which benefits from our sequence-aware algorithm. We additionally elaborate on the memory and communication overhead management of the hierarchical framework to optimize its performance. Through multiple experiments, we demonstrate that our proposed approach outperform other state-of-the-arts sequence parallelism approches in multiple metrics.

Modern Large Foundation Model (LFM) training has transformed the data pipeline from a static ingestion layer into a dynamic component that must co-evolve with the training process. Existing systems are ill-equipped: colocated dataloaders offer no failure isolation, while message queue-based disaggregated dataloaders operate on a record/offset abstraction that cannot express the batch-level semantics required by distributed training. We present BatchWeave, an object-store-native training data plane for distributed LFM training. BatchWeave uses versioned manifests and conditional object writes to coordinate batch publication, recovery, and lifecycle management. First, it introduces the Transactional Global Batch (TGB), which builds on versioned-manifest ACID storage semantics and extends them with training-specific consistency, including atomic all-rank batch visibility, a globally ordered step sequence, checkpoint-aligned lifecycle management, and end-to-end exactly-once recovery. Second, it realizes recovery and retention directly in the storage layer, by durably persisting producer state through the commit protocol and tying reclamation to distributed checkpoint state. Third, its Decentralized Adaptive Commit (DAC) algorithm sustains stable ingestion throughput as the manifest grows, without any inter-producer communication. Evaluations on large-scale multimodal pre-training and SFT workloads using 64 GPUs show that BatchWeave outperforms colocated dataloader throughput while providing full failure isolation, outperforms Apache Kafka in ingestion throughput, and achieves lower consumer read latency than Kafka.

VLA architectures that pair a pretrained VLM with a flow-matching action expert have emerged as a strong paradigm for language-conditioned manipulation. Yet the VLM, optimized for semantic abstraction and typically conditioned on static visual observations, tends to attenuate fine-grained geometric cues and often lacks explicit temporal evidence for the action expert. Prior work mitigates this by injecting auxiliary visual features, but existing approaches either focus on static spatial representations or require substantial architectural modifications to accommodate temporal inputs, leaving temporal information underexplored. We propose Plug-in Visual Injection (PVI), a lightweight, encoder-agnostic module that attaches to a pretrained action expert and injects auxiliary visual representations via zero-initialized residual pathways, preserving pretrained behavior with only single-stage fine-tuning. Using PVI, we obtain consistent gains over the base policy and a range of competitive alternative injection strategies, and our controlled study shows that temporal video features (V-JEPA2) outperform strong static image features (DINOv2), with the largest gains on multi-phase tasks requiring state tracking and coordination. Real-robot experiments on long-horizon bimanual cloth folding further demonstrate the practicality of PVI beyond simulation.

Nitrate (NO3-) is one of the major forms of nitrogen element and an essential nutrient for maintenance of crop yield and quality. Suboptimal nitrate use efficiency of traditional fertilizers necessitates the effective retention and slow-release capability, which is crucial for sustainable agriculture and cost saving. Here, the MgFe-layered double hydroxide (MgFe-LDH) assembled onto corn stalk derived biochar (MgFe-LDH@BC) composites were synthesized to serve as an effective slow-release carrier for nitrate. Batch microscopic characterizations demonstrated a nano-sized lamellar structure successfully formed onto the intricate pore skeleton of biochar (BC) with large specific surface area, abundant defect structure and good thermal stability. MgFe-LDH@BC exhibited a remarkable nitrate adsorption capacity of 41.72 mg/g. Anion exchange and electrostatic attraction dominated the adsorption process. After nitrate introducing, MgFe-LDH@BC could continuously release nitrate into water and reach a slow-release performance of 70.96 % within 144 h. The cumulative release rate and release span of nitrate-loaded MgFe-LDH@BC were profoundly influenced by temperature, pH of the aqueous solution and moisture content in soil. It improved the soil nitrate concentration by 198 % (30 % moisture) and 224 % (80 % moisture) relative to the control. Artificial neural network (ANN) model with higher R2 values (>0.97) was established and applied to predict the release data. Furthermore, the main framework and crystalline structure of MgFe-LDH@BC remained largely intact after successive steps of nitrate loading, restoring and releasing over a period of 6 days. These results illustrate MgFe-LDH@BC may work as a slow-release carrier for minimizing the nitrate loss to the environment.

Generating emotionally appropriate responses in conversations with large language models presents a significant challenge due to the complexities of human emotions and cognitive processes, which remain largely underexplored in their critical role in social interactions. In this study, we introduce a two-stage automatic data generation framework to create CAPE, a Chinese dataset named Cognitive Appraisal theory-based Emotional corpus. This corpus facilitates the generation of dialogues with contextually appropriate emotional responses by accounting for diverse personal and situational factors. We propose two tasks utilizing this dataset: emotion prediction and next utterance prediction. Both automated and human evaluations demonstrate that agents trained on our dataset can deliver responses that are more aligned with human emotional expressions. Our study shows the potential for advancing emotional expression in conversational agents, paving the way for more nuanced and meaningful human-computer interactions.

The rapid advancement of Large Language Models (LLMs) has provided new opportunities for edge applications. Embedded FPGA platforms are well-suited for deploying LLMs at the edge due to their hardware programmability and high energy efficiency. However, the limited hardware resources of embedded systems pose significant challenges to comprehensive acceleration. As not all operators can be offloaded to programmable logic (PL), the remaining operations must execute on the processing system (PS), creating potential performance bottlenecks and degrading inference efficiency. In this paper, we introduce Lembda, a collaborative optimization framework that harnesses the computational capability of both PL and PS for efficient embedded LLM inference. On the PL side, we employ W4A8 quantization and implement high-throughput GEMM/GEMV kernels. On the PS side, we optimize high-precision operations by exploiting model sparsity in attention layers and approximate nonlinear functions via lightweight polynomial fitting. Moreover, we carefully orchestrate PL/PS operations to exploit operation-level parallelism and further enhance performance. Evaluations on the AMD Kria KV260 platform demonstrates that Lembda delivers 187.9195 tok/s for prefilling and 9.7857 tok/s for decoding on the Qwen2.5-0.5B-Instruct model, achieving 65.9× / 3.8× speedup compared to the baseline method with negligible accuracy loss.

The development of digital twins for urban road intersections has attracted significant attention in recent research, driven by their potential to enable intelligent intersections that require accurate and real-time monitoring of dynamic environments. Lidar sensors, with their precise 3D perception capabilities, are particularly well-suited for enhancing intersection perception, offering accurate detection and tracking of road objects, as well as precise velocity estimation. However, existing roadside lidar datasets are deficient in terms of scene diversity and annotation information specifically tailored for intersection monitoring applications. To address the gap, we introduce IntDT, a comprehensive roadside lidar dataset designed specifically for digital twins of intersections. Collected using dual roadside lidar systems across diverse urban intersections, IntDT provides comprehensive annotations that support a range of tasks. Benchmark evaluations are performed for 3D object detection and tracking tasks to facilitate standardized comparative analysis. Additionally, semi-supervised object detection, moving object segmentation, and point cloud completion tasks are firstly benchmarked for roadside lidar data to stimulate more studies in this field. The dataset and code will be released at https://github.com/Geo3DSmart/IntDT.

The emergence of multi-codebook neutral audio codecs such as Residual Vector Quantization (RVQ) and Group Vector Quantization (GVQ) has significantly advanced Large-Language-Model (LLM) based Text-to-Speech (TTS) systems. These codecs are crucial in separating semantic and acoustic information while efficiently harnessing semantic priors. However, since semantic and acoustic information cannot be fully aligned, a significant drawback of these methods when applied to LLM-based TTS is that large language models may have limited access to comprehensive audio information. To address this limitation, we propose DistilCodec and UniTTS, which collectively offer the following advantages: 1) This method can distill a multi-codebook audio codec into a single-codebook audio codec with 32,768 codes while achieving a near 100% utilization. 2) As DistilCodec does not employ a semantic alignment scheme, a large amount of high-quality unlabeled audio (such as audiobooks with sound effects, songs, etc.) can be incorporated during training, further expanding data diversity and broadening its applicability. 3) Leveraging the comprehensive audio information modeling of DistilCodec, we integrated three key tasks into UniTTS's pre-training framework: audio modality autoregression, text modality autoregression, and speech-text cross-modal autoregression. This allows UniTTS to accept interleaved text and speech/audio prompts while substantially preserving LLM's text capabilities. 4) UniTTS employs a three-stage training process: Pre-Training, Supervised Fine-Tuning (SFT), and Alignment. Source code and model checkpoints are publicly available at https://github.com/IDEA-Emdoor-Lab/UniTTS and https://github.com/IDEA-Emdoor-Lab/DistilCodec.

Long-term memory is important for chatbots and dialogue systems (DS) to create consistent and human-like conversations, evidenced by numerous developed memory-augmented DS (MADS). To evaluate the effectiveness of such MADS, existing commonly used evaluation metrics, like retrieval accuracy and perplexity (PPL), mainly focus on query-oriented factualness and language quality assessment. However, these metrics often lack practical value. Moreover, the evaluation dimensions are insufficient for human-like assessment in DS. Regarding memory-recalling paradigms, current evaluation schemes only consider passive memory retrieval while ignoring diverse memory recall with rich triggering factors, e.g., emotions and surroundings, which can be essential in emotional support scenarios. To bridge the gap, we construct a novel Memory-Augmented Dialogue Benchmark (MADail-Bench) covering various memory-recalling paradigms based on cognitive science and psychology theories. The benchmark assesses two tasks separately: memory retrieval and memory recognition with the incorporation of both passive and proactive memory recall data. We introduce new scoring criteria to the evaluation, including memory injection, emotion support (ES) proficiency, and intimacy, to comprehensively assess generated responses. Results from cutting-edge embedding models and large language models on this benchmark indicate the potential for further advancement. Extensive testing further reveals correlations between memory injection, ES proficiency, and intimacy.

Training large language models (LLMs) to act as autonomous agents for multi-turn, long-horizon tasks remains significant challenges in scalability and training efficiency. To address this, we introduce L-Zero (L0), a scalable, end-to-end training pipeline for general-purpose agents. Featuring a low-cost, extensible, and sandboxed concurrent agent worker pool, L0 lowers the barrier for applying reinforcement learning in complex environments. We also introduce NB-Agent, the agent scaffold within L0, which operates in a "code-as-action" fashion via a Read-Eval-Print-Loop (REPL). We evaluate L0 on factuality question-answering benchmarks. Our experiments demonstrate that a base model can develop robust problem-solving skills using solely Reinforcement Learning with Verifiable Rewards (RLVR). On the Qwen2.5-7B-Instruct model, our method boosts accuracy on SimpleQA from 30

Recent advances in GUI agents have achieved remarkable grounding and action-prediction performance, yet existing models struggle with unreliable reward signals and limited online trajectory generation. In this paper, we introduce Orcust, a framework that integrates Principle-Constrained Reward Modeling (PCRM) and Online VM-Grounded Trajectory Construction (OVTC) to enhance reasoning reliability and data efficiency in interactive GUI tasks. We leverages environment-verifiable and LLM-derived principle to enforce interpretable reward signals that constrain long chain-of-thought reasoning and rule-based feedback. OVTC spins up instrumented virtual machines to autonomously collect structured GUI interaction trajectories with explicit procedural and structural objectives, enabling the training of a stepwise reward model that robustly captures human preferences and adheres to task-specific constraints. Extensive experiments on standard GUI benchmarks covering perceptual grounding, foundational operations, and end-to-end task execution reveal that Orcust achieves state-of-the-art performance, improving by 22.2% on ScreenSpot and 23.9% on ScreenSpot-Pro over the base model (i.e. Qwen2.5-VL-7B). The results demonstrate Orcust's effectiveness in enhancing the reasoning, adaptability and scalability of GUI agents across various environments and task complexities.

The anaerobic co-digestion of agricultural residues emerges as a promising strategy for energy recovery and nutrient recycling within circular agricultural systems. This study aimed to optimize co-digestion parameters for vinegar residue (VR) and cattle manure (CM) using an orthogonal experimental design. Three key variables were investigated which are the co-substrate ratio (VR to CM), feedstock-to-inoculum (F/I) ratio, and total solids (TS) content. Nine experimental combinations were tested to evaluate methane yield, feedstock degradation, and digestate characteristics. Results showed that the optimal condition for methane yield comprised a 2:3 co-substrate ratio, 1:2 F/I ratio, and 20% TS, achieving the highest methane yield of 267.84 mL/g volatile solids (VS) and a vs. degradation rate of 58.65%. Digestate analysis indicated this condition generated the most nutrient-rich liquid digestate and solid digestate, featuring elevated N, P, and K concentrations, acceptable seed germination indices (GI), and moderate humification levels. While total nutrient content did not meet commercial organic fertilizer standards, the digestate is suitable for direct land application in rural settings. This study underscores the need to balance energy recovery and fertilizer quality in anaerobic co-digestion systems, providing practical guidance for decentralized biogas plants seeking to integrate waste treatment with agricultural productivity.

Recent advancements in text-to-image models have significantly enhanced image generation capabilities, yet a notable gap of open-source models persists in bilingual or Chinese language support. To address this need, we present Taiyi-Diffusion-XL, a new Chinese and English bilingual text-to-image model which is developed by extending the capabilities of CLIP and Stable-Diffusion-XL through a process of bilingual continuous pre-training. This approach includes the efficient expansion of vocabulary by integrating the most frequently used Chinese characters into CLIP's tokenizer and embedding layers, coupled with an absolute position encoding expansion. Additionally, we enrich text prompts by large vision-language model, leading to better images captions and possess higher visual quality. These enhancements are subsequently applied to downstream text-to-image models. Our empirical results indicate that the developed CLIP model excels in bilingual image-text retrieval.Furthermore, the bilingual image generation capabilities of Taiyi-Diffusion-XL surpass previous models. This research leads to the development and open-sourcing of the Taiyi-Diffusion-XL model, representing a notable advancement in the field of image generation, particularly for Chinese language applications. This contribution is a step forward in addressing the need for more diverse language support in multimodal research. The model and demonstration are made publicly available at \href{https://huggingface.co/IDEA-CCNL/Taiyi-Stable-Diffusion-XL-3.5B/}{this https URL}, fostering further research and collaboration in this domain.

Recently, the increasing demand for superior medical services has highlighted the discrepancies in the medical infrastructure. With big data, especially texts, forming the foundation of medical services, there is an exigent need for effective natural language processing (NLP) solutions tailored to the healthcare domain. Conventional approaches leveraging pre-trained models present promising results in this domain and current large language models (LLMs) offer advanced foundation for medical text processing. However, most medical LLMs are trained only with supervised fine-tuning (SFT), even though it efficiently empowers LLMs to understand and respond to medical instructions but is ineffective in learning domain knowledge and aligning with human preference. In this work, we propose ChiMed-GPT, a new benchmark LLM designed explicitly for Chinese medical domain, and undergoes a comprehensive training regime with pre-training, SFT, and RLHF. Evaluations on tasks including information extraction, question answering, and dialogue generation demonstrate ChiMed-GPT's superior performance over general domain LLMs. Furthermore, we analyze possible biases through prompting ChiMed-GPT to perform attitude scales regarding discrimination of patients, so as to contribute to further responsible development of LLMs in the medical domain. The code and model are released at https://github.com/synlp/ChiMed-GPT.

Large Vision Language Models (LVLMs) have demonstrated impressive zero-shot capabilities in various vision-language dialogue scenarios. However, the absence of fine-grained visual object detection hinders the model from understanding the details of images, leading to irreparable visual hallucinations and factual errors. In this paper, we propose Lyrics, a novel multi-modal pre-training and instruction fine-tuning paradigm that bootstraps vision-language alignment from fine-grained cross-modal collaboration. Building on the foundation of BLIP-2, Lyrics infuses local visual features extracted from a visual refiner that includes image tagging, object detection and semantic segmentation modules into the Querying Transformer, while on the text side, the language inputs equip the boundary boxes and tags derived from the visual refiner. We further introduce a two-stage training scheme, in which the pre-training stage bridges the modality gap through explicit and comprehensive vision-language alignment targets. During the instruction fine-tuning stage, we introduce semantic-aware visual feature extraction, a crucial method that enables the model to extract informative features from concrete visual objects. Our approach achieves robust performance on 13 datasets across various vision-language tasks, and demonstrates promising multi-modal understanding, perception and conversation capabilities in 11 scenario-based benchmark toolkits.

While large language models (LLMs) are equipped with longer text input capabilities than before, they struggle to seek correct information in long contexts. The "lost in the middle" problem challenges most LLMs, referring to the dramatic decline in accuracy when correct information is located in the middle. To overcome this crucial issue, this paper proposes to enhance the information searching and reflection ability of LLMs in long contexts via specially designed tasks called PositionAgnostic Multi-step QA (PAM QA). Trained with this task, our model excels in focusing more precisely on the desired information. Experimental results show substantial improvement in Multi-doc QA and other benchmarks, surpassing state-of-the-art models by a 13.7% absolute gain in shuffled settings and by 21.5% in the passage retrieval task. We release our model and code to promote related research in the community.

Root rot is one of the main reasons for yield losses of red kidney bean (Phaseolus vulgaris) production. Pre-inoculation with Trichoderma harzianum can effectively lower the incidence of red kidney bean root rot. In this study, four treatments including CK (control), Fu13 (Fusarium oxysporum), T891 (T. harzianum) and T891 + Fu13 (T. harzianum + F. oxysporum) were arranged in a pot experiment to investigate how T891 affected the incidence and severity of root rot, plant growth, and changes of defense enzyme activity in red kidney bean plants. Community composition and diversity of the rhizosphere microbiota was evaluated through high-throughput sequencing, and co-occurrence network was analyzed. The results showed that when compared to the Fu13 treatment, pre-inoculation with T891 reduced the incidence and severity of red kidney bean root rot by 40.62 and 68.03% (p < 0.05), increased the root length, shoot length, total dry biomass by 48.63, 97.72, 122.17%. Upregulated activity of super-oxide dismutase (SOD), peroxidase (POD), catalase (CAT) by 7.32, 38.48, 98.31% (p < 0.05), and reduced malondialdehyde (MDA) by 23.70% (p < 0.05), respectively. Microbiological analyses also showed that F. oxysporum reduced alpha diversity resulting in alteration the composition of the rhizosphere microbial community in red kidney bean. T891 significantly reduced abundance of F. oxysporum, allowing the enrichment of potentially beneficial bacteria Porphyrobacter (ASV 46), Lysobacter (ASV 85), Microbacteriaceae (ASV 105), and Gemmatimonas (ASV 107), resulting in a more stable structure of the microbial network. The results of random forest analysis further revealed that ASV 46 (Porphyrobacter) was the primary influencing factor for the incidence of root rot after inoculation with T891, while ASV 85 (Lysobacter) was the primary influencing factor for the biomass of red kidney bean. In conclusion, T. harzianum promotes the growth of red kidney bean and inhibits root rot by improving plant antioxidant enzyme activity and regulating the rhizosphere microbial community.

Distillers' grains has high potential in methane production but the process is affected by the acidification in anaerobic digestion. To enhance the methane production in solid-state anaerobic digestion, distillers' grains was co-digested with cattle manure in different proportions in this study. Compared to distillers' grains, cattle manure performed lower biodegradability but higher alkalinity and nutrients, hence relieved acidification and drove microbial succession for methane enhancement in the co-digestion with distillers' grains. The highest increase in total methane yield (10.67 %) was observed in the co-digestion of distillers' grains and cattle manure at the proportion of 70:30 (T3). This was the synergistic action of reduced acidification, relatively high biodegradability of feedstocks, and balanced composition of hydrolytic bacteria, syntrophic bacteria and acetoclastic methanogens. Further analysis revealed the positive effects of indigenous microorganisms derived from distillers' grains and cattle manure in methane enhancement. The enrichment of Sedimentibacter, Desulfitibacter, Tepidanaerobacter from distiller's grains, and Eubacterium, Methanoculleus, Methanobacterium, Methanobrevibacter mainly from cattle manure synergistically promoted organics conversion and methane production in T3. This study further affirmed the potential use of distillers' grains in solid-state anaerobic digestion and provided a favorable strategy for its better methane production.