Sequential recommendation (SR) has received much attention and made promising progress in the past few years due to its high alignment with real recommendation scenarios. It models users' preferences and behavior patterns from their historical behavior sequences and provides personalized recommendations. However, the widespread problem of data sparsity limits the performance of sequential recommendation models. To tackle this, data augmentation (DA) provides a feasible solution by improving the quantity, quality, or diversity of the training samples without the need for additional data collection. In this survey, we present a systematic and timely review of research efforts on data augmentation for sequential recommendation. We start by providing a clear formulation of the problem and task. Then, we develop a unified taxonomy that categorizes existing augmentation methodologies regarding their augmentation objects and principles. Next, we conduct a comparative discussion on the advantages and disadvantages of different categories, supplemented with quantitative performance evaluations, time-complexity analyses, and visual case studies of representative methods, aiming to provide actionable guidance for the selection and development of augmentation methods in real-world scenarios. Finally, we present the future research directions and summarize this survey.

Large language models (LLMs) are increasingly applied in scientific research, offering new capabilities for knowledge discovery and reasoning. In single-cell biology, however, evaluation practices for both general and specialized LLMs remain inadequate: existing benchmarks are fragmented across tasks, adopt formats such as multiple-choice classification that diverge from real-world usage, and rely on metrics lacking interpretability and biological grounding. We present SC-ARENA, a natural language evaluation framework tailored to single-cell foundation models. SC-ARENA formalizes a virtual cell abstraction that unifies evaluation targets by representing both intrinsic attributes and gene-level interactions. Within this paradigm, we define five natural language tasks (cell type annotation, captioning, generation, perturbation prediction, and scientific QA) that probe core reasoning capabilities in cellular biology. To overcome the limitations of brittle string-matching metrics, we introduce knowledge-augmented evaluation, which incorporates external ontologies, marker databases, and scientific literature to support biologically faithful and interpretable judgments. Experiments and analysis across both general-purpose and domain-specialized LLMs demonstrate that (i) under the Virtual Cell unified evaluation paradigm, current models achieve uneven performance on biologically complex tasks, particularly those demanding mechanistic or causal understanding; and (ii) our knowledge-augmented evaluation framework ensures biological correctness, provides interpretable, evidence-grounded rationales, and achieves high discriminative capacity, overcoming the brittleness and opacity of conventional metrics. SC-Arena thus provides a unified and interpretable framework for assessing LLMs in single-cell biology, pointing toward the development of biology-aligned, generalizable foundation models.

Model merging is an efficient empowerment technique in the machine learning community that does not require the collection of raw training data and does not require expensive computation. As model merging becomes increasingly prevalent across various fields, it is crucial to understand the available model merging techniques comprehensively. However, there is a significant gap in the literature regarding a systematic and thorough review of these techniques. This survey provides a comprehensive overview of model merging methods and theories, their applications in various domains and settings, and future research directions. Specifically, we first propose a new taxonomic approach that exhaustively discusses existing model merging methods. Secondly, we discuss the application of model merging techniques in large language models, multimodal large language models, and more than ten machine learning subfields, including continual learning, multi-task learning, few-shot learning, and so on. Finally, we highlight the remaining challenges of model merging and discuss future research directions. A comprehensive list of papers about model merging is available at https://github.com/ EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications.

Sub-sequence splitting (SSS) has been demonstrated as an effective approach to mitigate data sparsity in sequential recommendation (SR) by splitting a raw user interaction sequence into multiple sub-sequences. Previous studies have demonstrated its ability to enhance the performance of SR models significantly. However, in this work, we discover that (i). SSS may interfere with the evaluation of the model's actual performance. We observed that many recent state-of-the-art SR models employ SSS during the data reading stage (not mentioned in the papers). When we removed this operation, performance significantly declined, even falling below that of earlier classical SR models. The varying improvements achieved by SSS and different splitting methods across different models prompt us to analyze further when SSS proves effective. We find that (ii). SSS demonstrates strong capabilities only when specific splitting methods, target strategies, and loss functions are used together. Inappropriate combinations may even harm performance. Furthermore, we analyze why sub-sequence splitting yields such remarkable performance gains and find that (iii). it evens out the distribution of training data while increasing the likelihood that different items are targeted. Finally, we provide suggestions for overcoming SSS interference, along with a discussion on data augmentation methods and future directions. We hope this work will prompt the broader community to re-examine the impact of data splitting on SR and promote fairer, more rigorous model evaluation. All analysis code and data will be made available upon acceptance. We provide a simple, anonymous implementation at https://github.com/KingGugu/SSS4SR.

The integration of Large Language Models (LLMs) into web applications is challenged by dynamic online content that quickly renders their parametric knowledge obsolete. While In-Context Knowledge Editing (ICE) offers a promising solution, existing methods often rely on structured knowledge triples and require training auxiliary models. This restricts their effectiveness in unstructured natural language environments and introduces computational overhead. To address these limitations, we propose DeCO, a training-free framework that operates directly on natural language from external sources. We identify that Chain-of-Thought (CoT) generates reasoning steps in a single forward pass, hindering the incorporation of corrections. DeCO reformulates CoT as an incremental process of generation and editing, where each step is dynamically revised using retrieved knowledge to guide the model with finer granularity. Furthermore, DeCO tackles two critical challenges: (1) the inaccuracy of semantic similarity-based retrieval, and (2) the tendency of LLMs to revert to parametric knowledge or refuse responses under counterfactual conditions. We introduce the Information Seeker, a two-stage module leveraging LLM semantics for precise retrieval, and the Dynamically Prefill Guidance Strategy, which utilizes autoregressive constraints to enforce instruction adherence. Extensive evaluations demonstrate that DeCO outperforms six strong baselines, achieving average gains of 23.3% on multi-hop editing benchmarks and 4.5% on comprehensive benchmarks, underscoring its broad applicability.

Test-time reinforcement learning generates multiple candidate answers via repeated rollouts and performs online updates using pseudo-labels constructed by majority voting. To reduce overhead and improve exploration, prior work introduces tree-structured rollouts, which share reasoning prefixes and branch at key nodes to improve sampling efficiency. However, this paradigm still faces two challenges: (1) high-entropy branching can trigger rollout collapse, where the branching budget concentrates on a few trajectories with consecutive high-entropy segments, rapidly reducing the number of effective branches; (2) early pseudo-labels are noisy and biased, which can induce self-reinforcing overfitting, causing the policy to sharpen prematurely and suppress exploration. To address these issues, we propose Entropy–Confidence Hybrid Group Relative Policy Optimization (ECHO). During rollout, ECHO jointly leverages local entropy and group-level confidence to adaptively control branch width, and further introduces online confidence-based pruning to terminate persistently low-confidence branches, avoiding high-entropy traps and mitigating collapse. During policy updates, ECHO employs confidence-adaptive clipping and an entropy–confidence hybrid advantage shaping approach to enhance training robustness and mitigate early-stage bias. Experiments demonstrate that ECHO achieves consistent gains on multiple mathematical and visual reasoning benchmarks, and generalizes more effectively under a limited rollout budget.

Sequential Recommendation (SR) learns user preferences from their historical interaction sequences and provides personalized suggestions. In real-world scenarios, most items exhibit sparse interactions, known as the tail-item problem. This issue limits the model's ability to accurately capture item transition patterns. To tackle this, large language models (LLMs) offer a promising solution by capturing semantic relationships between items. Despite previous efforts to leverage LLM-derived embeddings for enriching tail items, they still face the following limitations: 1) They struggle to effectively fuse collaborative signals with semantic knowledge, leading to suboptimal item embedding quality. 2) Existing methods overlook the structural inconsistency between the ID and LLM embedding spaces, causing conflicting signals that degrade recommendation accuracy. In this work, we propose a Fusion and Alignment Enhancement framework with LLMs for Tail-item Sequential Recommendation (FAERec), which improves item representations by generating coherently-fused and structurally consistent embeddings. For the information fusion challenge, we design an adaptive gating mechanism that dynamically fuses ID and LLM embeddings. Then, we propose a dual-level alignment approach to mitigate structural inconsistency. The item-level alignment establishes correspondences between ID and LLM embeddings of the same item through contrastive learning, while the feature-level alignment constrains the correlation patterns between corresponding dimensions across the two embedding spaces. Furthermore, the weights of the two alignments are adjusted by a curriculum learning scheduler to avoid premature optimization of the complex feature-level objective. Extensive experiments across three widely used datasets with multiple representative SR backbones demonstrate the effectiveness and generalizability of our framework.

Data augmentation (DA) has been proven to be an effective means for improving protein representation learning (PRL) by generating additional training samples. Although mainstream perturbation- and sampling-based augmentation methods can produce data containing sufficient variations, they carry the risk of disrupting the protein structure and function. Some crafted protein homology modeling tools can generate conformations, but reduce structural diversity. The above dilemmas lead us to a question: Can we restore the disrupted structure caused by DA operations, providing data with both the original structure and diverse variations? In this work, we first analyze and empirically reveal the structure defect and performance degradation issues of existing DA methods. Based on the findings, we propose a simple yet effective DA method, Manifold Restore Mixing (MRM), for protein representation learning. Specifically, inspired by manifold mixup, we mix the hidden representations of original and augmented protein data to generate new samples that restore structural information lost in DA while introducing diverse variations. Furthermore, we develop a sample difficulty scheduler that adjusts the beta distribution in mixup to provide models with progressively challenging mixed samples during training, which improves the final performance. Comprehensive experiments on various PRL backbones and downstream tasks demonstrate the effectiveness and generalization of our method. The complete code and weights will be released upon acceptance. We provide a implementation at https://github.com/KingGugu/MRM.

Logical reasoning-based recommendation methods formulate logical expressions to characterize user-item interaction patterns, incorporating regularization constraints to ensure consistency with logical rules. However, these methods face two critical challenges: (1) As sequence length increases, they cannot effectively capture the dynamic transfer of user interests across subsequences (i.e., subsequence interest drift), thereby degenerating logical expressions to single-subsequence inference. (2) The time complexity of logical reasoning and rule learning scales quadratically with the sequence length, severely constraining computational efficiency in long-sequence recommendation. To address these challenges, we propose ELECTOR, an intErest-shift-aware long-sequence Logical reasoning for EffiCienT lOng-sequence Recommendation method. Specifically, we design a Subsequence Interest Learning Module (SIL) to model cross-subsequence interest drifts in long sequences. SIL employs a local attention mechanism to extract subsequence interests effectively and a global attention mechanism to capture the correlations among subsequence interests. Subsequently, we propose an Interest-aware Logical Reasoning (ILR) mechanism that performs logical reasoning using a limited set of subsequence and short-term interests, rather than reasoning over the entire sequence, significantly reducing time complexity. Additionally, ILR employs interest logical reasoning contrastive loss to ensure the model simultaneously considers multiple interests. Experiments on four real-world datasets demonstrate that our method significantly outperforms all baselines regarding computational efficiency and recommendation accuracy, confirming its effectiveness.

Data augmentation is an effective technique for tackling data sparsity in sequential recommendation (SR). Existing methods generate new data during the model training to improve the performance. However, deploying them on a backbone model requires retraining, architecture modification, or introducing additional modules and learnable parameters. These processes are time-consuming and costly for well-trained models, especially when the model and data scales become large. In this work, we explore the test-time augmentation (TTA) for SR, which augments the input sequences during the inference phase and then fuses the model's predictions to improve final accuracy. It avoids the significant overhead associated with training-time augmentation. We first experimentally examine the potential of existing augmentation operators for TTA and find that the Substitute and Mask consistently achieve better performance. Further analysis reveals that these two operators retain the original sequential pattern while adding appropriate perturbations. Moreover, the random selection of augmentation positions creates suitable augmented samples from both semantic and temporal perspectives. Meanwhile, we find that the fixed operation ratio limits the diversity of augmented data, and the TTA may impair the model's performance on long sequences. In addition, the two operators still face time-consuming similarity-based item selection or interference from mask tokens. Based on the analysis and limitations, we present TNoise and TMask. The former injects uniform noise into the representation, avoiding the computational overhead of item selection. The latter blocks mask tokens from participating in model calculations (TMask-B) or directly removes interactions that should have been replaced with mask tokens (TMask-R). Further, we sample the augmentation ratio from a uniform distribution to improve the data diversity. For short sequences, we introduce a sequence smoothing and lengthening method based on inter-item interpolation. For long sequences, we set a threshold to avoid the negative effects of TTA. Comprehensive experiments demonstrate the effectiveness, efficiency, and generalizability of our method.

Padding is a standard technique for training sequential recommendation models. It fills the short sequences to the same length so they can be fed into the model in batches. Traditional zero-padding uses the special value 0 as the padding content. However, zero-padding results in a significant waste of input space due to widespread data sparsity and the fact that the special value 0 is not involved in the model calculations. To tackle that, researchers propose repeated padding (RepPad) [1], which uses the original sequence as the padding content during training. Although this method turns out to be effective on many different types of sequential recommendation models, what underlies the performance gains is still a mystery. In this paper, we first review the operations of RepPad for input, positive, and negative sequences. Based on the operations, we decompose RepPad into twelve variants ranging from zero padding to full RepPad. Through an empirical study, we disclose that RepPad's effectiveness derives from performing repeated padding on input, positive sequences, and the extended random negative sequence. These three operations must exist simultaneously. Furthermore, we derive and analyze the twelve variants step-by-step from a loss function perspective, which helps clarify the changes brought about by RepPad compared to zero-padding during loss calculations. Based on these analyses, we refine the previous three operations into three reasons why RepPad is effective: maintaining sequence alignment and causality, increasing information density and diversity, and generating more negative samples in the single propagation. Based on these findings, we put forward the Repeated Padding with Extended Negative Sequence (RepPad-ENS) for sequential recommendation, which samples multiple extended random negative sequences based on RepPad. It unlocks the potential of RepPad by improving the quantity and diversity of negative samples in the single prop agation, thereby facilitating more accurate preference learning. Comprehensive experiments on various categories of baselines with real-world datasets demonstrate the effectiveness, efficiency, and generalizability of our method. Our code is available at https://github.com/KingGugu/RepPad-ENS.

Multi-task learning (MTL) leverages a shared model to accomplish multiple tasks and facilitate knowledge transfer. Recent research on task arithmetic-based MTL demonstrates that merging the parameters of independently fine-tuned models can effectively achieve MTL. However, existing merging methods primarily seek a static optimal solution within the original model parameter space, which often results in performance degradation due to the inherent diversity among tasks and potential interferences. To address this challenge, in this paper, we propose a Weight-Ensembling Mixture of Experts (WEMoE) method for multi-task model merging. Specifically, we first identify critical (or sensitive) modules by analyzing parameter variations in core modules of Transformer-based models before and after finetuning. Then, our WEMoE statically merges non-critical modules while transforming critical modules into a mixture-of-experts (MoE) structure. During inference, expert modules in the MoE are dynamically merged based on input samples, enabling a more flexible and adaptive merging approach. Building on WEMoE, we further introduce an efficient-and-effective WEMoE (E-WEMoE) method, whose core mechanism involves eliminating non-essential elements in the critical modules of WEMoE and implementing shared routing across multiple MoE modules, thereby significantly reducing both the trainable parameters, the overall parameter count, and computational overhead of the merged model by WEMoE. Experimental results across various architectures and tasks demonstrate that both WEMoE and E-WEMoE outperform state-of-the-art (SOTA) model merging methods in terms of MTL performance, generalization, and robustness.

Multi-subject video generation aims to synthesize videos from textual prompts and multiple reference images, ensuring that each subject preserves natural scale and visual fidelity. However, current methods face two challenges: scale inconsistency, where variations in subject size lead to unnatural generation, and permutation sensitivity, where the order of reference inputs causes subject distortion. In this paper, we propose MoFu, a unified framework that tackles both challenges. For scale inconsistency, we introduce Scale-Aware Modulation (SMO), an LLM-guided module that extracts implicit scale cues from the prompt and modulates features to ensure consistent subject sizes. To address permutation sensitivity, we present a simple yet effective Fourier Fusion strategy that processes the frequency information of reference features via the Fast Fourier Transform to produce a unified representation. Besides, we design a Scale-Permutation Stability Loss to jointly encourage scale-consistent and permutation-invariant generation. To further evaluate these challenges, we establish a dedicated benchmark with controlled variations in subject scale and reference permutation. Extensive experiments demonstrate that MoFu significantly outperforms existing methods in preserving natural scale, subject fidelity, and overall visual quality.

Natural language user preferences provide an interpretable interface for LLM personalization. However, universal preference summaries often contain information irrelevant to a particular downstream task. Directly supplying the full preference summary therefore wastes context capacity and introduces cross-task distraction, while manually designing task-specific preference views is difficult to scale. In this work, we study task-specific preference adaptation: given a universal user preference summary and a downstream task, derive a task-conditioned representation that preserves sufficient decision-relevant evidence while removing redundant context. To this end, we propose AlignXada, a training-free meta-learning framework that induces reusable textual refinement policies for adapting universal preference summaries to task-specific ones. The refinement policy is iteratively optimized by a meta learner through verbal reinforcement learning. Across 13 tasks and three downstream models (39 task–model cells), AlignXada achieves an average gain of 3.82 points, improving 33 cells while retaining only 22.8% of the original profile tokens and outperforming RAG in 36 cells. An extended faithfulness analysis further shows that the refined profiles remain largely grounded in the source preferences while preserving task-relevant personalization signals, suggesting that profile-side adaptation serves as a practical complement to universal memory construction for lifelong personalized agents.

Few-shot knowledge graph completion (FKGC) aims to infer missing facts by learning representations of un seen relations from limited head-tail pairs. However, existing methods often suffer from inherent data sparsity in knowledge graphs when learning relation representations, leading to overfitting to local samples and slow convergence. Although some works have employed data augmentation to synthesize additional training samples to alleviate data sparsity, these methods do not account for the complex relation types in the triplet genera tion process. As a result, they may introduce noisy or incorrect relational links, which negatively impact the model's completion performance. To address these limitations, we propose a Relation-Type Constrained Mixup (RTCM) for FKGC. Specifically, we first introduce the relation-type-constrained augmentation strategies to en sure that augmented samples preserve the original relation types. Then, we present an entity and structural augmentation module: entity augmentation selects suitable entities from a pool of semantically similar candi dates to replace original entities, thereby synthesizing more diverse, denser training data. Structural augmentation expands first-order neighbors under the same constraint to build synthetic subgraphs that retain comparable topo logical structures. Finally, we adopt a Mixup-based joint training strategy that fuses raw and augmented data to enrich entity representations and accelerate model convergence. Extensive comparative experiments demonstrate that our method outperforms state-of-the-art baselines.

Direct Preference Optimization (DPO) guides large language models (LLMs) to generate recommendations aligned with user historical behavior distributions by minimizing preference alignment loss. However, our systematic empirical research and theoretical analysis reveal that DPO tends to amplify spurious correlations caused by environmental confounders during the alignment process, significantly undermining the generalization capability of LLM-based generative recommendation methods in out-of-distribution (OOD) scenarios. To mitigate this issue, we propose CausalDPO, an extension of DPO that incorporates a causal invariance learning mechanism. This method introduces a backdoor adjustment strategy during the preference alignment phase to eliminate interference from environmental confounders, explicitly models the latent environmental distribution using a soft clustering approach, and enhances robust consistency across diverse environments through invariance constraints. Theoretical analysis demonstrates that CausalDPO can effectively capture users' stable preference structures across multiple environments, thereby improving the OOD generalization performance of LLM-based recommendation models. We conduct extensive experiments under four representative distribution shift settings to validate the effectiveness of CausalDPO, achieving an average performance improvement of 24.10\% across four evaluation metrics.

Model merging aims to integrate multiple independently fine-tuned expert models into a single model while preserving the knowledge of all experts. However, existing approaches mainly address parameter conflicts at the merging stage and overlook the role of the fine-tuning process, which often leads to significant post-merge performance degradation. To address this limitation, we propose a novel merging-aware optimizer (abbreviated as MergOPT) that injects principled merge-induced parameter shifts into the weight update steps so that the fine-tuned model exhibits a more stable loss landscape under subsequent merging operations. Specifically, we first formulate model merging as a distributionally robust optimization problem in the weight space: the parameters of other experts to be merged are viewed as adversarial merge-offsets, and fine-tuning adapts to the worst-case merging scenario. Building on this formulation, we analyze the distribution of parameter updates and the effects of merging hyperparameters, from which we derive a merging-guided feasible region for weight shifts. Finally, extensive experiments across four large language models (LLMs) and one vision model show that our approach consistently outperforms standard fine-tuning, yielding an average relative gain of 3.5\% and a maximum gain of 9.5\% across four merging strategies when merging seven experts.

Sequential recommendation (SR) learns user preferences based on their historical interaction sequences and provides personalized suggestions. In real-world scenarios, most users can only interact with a handful of items, while the majority of items are seldom consumed. This pervasive long-tail challenge limits the model's ability to learn user preferences. Despite previous efforts to enrich tail items/users with knowledge from head parts or improve tail learning through additional contextual information, they still face the following issues: 1) They struggle to improve the situation where interactions of tail users/items are scarce, leading to incomplete preferences learning for the tail parts. 2) Existing methods often degrade overall or head parts performance when improving accuracy for tail users/items, thereby harming the user experience. We propose Tail-Aware Data Augmentation (TADA) for long-tail sequential recommendation, which enhances the interaction frequency for tail items/users while maintaining head performance, thereby promoting the model's learning capabilities for the tail. Specifically, we first capture the co-occurrence and correlation among low-popularity items by a linear model. Building upon this, we design two tail-aware augmentation operators, T-Substitute and T-Insert. The former replaces the head item with a relevant item, while the latter utilizes co-occurrence relationships to extend the original sequence by incorporating both head and tail items. The augmented and original sequences are mixed at the representation level to preserve preference knowledge. We further extend the mix operation across different tail-user sequences and augmented sequences to generate richer augmented samples, thereby improving tail performance. Comprehensive experiments demonstrate the superiority of our method. The codes are provided at https://github.com/KingGugu/TADA.

Data augmentation mitigates data sparsity in sequential recommendations by generating new yet effective data. Most existing work focuses on a single original sequence with item-level augmentations. It ignores the correlations between different users (sequences) and struggles to produce diverse yet reasonable new data across sequences, leading to limited performance improvements. Also, the item-level operation may destroy the integrity of the preference knowledge contained in the original sequences, resulting in incomplete preference learning. In this work, we propose a novel user correlation guided cross-sequence mixing plug-in for sequential recommendation (UCMRec). Our core idea is to select the appropriate sequence based on the user correlation graph and perform cross-sequence mixup operations at the representation level to generate high-quality samples. Specifically, we construct a user-user graph based on joint interactions and perform two types of searches to get the candidate users with different correlations. Then, we introduce a mixup operation at the sequence representation level to generate diverse yet reasonable samples across sequences. Furthermore, we propose a topology-aware reweighting module to enable the model to adjust the learning intensity based on preference similarity. Comprehensive experiments demonstrate the superiority of our method.