Loop closure is a crucial component in 3DGS-based SLAM methods for mitigating accumulated tracking errors and map drift, especially in large-scale environments. Existing approaches typically rely on 2D image-level features to detect loops, overlooking the rich information inherent in the 3D scene. While directly utilizing 3D Gaussian attributes as local features is desirable, constructing distinctive global descriptors is non-trivial due to the high density and explicit nature of Gaussian primitives. To address these challenges, we propose GLC-SLAM, a monocular Gaussian Splatting SLAM system augmented with a robust loop closure module. Specifically, to tackle the high density of primitives, we introduce Spatio-Temporal Uncertainty Sampling (STUS) to select an informative Gaussian subset from each submap. The explicit parameters of these sampled primitives are then encoded into a distinctive global descriptor for candidate retrieval, thereby overcoming the non-descriptive nature of the raw representation. To avoid false positives, geometric validation is performed via synthesized view rendering. With the validated loops, we conduct hierarchical loop correction via global bundle adjustment and subsequent map deformation, ensuring consistency both across and within submaps. Experimental results on synthetic and real-world datasets demonstrate that GLC-SLAM achieves state-of-the-art performance compared to existing dense SLAM methods, surpassing the best competing method by 5% in tracking accuracy on ScanNet and yielding the highest rendering fidelity of 34.72 dB PSNR on Replica.

Learning visuomotor policies with imitation learning from 3D observations is a primary research direction in robotic manipulation, as 3D data inherently captures spatial features critical for action. While many existing methods rely on multiview point cloud fusion, recent studies like 3D Diffusion Policy have shown that using a single-view observation setting with a lightweight point cloud encoder can achieve robust 3D policy learning. This design yields a simpler and more efficient pipeline. However, we observe that such a setting suffers from performance degradation under significant viewpoint changes. To overcome this limitation without requiring extensive multi-view data acquisition, we propose a digital twin-based real-time point cloud augmentation method named Twin-DP3. Our approach leverages a 6-DoF pose estimation algorithm to track the manipulated object and uses a robot model to generate robot component point clouds in real time. We validate the effectiveness of this method through comprehensive experiments in both simulation and realworld environments, demonstrating notable improvements in policy robustness under novel viewpoints.

Federated Prototype Learning (FedPL) has emerged as an effective strategy for handling data heterogeneity in Federated Learning (FL). In FedPL, clients collaboratively construct a set of global feature centers (prototypes), and let local features align with these prototypes to mitigate the effects of data heterogeneity. The performance of FedPL highly depends on the quality of prototypes. Existing methods assume that larger inter-class distances among prototypes yield better performance, and thus design different methods to increase these distances. However, we observe that while these methods increase prototype distances to enhance class discrimination, they inevitably disrupt essential semantic relationships among classes, which are crucial for model generalization. This raises an important question: how to construct prototypes that inherently preserve semantic relationships among classes? Directly learning these relationships from limited and heterogeneous client data can be problematic in FL. Recently, the success of pre-trained language models (PLMs) demonstrates their ability to capture semantic relationships from vast textual corpora. Motivated by this, we propose FedTSP, a novel method that leverages PLMs to construct semantically enriched prototypes from the textual modality, enabling more effective collaboration in heterogeneous data settings. We first use a large language model (LLM) to generate fine-grained textual descriptions for each class, which are then processed by a PLM on the server to form textual prototypes. To address the modality gap between client image models and the PLM, we introduce trainable prompts, allowing prototypes to adapt better to client tasks. Extensive experiments demonstrate that FedTSP mitigates data heterogeneity while significantly accelerating convergence.

Object Navigation (ObjectNav) in unfamiliar environments is a complex challenge, particularly in zero-shot settings where agents must locate unseen targets without prior training. Existing approaches typically extract semantic cues from a single level—either the image or instance level—limiting the depth of scene understanding. Furthermore, in long-horizon tasks, the spatial sparsity of semantic features often leads to navigational disorientation and performance deterioration. To address these limitations, we propose a novel zero-shot method called Multi-Dimensional-Weighted Frontier Maps (MDWFM). Our approach synergizes semantic and geometric cues by conducting a hierarchical analysis of visual observations. Specifically, we utilize a Vision-Language Model (BLIP-2) for coarse image-level understanding and a semantic-enhanced Segment Anything Model (SAM) for fine instance-level inspection. Crucially, MDWFM employs an adaptive weighting mechanism to dynamically evaluate candidate frontiers. This strategy effectively balances multi-dimensional features, leveraging geometric cues to compensate for semantic sparsity, which prevents performance degradation and ensures the agent selects the most promising directions for navigation. We evaluate MDWFM in photo-realistic environments using the Habitat-Matterport 3D (HM3D) and Gibson datasets within the Habitat simulator. Our approach achieves state-of-the-art zero-shot performance on HM3D and competitive results on Gibson, particularly in SPL, for the Object Goal Navigation task.

Facing the rapid development of both user equipment (UE) and mobile applications, mobile edge computing (MEC) emerged as a new paradigm to furnish UE with satisfactory computational capability and task latency. Studies on MEC have put a great deal of focus on computation offloading, mainly involving multi-step task offloading and resource allocation (OffAll) which is mostly non-trivial. Concerning the heavy burden of online OffAll optimization on scalability and efficiency, offline learning-based OffAll approaches are of high popularity. However, most existing learning-based OffAll is still short of generalizability and adaptability across different MECs, thus difficult for general-purpose application. Inspired by the remarkable success of LLMs for their impressive generalization ability across different tasks by conditioning on instructions or prompts, this paper proposes a diffusion-based OffAll approach (termed DiffLoader) by utilizing the generative conditional-planning ability of diffusion models learned from multi-MEC OffAll experiences. Specifically, we leverage a few-shot trajectory of MEC states transitions as MEC-specific prompts, adopt diffusion to learn multi-MEC conditional OffAll distribution, and generate OffAll decisions via distribution-sampling during execution. We conduct extensive experiments to verify the advantages of DiffLoader over state-of-the-art learning-based multi-MEC OffAll approaches, and also show the compositive generating ability of DiffLoader even for unseen MEC.

With the rapid development of digital technologies, a large range of real-world systems, spanning from cloud servers, IoT devices, to industrial control systems, continuously generate vast amounts of time series data. Time series anomaly detection (AD) plays a crucial role in maintaining system stability by identifying unusual patterns from normal distributions, with the primary challenge lies in learning effective anomaly-discriminative representations. Recently, diffusion models have been applied to time series AD due to their strong representational capabilities. However, existing diffusion-based methods typically rely on reconstruction errors, which not only fail to fully exploit the representational potential of diffusion models but also be computationally intensive. To address these limitations, through experimental observation and theoretical analysis, we show that specific regions of the diffusion noises exhibit stronger representation capabilities for normal patterns, which can be leveraged to enhance AD performance and reduce computational costs. Building on these insights, we propose NoiseAD, a diffusion noise-guided anomaly detection method incorporating an optimal noise steps selection approach to identify diffusion steps with higher resolution. Extensive experiments on diverse benchmarks demonstrate the superiority of NoiseAD over state-of-the-art methods, further substantiated by insightful visualizations.

Due to the selective absorption and scattering of light by diverse aquatic media, underwater images usually suffer from various visual degradations. Existing underwater image enhancement (UIE) approaches that combine underwater physical imaging models with neural networks often fail to accurately estimate imaging model parameters such as scene depth and veiling light, resulting in poor performance in certain scenarios. To address this issue, we propose a physical model-guided framework for jointly training a Deep Degradation Model (DDM) with any advanced UIE model. DDM includes three well-designed sub-networks to accurately estimate various imaging parameters: a veiling light estimation sub-network, a factors estimation sub-network, and a depth estimation sub-network. Based on the estimated parameters and the underwater physical imaging model, we impose physical constraints on the enhancement process by modeling the relationship between underwater images and desired clean images, i.e., outputs of the UIE model. Moreover, while our framework is compatible with any UIE model, we design a simple yet effective fully convolutional UIE model, termed UIEConv. UIEConv utilizes both global and local features through a dual-branch structure. UIEConv trained within our framework achieves remarkable enhancement results across diverse underwater scenes. Furthermore, as a byproduct of UIE, the trained depth estimation sub-network enables accurate underwater scene depth estimation. Extensive experiments conducted in various real underwater imaging scenarios, including deep-sea environments with artificial light sources, validate the effectiveness of our framework and the UIEConv model. Code is available at https://github.com/ddz16/UWEnhancer

Heterogeneous federated learning (HtFL) aims to enable collaboration among clients that differ in both data distributions and model architectures. Prototype-based methods, which communicate class-level feature centers (prototypes) instead of full model parameters, have recently shown strong potential for HtFL. Existing prototype-based HtFL methods typically reuse the MSE-based or cosine-based alignment mechanism developed for homogeneous FL when aligning client-specific representations with global prototypes. These approaches are essentially coordinate alignment, where representations of clients are forced to match the global prototypes in the embedding space in an element-wise manner. Such alignment implicitly assumes that all clients should map their representations into the feature subspace defined by the global prototypes. This assumption is reasonable in homogeneous FL, where all clients share the same feature extractor. However, it becomes problematic in HtFL, since heterogeneous feature extractors naturally induce client-specific feature subspaces, and forcing all clients to optimize within a single global subspace unnecessarily suppresses their learning capacity. We observe that coordinate alignment implicitly couples two distinct objectives: aligning inter-class semantic structure, which is directly beneficial for classification, and enforcing a shared feature basis, which is unnecessary and even harmful under model heterogeneity. Building on this insight, we design FedSAF, which shifts the alignment objective from absolute coordinates to inter-class relational structure. We demonstrate that structural alignment consistently outperforms coordinate alignment in heterogeneous settings. Experiments on multiple benchmarks show that our structural alignment outperforms state-of-the-art prototype-based HtFL methods by up to 3.52\%.

Recent breakthroughs in Transformer-based large models, have driven widespread tasks, yet their reliance on centralized cloud deployment raises significant privacy risks due to sensitive data exposure. While edge-based collaborative inference offers a privacy-preserving alternative, existing methods face critical limitations: static model partitioning cannot adapt to dynamic edge resource fluctuations, and rigid multi-head attention handling overlooks semantic-critical prioritization and parallelism. We propose EdgeFormer, a latency-aware framework for distributed Transformer inference in resource-constrained edge networks. EdgeFormer dynamically allocates model blocks across devices via efficiency-storage trade-off optimization and introduces collaborative Multi-Head Attention (cMHA), which distributes semantic-critical attention heads across devices while pruning redundant ones under real-time constraints. We further develop LiScore, a composite metric integrating attention diversity and latency costs, alongside a similarity-based retrieval method to reduce recomputation overhead. Extensive experiments demonstrate that EdgeFormer achieves up to 2.01 \\times inference acceleration over state-of-the-art baselines with \\leq 1.06% accuracy loss, maintaining robustness under varying edge conditions.

Intelligent Transportation System (ITS) employs traditional optimization methods, including gradient-based or evolutionary methods. These methods often fail to simultaneously balance predictive accuracy, latency, and resource efficiency while respecting strict operational constraints. Multi-objective system design under explicit constraints is also a challenge in engineering and artificial intelligence. To address these gaps, we propose a Transfer Learning-Enhanced Multi-Objective Whale Optimisation Algorithm (TL-MOWOA) for constrained multi-objective optimisation of control parameters in Intelligent Transportation Systems (ITS). The framework integrates transfer learning to accelerate convergence by reusing knowledge from related optimisation tasks. In this way, it improves adaptability across diverse traffic scenarios. Evolutionary operators of the Whale Optimisation Algorithm, within the TL-MOWOA Algorithm, preserve population diversity and ensure robust global search. The transfer-learned priors guide the search toward promising feasible regions, hence reducing the risk of stagnation. Experimental evaluation was conducted on synthetic benchmarks, including balanced and high-dimensional constrained. The results signify that the TL-MOWOA Algorithm better for state-of-the-art algorithms. The proposed TL-MOWOA Algorithm achieves up to 15% reduction in average travel time, 12% reduction in fuel consumption, 20% reduction in congestion index, and 18% reduction in emissions as compared to NSGA-III, MOPSO, MOWOA, and standalone TL-based EAs. The proposed TL-MOWOA Algorithm provides a scalable and adaptive solution for ITS optimisation.

Recently, Large language models (LLMs) have revolutionized Natural Language Processing (NLP). Pretrained LLMs, due to limited training context size, struggle with handling long token sequences, limiting their performance on various downstream tasks. Current solutions toward long context modeling often employ multi-stage continual pertaining, which progressively increases the effective context length through several continual pretraining stages. However, those approaches require extensive manual tuning and human expertise. In this paper, we introduce a novel single-stage continual pretraining method, Head-Adaptive Rotary Position Encoding (HARPE), to equip LLMs with long context modeling capabilities while simplifying the training process. Our HARPE leverages different Rotary Position Encoding (RoPE) base frequency values across different attention heads and directly trains LLMs on the target context length. Extensive experiments on 4 language modeling benchmarks, including the latest RULER benchmark, demonstrate that HARPE excels in understanding and integrating long-context tasks with single-stage training, matching and even outperforming existing multi-stage methods. Our results highlight that HARPE successfully breaks the stage barrier for training LLMs with long context modeling capabilities.

Multi-access edge computing provides mobile devices (MDs) with both satisfactory computing resources and task latency, by offloading MDs' tasks to nearby edge servers. There is a popular trend to develop decentralized offloading (dec-offloading) approaches using multi-agent reinforcement learning (MARL), primarily based on centralized-training and decentralized-execution. However, the dec-offloading policies together could also lack exploration and collaboration since each MD is guided by the policy-critic only through offloading costs without explicitly considering the impacts of other MDs' offloading behaviors. Motivated by this, we propose Explorative and collaborative Offloading (ExplabOff) that can achieve superior dec-offloading by consciously exploiting the implicit exploration and collaboration information involved in MDs' states and actions. Specifically, we design two additional policy-learning metrics, the exploration-metric based on the maximum entropy of MDs' joint offloading actions and collaboration-metric based on one MD's belief about others' offloading behaviors. Then, we assemble these metrics into a new criterion defined as the mutual information (MI) between MDs' states and actions, and adopt it as an additive reward except for the vanilla reward during centralized-training. Furthermore, we distinguish MI between superior and inferior offloading, strengthening and weakening them discriminatively. Experiments on both simulation and real-testbed verify the effectiveness of ExplabOff over state-of-the-art dec-offloading.

Byte Pair Encoding (BPE) serves as a foundation method for text tokenization in the Natural Language Processing (NLP) field. Despite its wide adoption, the original BPE algorithm harbors an inherent flaw: it inadvertently introduces a frequency imbalance for tokens in the text corpus. Since BPE iteratively merges the most frequent token pair in the text corpus to generate a new token and keeps all generated tokens in the vocabulary, it unavoidably holds tokens that primarily act as components of a longer token and appear infrequently on their own. We term such tokens as Scaffold Tokens. Due to their infrequent occurrences in the text corpus, Scaffold Tokens pose a learning imbalance issue. To address that issue, we propose Scaffold-BPE, which incorporates a dynamic scaffold token removal mechanism by parameter-free, computation-light, and easy-to-implement modifications to the original BPE method. This novel approach ensures the exclusion of low-frequency Scaffold Tokens from the token representations for given texts, thereby mitigating the issue of frequency imbalance and facilitating model training. On extensive experiments across language modeling and even machine translation, Scaffold-BPE consistently outperforms the original BPE, well demonstrating its effectiveness.

Recently, Large Language Models (LLMs) have been widely adopted in a wide range of tasks, leading to increasing attention towards the research on how scaling LLMs affects their performance. Existing works, termed Scaling Laws, have discovered that the final test loss of LLMs scales as power-laws with model size, computational budget, and dataset size. However, the temporal change of the test loss of an LLM throughout its pre-training process remains unexplored, though it is valuable in many aspects, such as selecting better hyperparameters directly on the target LLM. In this paper, we propose the novel concept of Temporal Scaling Law, studying how the test loss of an LLM evolves as the training steps scale up. In contrast to modeling the test loss as a whole in a coarse-grained manner, we break it down and dive into the fine-grained test loss of each token position, and further develop a dynamic hyperbolic-law. Afterwards, we derive the much more precise temporal scaling law by studying the temporal patterns of the parameters in the dynamic hyperbolic-law. Results on both in-distribution (ID) and out-of-distribution (OOD) validation datasets demonstrate that our temporal scaling law accurately predicts the test loss of LLMs across training steps. Our temporal scaling law has broad practical applications. First, it enables direct and efficient hyperparameter selection on the target LLM, such as data mixture proportions. Secondly, viewing the LLM pre-training dynamics from the token position granularity provides some insights to enhance the understanding of LLM pre-training.

In federated learning (FL), model aggregation is a critical step by which multiple clients share their knowledge with one another. However, it is also widely recognized that the aggregated model, when sent back to each client, performs poorly on local data until after several rounds of local training. This temporary performance drop can potentially slow down the convergence of the FL model. Most research in FL regards this performance drop as an inherent cost of knowledge sharing among clients and does not give it special attention. While some studies directly focus on designing techniques to alleviate the issue, an in-depth investigation of the reasons behind this performance drop has yet to be conducted.To address this gap, we conduct a layer-peeled analysis of model aggregation across various datasets and model architectures. Our findings reveal that the performance drop can be attributed to two major consequences of the aggregation process: (1) it disrupts feature variability suppression in deep neural networks (DNNs), and (2) it weakens the coupling between features and subsequent parameters.Based on these findings, we propose several simple yet effective strategies to mitigate the negative impacts of model aggregation while still enjoying the benefit it brings. To the best of our knowledge, our work is the first to conduct a layer-peeled analysis of model aggregation, potentially paving the way for the development of more effective FL algorithms.

The prevalent use of Byte Pair Encoding (BPE) in Large Language Models (LLMs) facilitates robust handling of subword units and avoids issues of out-of-vocabulary words. Despite its success, a critical challenge persists: long tokens, rich in semantic information, have fewer occurrences in tokenized datasets compared to short tokens, which can result in imbalanced learning issue across different tokens. To address that, we propose LBPE, which prioritizes long tokens during the encoding process. LBPE generates tokens according to their descending order of token length rather than their ranks in the vocabulary, granting longer tokens higher priority during the encoding process. Consequently, LBPE smooths the frequency differences between short and long tokens, and thus mitigates the learning imbalance. Extensive experiments across diverse language modeling tasks demonstrate that LBPE consistently outperforms the original BPE, well demonstrating its effectiveness.

Fine-tuning large language models (LLMs) with local data is a widely adopted approach for organizations seeking to adapt LLMs to their specific domains. Given the shared characteristics in data across different organizations, the idea of collaboratively fine-tuning an LLM using data from multiple sources presents an appealing opportunity. However, organizations are often reluctant to share local data, making centralized fine-tuning impractical. Federated learning (FL), a privacy-preserving framework, enables clients to retain local data while sharing only model parameters for collaborative training, offering a potential solution. While fine-tuning LLMs on centralized datasets risks data leakage through next-token prediction, the iterative aggregation process in FL results in a global model that encapsulates generalized knowledge, which some believe protects client privacy. In this paper, however, we present contradictory findings through extensive experiments. We show that attackers can still extract training data from the global model, even using straightforward generation methods, with leakage increasing as the model size grows. Moreover, we introduce an enhanced attack strategy tailored to FL, which tracks global model updates during training to intensify privacy leakage. To mitigate these risks, we evaluate privacy-preserving techniques in FL, including differential privacy, regularization-constrained updates and adopting LLMs with safety alignment. Our results provide valuable insights and practical guidelines for reducing privacy risks when training LLMs with FL.

In the process of continuous Intelligentization of satellite applications, it has become inevitable to equip on-board computers with real-time operating system software. Considering the complexity of the space radiation environment which mainly includes electrons, protons and heavy ions, it is of great significance to improve the reliability of the real-time operating system to achieve application programs insensitivity to radiation effects. This paper presents reliability enhancement technical scheme for real-time operation system software. Five energy pointirradiation tests were carried out in Xi'an. The real-time operating system does not have abnormal resets, which verifies the effectiveness of the reliability enhancement technology.

Effective information interaction can enhance the coordination capabilities in collaborative multi-agent reinforcement learning (MARL). A popular communication scheme is the exchange of agents’ intention information, which typically involves broadcasting agents’ intention information to all other agents. This not only increases the communication overhead of the entire system but also interferes with the decision-making of agents to some extent due to the reception of irrelevant intention information from other agents. In this paper, we propose the Multi-Agent Intention Filter (MAIF), which filters out intentions irrelevant to the agent, allowing the agent to focus on the intentions of agents with whom it is more likely to collaborate, thus promoting effective cooperation among agents. Specifically, our method first uses a State Simulator to coordinate the joint intentions of agents based on their current joint observations and predict target states, providing prior knowledge for subsequent intention filtering. Then, we calculate the causal effect of agents’ intentions on target states and filter out intentions that are irrelevant to the agent, thereby promoting effective cooperation. Each agent will use the filtered intention information to assist in decision-making. Experimental results show that our method outperforms strong baselines in multiple cooperative MARL tasks under various task settings.