Federated Learning (FL) provides an effective framework for large model training or adaption, which achieve privacy preservation and reducing the need for extensive data transmission. However, in Federated Learning, the heterogeneity of the data distributions between clients often hinders optimized model fusion, causing local models to converge prematurely to suboptimal solutions. In this paper, we propose AntiTrap-FL, an efficient federated learning approach designed to help clients escape from such local traps. AntiTrap-FL identifies trapped clients by analyzing gradient and accuracy trends. Then, it pairs these clients with the most distant partners in parameter space, introducing perturbations that help them escape local traps. To stabilize knowledge transfer, AntiTrap-FL applies symmetric or asymmetric parameter corrections with adaptive weighting. Comprehensive theoretical analysis provides strong convergence guarantees,and extensive experiments show that AntiTrap-FL achieves up to 4.05% improvement in accuracy over existing FL methods under Non-IID conditions, while maintaining the computational efficiency of edge devices. Additionally, experiments on large vision model fine-tuning show that AntiTrap-FL consistently outperforms baseline methods across varying client scales, achieving up to 4.82% improvement over existing FL methods.
This study presents an optimization strategy for classifiers that rely on the perplexity of paired language models. Perplexity-based features are commonly employed in diverse tasks such as human–machine text discrimination, stylometry, authorship attribution, reading-time prediction, sentiment analysis, and Alzheimer’s disease detection. We identify a systematic Dominant-Model Bias, wherein one model disproportionately favors its own class, thereby degrading classification accuracy. This bias tends to amplify under conditions of training data imbalance. We provide both empirical evidence and a formal characterization of this phenomenon, and we introduce the Delta recalibration strategy to mitigate it. Experiments on independent tasks—Alzheimer’s disease detection, authorship attribution, and native language identification—demonstrate that the Delta recalibration strategy consistently enhances classification accuracy. Its benefits increase with growing class imbalance: in highly skewed scenarios, accuracy improvements reach up to 20% over the unoptimized baseline.11 The code to replicate the reported experiments is available on Github.
Mamba has demonstrated significant potential in image super-resolution due to its linear computational complexity and excellent long-range modeling capabilities. Existing studies typically employ multi-directional scanning to overcome Mamba’s inherent causal constraints, enhancing spatial relationship modeling. However, multi-directional scanning significantly increases computational overhead, undermining Mamba’s advantage in terms of computational complexity. In this work, we conduct a systematic analysis of multi-directional scanning schemes through LAM visualization results and quantitative metric evaluation. Then, we propose CAMSR, a non-causal lightweight super-resolution model based on Mamba that achieves similar content perception. Specifically, it uses a content-aware sequence generator to cluster content-similar feature tokens and generate a reordered sequence of similar feature clusters. Subsequently, a content-aware state-space equation facilitates long-range pixel interactions and captures global context through prompt learning. Finally, spatial and channel-level attention mechanisms further refine features, achieving an effective balance between reconstruction quality and computational complexity. The experimental results demonstrate that our approach achieves superior performance compared with state-of-the-art methods while using fewer parameters. The code is available at https://github.com/nathan66666/CAMSR.git.
Incremental Named Entity Recognition (INER) stands as a pivotal task in information extraction, emphasizing the successive identification of new entity types within unstructured text. Faced with the continuous influx of entity types, INER grapples with two significant challenges: the widespread issue of catastrophic forgetting and the unique shift issue of the non-entity type semantics. While pseudo-labeling-based INER methods have proven effective in addressing these challenges, a previously overlooked issue arises: the biased context problem. Our analysis shows that, in new sentences, the contextual associations of tokens representing old entity types exhibit a significantly stronger bias towards new entity types compared to their contexts in old sentences. This tendency intensifies the degradation of old knowledge while promoting the overfitting of new knowledge. To solve this biased context, we propose a Type-Balanced Contextual Learning (TBCL) method, featuring a sentence-duplet learning scheme and a contextual consistency loss. This approach offers a fresh perspective for INER through context analysis. Extensive experiments across ten INER settings on three highly recognized datasets showcase the efficacy of our TBCL method, highlighting its proficiency in resolving the biased context issue inherent in pseudo-labeling based INER approaches.
Large Language Models suffer from the hallucination that leads to the generation of factually incorrect and logically inconsistent responses. To make a solution to this problem, we propose the SIC-Autonomous Ensemble (SICAE), a Self Iterative Contradiction Autonomous Ensembling framework, for mitigating component LLM hallucination and gaining performance with model ensembling and self-iterative contradiction refinement. We use self-correcting loops for each component LLM for knowledge refinement before aligning the components for generating candidate responses for ranking. Then top candidate responses are selected as intermediate responses and fused to generate the final response. Our framework incorporates three modules, self-iterative refinement, ranking module, and fusion module where final response is obtained using an autonomous weight update mechanism from a pool of candidate LLM responses. To keep the framework computationally less heavy, we are using low-parameter open-source LLMs to generate candidate responses. To ensure the generalizability of our framework, we are evaluating the responses using three benchmark datasets containing factual and logical reasoning questions and proposing an autonomous hyperparameter selection technique for SIC-AE. We validate our approach while comparing the performance of SIC-AE with larger parameterized model such as llama-2-70B and existing state of the art self-iterative and hallucination mitigation techniques while evaluating on corresponding benchmarks as well. Results are evaluated for base LLMs, and after integrating self-iterative contradiction refinement, and finally for the proposed ensemble framework. We achieved a comparative higher accuracy of 1-5% than the SOTA self-refinement techniques and see further improvement of 1-8% in generating responses on different benchmarks for hallucination mitigation while adding LLM ensemble to our framework.
In the realm of fashion design, sketches serve as a medium for expressing distinctive drawing styles and creative intent, capturing intricate details like stroke variations and texture nuances. The advent of sketch-to-image translation technology has notably aided designers. However, existing methods often compromise these sketch details during image generation, resulting in images that deviate from the intended structural and stylistic characteristics of the input sketch. This limitation hampers the ability to offer designers a precise preview of the final output. To overcome this challenge, we introduce HAIFIT, a novel approach that transforms sketches into high-fidelity, lifelike clothing images by integrating multi-scale features and capturing multi-directional feature dependencies to encourage structurally consistent representations. In addition, we develop a diffusion-based variant, D-HAIFIT, to provide a complementary benchmark for evaluating structural consistency and generative fidelity. Through extensive qualitative and quantitative evaluations conducted on the HAIFashion dataset, our method demonstrates superior experimental performance compared to existing approaches, generating clothing images with improved structural consistency and fine-grained detail preservation for practical fashion design workflows. Furthermore, HAIFIT achieves faster training and inference, contributing to improved computational efficiency and reduced design iteration time.
Fixed length time series data is seldom found in real world datasets. In cases such as language, wildlife, and machining processes, data can vary in length. Similarly to real-world scenarios, it is not always feasible to retrain a Machine Learning (ML) model to handle new and old data simultaneously. Continual Learning (CL) was introduced to adapt ML systems to new data while retaining performance on past information and minimizing large training runs. Replay-based methods which use a memory buffer to augment incoming data have emerged as the preeminent approach type for CL. While CL has been applied to many scenarios including time series and images, their performance has yet to be evaluated on Variable-Length Time Series (VLTS) data such as handwriting trajectories, industrial sensing, insect wingbeat spectrograms, and spoken digits. When implementing CL-ML systems, factors affecting performance and implementation cost are important research areas in ML. This paper reports an empirical comparison of padding choices, ML models, CL methods, replay buffer size, computational cost, and accuracy on 4 VLTS datasets. The role of padding in transforming VLTS is examined, with experiments showing different mechanisms affect CL methods dissimilarly with padding responsible for an average of 32.3% accuracy range within CL methods. The computational load-accuracy tradeoff analysis shows that traditional methods such as a Multi-layer Perceptron coupled with Experience Replay and a small buffer improved memory and energy-accuracy efficiency by 111.5% and 14.6%, respectively.
This study presents one of the first systematic investigations of domain-adaptive pretraining for propaganda technique detection in Lithuanian. We perform masked language modeling-based domain-adaptive pretraining on a corpus of 63,648 news articles from non-credible outlets. We evaluate its impact using masked-language probing, propaganda technique recognition through sequence tagging and sentence classification, and comparison with a large language model in a zero-shot setting. In addition to quantitative evaluation, we conduct a qualitative analysis based on a case study of annotation behavior. The results show that domain-adaptive pretraining improves propaganda technique detection in Lithuanian, enhances the ranking of contextually meaningful terms, and improves performance for most techniques, with gains varying across tasks.
Accurate traffic prediction is critical for the development of smart cities, but remains difficult due to complex spatio-temporal dependencies in road networks. Existing methods typically rely on either intra-mixing techniques that process each node independently with shared parameters, or intermixing models that capture aggregate spatial traffic dynamics. However, these approaches often overlook key flow exchange mechanisms or introduce spurious dependencies. To overcome these limitations, we propose a Dual mixing Multi-scale Spatio-Temporal neural network (DiMST), tailored for traffic prediction. DiMST integrates three principal components: (i) a temporal encoder that fuses raw traffic sequences with periodic contextual information; (ii) a spatial encoder that encodes node identities and positional relationships by leveraging spectral representations in both graph and hypergraph domains; and (iii) a dual mixing multi-scale spatio-temporal encoder comprising intra-and inter-mixing layers, realized through multilayer perceptrons and graph and hypergraph wavelet networks across multiple spectral resolutions, enables more accurate modeling of complex dependencies in traffic data. Extensive experiments on real-world traffic datasets demonstrate the effectiveness and superiority of the proposed model for traffic prediction.
This paper studies optimal aggregative coordination in open multi-agent systems, where agent membership, communication topology, and local objectives may vary over time. The problem is formulated as a distributed aggregative optimization task, in which each agent’s local objective depends on its own decision variable and an aggregate generated by all currently active agents. An open-network distributed aggregative gradient tracking algorithm with a residual handoff mechanism for departing agents is developed, which preserves aggregate and gradient-tracking balance across topology transitions. Furthermore, an auxiliary frozen closed-network iteration is introduced, based on which a one-step open-network error recursion is derived by bounding the effects of optimal-point variation, arrivals, departures, and objective switching. Under suitable contraction and handoff small-gain conditions, an explicit ultimate bound is established for the normalized error relative to the optimal solution trajectory. The bound separates the effects of optimal-point variation, objective-switching variation, and open-network membership changes, and quantifies the influence of arrivals, departures, and residual handoff on the tracking accuracy. Numerical experiments on open multi-robot surveillance problems support the theoretical findings and demonstrate the effectiveness of the proposed method under dynamic network changes.
Heterogeneous data, specifically non-independent and identically distributed (non-IID) data, remains a fundamental challenge that degrades the performance of federated learning (FL). This degradation can be alleviated by employing pretrained diffusion models to generate high-quality synthetic data to facilitate training. However, centralized fine-tuning requires access to client data, which conflicts with the decentralized training paradigm of FL. Moreover, the continuous optimization of local models causes knowledge to drift away from the original distribution, impairing the utility of the synthetic data in mitigating data heterogeneity. To address these issues, we propose a novel framework, termed federated diffusion learning with dual-side calibration (FedDiDC), which effectively enhances the global model’s accuracy across heterogeneous distributions without compromising data privacy. Specifically, we decouple the conventional diffusion process into federated fine-tuning of lightweight embedding networks on clients and centralized generation on the server, thereby avoiding direct transmission of raw client data. To further enhance the data utility and tackle the non-IID challenge, a dual-side calibration mechanism is designed: client-side calibration aligns the local optimization trajectory towards the actual data distribution by adjusting the local loss function, and server-side calibration groups clients according to distributional statistics and performs cluster-wise client selection and aggregation, which guides the model aggregation within statistically balanced client clusters. Experimental results on benchmark datasets, including CIFAR-10, CIFAR-100, and Tiny-ImageNet, show the effectiveness of the proposed framework. Particularly under extreme non-IID settings (β = 0.1), FedDiDC achieves up to a 17.76% accuracy improvement compared to existing baseline methods.
With the rapid development of the Industrial Internet of Things (IIoT), the large-scale acquisition of industrial time-series data has become increasingly feasible, providing valuable temporal insights for timely decision-making. Nonetheless, efficient data transmission is often constrained by communication bandwidth limitations. Consequently, the monitoring data are frequently sampled at sparse intervals in practical industrial scenarios, potentially obscuring fine-grained temporal patterns and reducing the effectiveness of downstream analytical tasks. To tackle this challenge, a novel fine-grained data reconstruction framework is proposed for sparsely sampled industrial time-series. Specifically, a semantic-aware loss function is designed to align network output with the ground truth while preserving structural patterns and semantic richness. Additionally, domain knowledge that incorporates measurement and correlation constraints is embedded into data reconstruction via an augmented Lagrangian-based learning paradigm. In this scheme, the reconstructed time-series maintain numerical alignment with coarse-grained observations and preserve intrinsic physical dependencies, facilitating reliable downstream analytical tasks. Extensive experiments on the C-MAPSS dataset, as well as a real-world steel conveyor roller dataset, demonstrate that the proposed framework outperforms state-of-the-art approaches across different downstream analytical tasks. The code is publicly available on https://github.com/ydzhang-cyber/FDRF.
3D anomaly detection has recently emerged as an effective means of high-precision inspection for industrial products. However, most existing methods are typically trained in a single category-specific manner and face significant challenges in multi-category 3D anomaly detection. In this study, we propose an Information-aware Reconstruction model for Multi-category 3D Anomaly Detection (IRM3D-AD). Firstly, an Information Compression Module (ICM) is introduced to improve category specificity by removing redundant group-level features. Then, an Information-aware Mask Attention (IMA) mechanism is developed to mask key features dynamically by evaluating their information contents, thereby enhancing representation ability for all categories. Finally, an Information-aware Encoder and Decoder (IED) enforced by IMA is presented to identify anomalies from multiple categories by calculating reconstruction errors, which is optimized using information bottleneck constraints to maintain specificity and generalization. Experimental results demonstrate that our method outperforms both single-category and multicategory methods, achieving an average object-level AUROC improvement of 5.7%, 3.3%, and 6.3% on Anomaly-ShapeNet, Real3D-AD, and MulSen-AD, respectively.
While latent variable models have been commonly used for industrial soft sensing in the past decade, the performance was recently improved by the idea of deep learning. Through layer-by-layer feature extraction across different layers of the deep model, much more features can be explored from the original process data compared to the shallow model. However, using more features does not mean more satisfactory soft sensing performance, instead, the overfitting risk could be increased and the model interpretability may be reduced due to the large number of deep latent features. In this paper, a sparse learning and regularization method is introduced for modeling the set of latent features captured from the deep latent variable regression model. Through the introductions of both ℓ1 norm and ℓ2 norm into the regression model, feature selection and parameter regularization can be simultaneously implemented. A detailed industrial application example is used to demonstrate the feasibility and effectiveness of the proposed method. Compared to the basic latent variable regression model, the soft sensing performance has been significantly improved (more than 20%) by the sparse learning algorithm.
Alzheimer’s disease is characterized by heterogeneous changes across multimodal biological data, necessitating automated diagnostic frameworks that can effectively integrate diverse information sources. We propose GDG-NET, a dualgraph learning framework guided by data-driven key feature identification, which mines discriminative information from both population-level relationships and individual-level feature interactions. Specifically, we introduce a Global Association Ranking Strategy (GARS) that quantifies feature importance via randomwalk centrality on a feature correlation network. The identified key features then guide the construction of a high-quality population graph that captures clinically relevant inter-subject similarities. Concurrently, individual-level graphs model finegrained intra-subject feature interactions on a shared correlation template, with subject-specific attention producing individualized representations. An adaptive gating module dynamically balances population commonalities and subject-specific patterns for robust classification. On the ADNI dataset, GDG-NET achieves accuracies of 93.68%, 82.53%, and 69.85% on the AD vs. NC, EMCI vs. AD, and EMCI vs. LMCI tasks, respectively, significantly outperforming all baselines. Its generalizability is further validated on the PPMI Parkinson’s disease dataset. Code is available at https://github.com/xuhuisong/GDG-Net.
Hallucination in large language models (LLMs) remains a critical barrier to their reliable and trustworthy deployment, particularly in domains where factual accuracy is essential. Existing detection pipelines often rely on LLMs, which are computationally expensive and prone to inconsistent judgments across model variants and prompts. These challenges underscore the need for efficient and task-specific frameworks for hallucination detection that operate independently of large generative models. To address these concerns, this work introduces GraphHall, a unified graph-based framework that reformulates hallucination detection as relational node classification on semantic similarity graphs. Within GraphHall, three representative relational learners, Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Graph Transformers (GT), are instantiated and evaluated across three major hallucination benchmarks: PolyFEVER, FactCHD, and HaluEval, spanning dialogue, question answering, summarization, and general user-query domains. Despite operating at parameter scales several orders of magnitude smaller than modern LLMs, GraphHall achieves competitive or superior performance, approaching strong LLM baselines on PolyFEVER, surpassing them on FactCHD, and delivering strong results on HaluEval, particularly in QA and summarization. These findings demonstrate that relational modeling via semantic graphs offers an efficient, transparent, and highly effective paradigm for hallucination detection.
Recently, solving dynamic multiobjective optimization problems has become a rising research area in the evolutionary computation community. In this area, change detection is an important topic and researchers have developed several methods to detect changes. However, elusive environmental changes have brought much difficulty in existing detection methods. In order to address this issue, an indicator-based sliding window detection evolutionary algorithm, named ISEA, is proposed in this paper. Dynamic detection is important to ensure the algorithm effectively respond to environmental changes. Thus, we first propose an indicator-based sliding window detection strategy to detect environmental changes. The sliding window is formulated to store the approximated Pareto fronts at multiple consecutive time points. Subsequently, the minimum value of generational distance and inverted generational distance is designed to quantify the proximity between adjacent fronts. Second, a multi-step migration strategy is introduced to address the differences in environmental changes across various regions. The algorithm guide non-dominated points to migrate in multiple levels after environmental changes. Finally, an elite bi-directional learning strategy is developed to balance exploration and exploitation abilities. This strategy selects the proper points from the non-dominated points in the objective space and conducts bi-directional learning in the decision space. Experimental results on a series of benchmark functions with elusive change verify the efficiency of the proposed algorithm in comparison with several related algorithms.
3D object detection from point clouds is a fundamental task in computer vision, aiming to accurately localize and identify objects in 3D space. A major challenge in this domain lies in balancing detection performance with computational efficiency. Knowledge distillation has emerged as a promising solution, transferring knowledge from a large teacher network to a lightweight student model. However, existing distillation methods often treat spatial and semantic dimensions separately, fail to model geometric deformations, and rely on static teacher models, which collectively limit their effectiveness. To address these challenges, we propose a novel framework named Dynamic Riemannian Geometry Relational Distillation (D-RGRD). It integrates three key innovations for effective and dynamic knowledge transfer. First, we develop a unified distillation strategy that jointly captures spatial structures and depth-wise semantic cues, enabling more comprehensive and correlated feature transfer across 3D domains. Second, we introduce a Riemannian relational distillation module specifically designed for non-rigid structures, which embeds features into multi-curvature manifolds (spherical and hyperbolic spaces). This relaxes the rigid continuity assumptions of Euclidean space and enhances the student model’s ability to represent deformable objects such as pedestrians and cyclists. Third, we propose a dynamic teacher–student interaction mechanism that facilitates mutual learning, allowing the teacher model to dynamically adjust based on the student’s learning status. This breaks the constraints of static distillation and supports iterative, personalized knowledge transfer. Extensive experiments on the KITTI and nuScenes datasets demonstrate that D-RGRD significantly improves detection accuracy for both rigid and non-rigid objects, while maintaining high computational efficiency, thus offering a robust solution for real-world 3D perception tasks.
