Recently, instruction-driven image editing methods have demonstrated promising capabilities, requiring only a brief text to guide image modifications. However, most of them often yield suboptimal results for object editing in complex scenes, due to two major defects: (1) Over-editing, where unintended regions of the image are inadvertently altered; (2) Inability to precisely adhere to instructions, particularly in scenes with numerous elements. To resolve these issues, we propose a Single object Editing scheme, termed SoEdit, which distills complex editing tasks into single-object editing within cropped regions through a pipeline that integrates task parsing, object localization, editing, and context blending. This approach minimizes interference from irrelevant areas, ensures proper object size and placement, and ultimately enhances model performance. Furthermore, we introduce a lightweight Spatially-Adaptive Mixture of Experts (SAMOE) to better model spatial heterogeneity, enabling tokenwise adaptive processing and further enhancing the overall editing capability with minimal additional parameters. Moreover, we introduce a large-scale object-centric dataset to further optimize the model. Extensive experiments demonstrate that SoEdit outperforms existing methods, especially in precise responses to fine-grained editing requirements, such as multi-action and quantity-sensitive object editing.

Recent research on anomaly detection in power grid communications has evolved from traditional rule-based systems to deep learning paradigms. However, the complex spatio-temporal dependencies of network traffic and the lack of labeled samples limit the practical performance of existing models. To address this, we propose an unsupervised Spatio-Temporal Graph Autoencoder framework. By modeling network traffic as dynamic directed graphs, this framework integrates Graph Convolutional Networks (GCNs) with Long Short-Term Memory (LSTM) units to jointly encode spatial correlations and temporal dynamics into a low-dimensional latent space. Our method characterizes normal traffic patterns using only benign samples and identifies anomalies by measuring the reconstruction error between the observed traffic and the learned normal profile. We perform evaluations on the UNSW-NB15 benchmark and a real-world power grid network dataset. Our method not only achieves high detection accuracies of 95.20% and 94.28% on the respective datasets, but also delivers superior comprehensive performance on the UNSW-NB15 benchmark with an F1-score of 95.22%, a precision of 94.80%, and a recall of 95.65%. Experimental results indicate that the proposed method outperforms supervised baselines and standalone deep learning models, effectively improving the accuracy and robustness of anomaly detection for power grid communication networks.

Remote photoplethysmographic (rPPG) technology enables non-contact monitoring of vital signs. However, the existing methods still face challenges such as insufficient video modeling capability in single-expert architectures and strong coupling-induced interference between different physiological signals. To address these issues, we propose PhysHybrid, which integrates the local dynamics captured by a convolutional neural network (CNN) and the global dependencies modeled by a Transformer through a gated fusion module. Subsequently, a shared decoder is used to predict the blood volume pulse (BVP) signal, leveraging the advantages of both CNN and Transformer. In addition, we further explore the feasibility of estimating oxygen saturation (SpO2) from facial videos by disentangling its composition from BVP in frequency domain, so as to remove the components related to heart rate (HR) that interfere with SpO2 estimation. Extensive experiments conducted on multiple public datasets demonstrate that PhysHybrid significantly outperforms single-expert (CNN or Transformer) baselines and the multi-task learning baselines without using disentangled BVP clue a priori, while exhibiting stronger robustness and interpretability.

Artistic font generation aims to synthesize stylized glyphs based on a reference style. However, existing approaches suffer from limited style diversity and coarse control. In this work, we explore the potential of element-driven artistic font generation. Elements are the fundamental visual units of a font, serving as reference images for the desired style. Conceptually, we categorize elements into object elements (e.g., flowers or stones) with distinct structures and amorphous elements (e.g., flames or clouds) with unstructured textures.We introduce FontCrafter, an element-driven framework for font creation, and construct a large-scale dataset, ElementFont, which comprises a diverse set of element types and high-quality glyph images. However, achieving high-fidelity reconstruction of both the texture and structure of reference elements remains challenging. To address this, we propose an in-context generation strategy that treats element images as visual context and uses an inpainting model to transfer element styles into glyph regions at the pixel level.To further control glyph shapes, we design a lightweight Context-aware Mask Adapter (CMA) that injects shape information while maintaining style consistency. Moreover, a training-free attention redirection mechanism enables region-aware style control and suppresses stroke hallucination. Extensive experiments demonstrate that FontCrafter achieves strong zero-shot generation performance, especially in preserving the structural and textural fidelity, while supporting flexible controls, such as style mixture. The model and dataset will be made publicly available.

The proliferation of privacy-sensitive data has spurred the development of federated learning (FL), which is an important technology for state-of-the-art machine learning and responsible AI. However, most existing FL methods are constrained in their applicability and generalizability due to their narrow focus on specific tasks. This paper presents a novel federated multi-task learning (FMTL) framework that is capable of acquiring knowledge across multiple tasks. To address the challenges posed by Non-IID data and task imbalance in FMTL, this study proposes a federated fusion strategy based on dynamic guided attention (FedDGA), which adaptively fine-tunes local models for multiple tasks with personalized attention. In addition, this paper designed dynamic batch weight (DBW) to balance the task losses and improve the convergence speed. Extensive experiments were conducted on various datasets, tasks and settings, and the proposed method was compared with state-of-the-art methods such as FedAvg, FedProx and SCAFFOLD. The results show that our method achieves significant performance gains, with up to 11.1% increase in accuracy over the baselines.

Various multi-instance learning (MIL) based approaches have been developed and successfully applied to whole-slide pathological images (WSI). Existing MIL methods emphasize the importance of feature aggregators, but largely neglect the instance-level representation learning. They assume that the availability of a pre-trained feature extractor can be directly utilized or fine-tuned, which is not always the case. This paper proposes to pre-train feature extractor for MIL via a weakly-supervised scheme, i.e., propagating the weak bag-level labels to the corresponding instances for supervised learning. To learn effective features for MIL, we further delve into several key components, including strong data augmentation, a non-linear prediction head and the robust loss function. We conduct experiments on common large-scale WSI datasets and find it achieves better performance than other pre-training schemes (e.g., ImageNet pre-training and self-supervised learning) in different downstream tasks. We further show the compatibility and scalability of the proposed scheme by deploying it in fine-tuning the pathological-specific models and pre-training on merged multiple datasets. To our knowledge, this is the first work focusing on the representation learning for MIL.

IoT devices have been widely deployed in medical industry, in the objective of improving diagnostic accuracy and increasing the efficiency of healthcare systems. However, traditional centralized learning approaches often fall short in meeting strict privacy requirements and adapting to emerging diseases in clinical environment. To address this, we propose a novel federated continual learning (CL) framework for disease diagnosis (FCL4DD), designed to enable distributed and incremental learning of new disease classes while safeguarding data privacy. To combat catastrophic forgetting in CL, FCL4DD integrates a replay strategy powered by a weakly supervised diffusion model (WSDM) to generate historical data for diagnosis model training. The WSDM leverages weak supervision into diffusion model to capture the diverse characteristics of the real data, enabling the generation of high-quality synthetic samples that maintain the data's inherent variability. To overcome the challenges of nonindependent and identically distributed (non-IID) data in federated learning, WSDM is deployed at the central server to generate synthetic disease data that conforms to the global distribution. This synthetic data is then used to retrain client models, reducing discrepancies and enhancing performance consistency across clients. Evaluations on various datasets demonstrates that our method outperforms other state-of-the-art approaches, such as FedEWC, FedLwF, FedWeIT, TARGET, and DDDR, achieving up to a 4.85% accuracy improvement over the second-best method. Code are available at https://github.com/hysshy/FCL4DD.

Federated learning (FL) faces significant challenges in scenarios with nonindependent and identically distributed (non-IID) data distributions across participating clients. Traditional aggregation-based approaches often struggle with the inherent misalignment between local and global optimization objectives, which leads to gradient divergence and suboptimal generalization performance. This article proposes a novel FL framework that replaces conventional aggregation with a biologically inspired model symbiosis approach called FedSym, which employs a dual-level symbiotic mechanism. Ectosymbiosis performs coarse-grained hierarchical parameter recombinations through random layer-wise model combination, while endosymbiosis enables fine-grained intralayer parameter fusion through weighted averaging, collectively steering model updates toward flatter loss landscapes. Our theoretical analysis demonstrates that FedSym's convergence rate is O(1/T) under non-IID conditions, which matches the convergence properties of FedAvg. Extensive evaluations across multiple datasets and model architectures show that FedSym achieves substantial improvements over state-of-the-art FL methods, particularly in challenging scenarios with high data heterogeneity, and demonstrates robust performance across varying numbers of participating clients and federation scales.

Personalized federated learning (PFL) has emerged as an efficient way to tailor global models to individual data features. However, in real-world deployments, the dynamic nature of client participation-where clients may join or quit arbitrarily-poses significant challenges for PFL systems. We propose elastic PFL (EPFL), an EPFL framework that accommodates dynamic client participation while preserving model performance. Our framework addresses two fundamental challenges: 1) seamlessly generating personalized models for newly joining unlabeled clients and 2) achieving efficient and precise multigrain unlearning for quitting clients. To address these challenges, we utilize hypernetworks for client-specific model generation and introduce a novel client embedding regularization (CER) to enhance generalization capabilities for new clients. Furthermore, we design an adversarial purification mechanism that enables efficient multigrained unlearning at the client, sample, and class levels. We provide theoretical analysis establishing both generalization bounds for new clients and convergence guarantees for the unlearning process. Extensive experimental results demonstrate that EPFL outperforms the existing PFL methods in terms of both accuracy and generalization capabilities. Through backdoor trigger experiments, we show that EPFL achieves unlearning efficacy and fidelity comparable to complete retraining while requiring only 3%-10% of the computational time of these existing approaches.

This study focuses on the classification of space signals detected by the Solar Orbiter mission. Due to the complex space environment, dust impacts and natural waves are crucial to understand. Dust impacts have been studied using various methods and instruments over the years. Langmuir waves and electrostatic solitary waves (ESWs) are also important phenomena in space plasmas. This study considers four classes: non-dust signals, dust signals, Langmuir waves, and ESWs. The study uses Solar Orbiter data from November 2022 to September 2023. The data is preprocessed to reduce noise and improve its quality. We utilize the inverse Fourier transform for reconstructing signals, median filtering for noise reduction, resampling to achieve uniformity, and data augmentation to increase the diversity of the dataset. The dataset contains 4800 samples, which are divided into training, validation, and test sets. Multiple classification models are utilized, among which the multiscale residual network (MSRN) performed excellently, achieving the highest performance with a classification precision of 99.37%. The MSRN model shows superior performance with high accuracy, recall, and F1-scores. The confusion matrix and t-SNE visualization further confirm the model’s effectiveness in capturing signal features and distinguishing different classes.

Non-contact facial video-based heart rate estimation using remote photoplethysmography (rPPG) has shown great potential in many applications (e.g., remote health care) and achieved creditable results in constrained scenarios. However, practical applications require results to be accurate even under complex environment with head movement and unstable illumination. Therefore, improving the performance of rPPG in complex environment has become a key challenge. In this paper, we propose a novel video embedding method that embeds each facial video sequence into a feature map referred to as Multi-scale Adaptive Spatial and Temporal Map with Overlap (MAST_Mop), which contains not only vital information but also surrounding information as reference, which acts as the mirror to figure out the homogeneous perturbations imposed on foreground and background simultaneously, such as illumination instability. Correspondingly, we propose a two-stream Transformer model to map the MAST_Mop into heart rate (HR), where one stream follows the pulse signal in the facial area while the other figures out the perturbation signal from the surrounding region such that the difference of the two channels leads to adaptive noise cancellation. Our approach significantly outperforms all current state-of-the-art methods on two public datasets MAHNOB-HCI and VIPL-HR. As far as we know, it is the first work with Transformer as backbone to capture the temporal dependencies in rPPGs and apply the two stream scheme to figure out the interference from backgrounds as mirror of the corresponding perturbation on foreground signals for noise tolerating.

Transfer learning aims to reuse the learnt representations or subnetworks to a new domain with minimum effort for adaption. Here, the challenge lies in the mismatch between source domain and target domain, which is the major gap to be tackled by transfer learning. Hence, how to identify the mismatch between source and target domain becomes a critical problem. We propose an end-to-end framework to learn feature compensation for transfer learning with soft gating to decide whether and how much feature compensation is needed, accounting for the mismatch between source domain and target domain. To enable identifying the position of the input in reference to the overall data distribution of source domain, we perform clustering at first to figure out the data distribution in a compact form represented by cluster centers, and then use the similarities between the input and the cluster centers to describe the relative position of the input in reference to the cluster centers. This acts as the context to indicate whether and how much feature compensation is needed for the input to compensate for the mismatch between source domain and target domain. To approach that, we add only two subnetworks in the form of Multilayer Perceptron, one for computing the feature compensation and the other for soft gating the compensation, where both are computed based on the context. The experiments show that such minor change to backbone network can result in significant performance improvements compared with the baselines on some widely used benchmarks.

Accurate forecasting for photovoltaic power generation is one of the key enablers for the integration of solar photovoltaic systems into power grids. Existing deep-learning-based methods can perform well if there are sufficient training data and enough computational resources. However, there are challenges in building models through centralized shared data due to data privacy concerns and industry competition. Federated learning is a new distributed machine learning approach which enables training models across edge devices while data reside locally. In this paper, we propose an efficient semi-asynchronous federated learning framework for short-term solar power forecasting and evaluate the framework performance using a CNN-LSTM model. We design a personalization technique and a semi-asynchronous aggregation strategy to improve the efficiency of the proposed federated forecasting approach. Thorough evaluations using a real-world dataset demonstrate that the federated models can achieve significantly higher forecasting performance than fully local models while protecting data privacy, and the proposed semi-asynchronous aggregation and the personalization technique can make the forecasting framework more robust in real-world scenarios.

Video Compression Artifact Reduction aims to reduce the artifacts caused by video compression algorithms and improve the quality of compressed video frames. The critical challenge in this task is to make use of the redundant high-quality information in compressed frames for compensation as much as possible. Two important possible compensations: Motion compensation and global context, are not comprehensively considered in previous works, leading to inferior results. The key idea of this paper is to fuse the motion compensation and global context together to gain more compensation information to improve the quality of compressed videos. Here, we propose a novel Spatio-Temporal Compensation Fusion (STCF) framework with the Parallel Swin-CNN Fusion (PSCF) block, which can simultaneously learn and merge the motion compensation and global context to reduce the video compression artifacts. Specifically, a temporal self-attention strategy based on shifted windows is developed to capture the global context in an efficient way. for which we use the Swin transformer layer in the PSCF block. Moreover. an additional Ada-CNN layer is applied in the PSCF block to extract the motion compensation. Experimental results demonstrate that our proposed STCF framework outperforms the state-of-the-art methods up to 0.23dB (27% improvement) on the MFQEv2 dataset.

Semantic image editing provides users with a flexible tool to modify a given image guided by a corresponding segmentation map. In this task, the features of the foreground objects and the backgrounds are quite different. However, all previous methods handle backgrounds and objects as a whole using a monolithic model. Consequently, they remain limited in processing content-rich images and suffer from generating unrealistic objects and texture-inconsistent backgrounds. To address this issue, we propose a novel paradigm, Semantic Image Editing by Disentangling Object and Background (SIEDOB), the core idea of which is to explicitly leverages several heterogeneous subnetworks for objects and backgrounds. First, SIEDOB disassembles the edited input into background regions and instance-level objects. Then, we feed them into the dedicated generators. Finally, all synthesized parts are embedded in their original locations and utilize a fusion network to obtain a harmonized result. Moreover, to produce high-quality edited images, we propose some innovative designs, including Semantic-Aware Self-Propagation Module, Boundary-Anchored Patch Discriminator, and Style-Diversity Object Generator, and integrate them into SIEDOB. We conduct extensive experiments on Cityscapes and ADE20K-Room datasets and exhibit that our method remarkably outperforms the baselines, especially in synthesizing realistic and diverse objects and texture-consistent backgrounds. Code is available at https://github.com/WuyangLuo/SIEDOB.

Choreography refers to creation of dance motions according to both music and dance knowledge, where the created dances should be style-specific and consistent. However, most of the existing methods generate dances using the given music as the only reference, lacking the stylized dancing knowledge, namely, the flag motion patterns contained in different styles. Without the stylized prior knowledge, these approaches are not promising to generate controllable style or diverse moves for each dance style, nor new dances complying with stylized knowledge. To address this issue, we propose a novel music-to-dance generation framework guided by style embedding, considering both input music and stylized dancing knowledge. These style embeddings are learnt representations of style-consistent kinematic abstraction of reference dance videos, which can act as controllable factors to impose style constraints on dance generation in a latent manner. Hence, we can make the style embedding fit into any given style while allowing the flexibility to generate new compatible dance moves by modifying the style embedding according to the learnt representations of a certain style. We are the first to achieve knowledge-driven style control in dance generation tasks. To support this study, we build a large multi-style music-to-dance dataset referred to as I-Dance. The qualitative and quantitative evaluations demonstrate the advantage of the proposed framework, as well as the ability to synthesize diverse moves under a dance style directed by style embedding.

Customer volume prediction is crucial for a variety of urban applications, such as store location selection. So far, the key challenge lies in how to fuse multiple modalities from different data sources, on account of the massive amount of data accessible, for example, spatio-temporal data and satellite images. In this article, we investigate three dynamic weighting ensemble learning models to fuse spatio-temporal features and visual features for predicting customer volume in the urban commercial district of interest. Specifically, we propose the shared-private dynamic weighting model by incorporating graph neural networks, which is proposed to capture geographic dependencies (i.e., competitiveness or dependencies) between urban commercial districts in an end-to-end manner. To the best of our knowledge, it is the first work to utilize graph neural networks to model such geographic relationships. We conduct a series of experiments to demonstrate the effectiveness of the proposed models based on two real datasets. Furthermore, an elaborated visualization method is performed for knowledge discovery.

Token filtering to reduce irrelevant tokens prior to self-attention is a straightforward way to enable efficient vision Transformer. This is the first work to view token filtering from a feature selection perspective, where we weigh the importance of a token according to how much it can change the loss once masked. If the loss changes greatly after masking a token of interest, it means that such a token has a significant impact on the final decision and is thus relevant. Otherwise, the token is less important for the final decision, so it can be filtered out. After applying the token filtering module generalized from the whole training data, the token number fed to the self-attention module can be obviously reduced in the inference phase, leading to much fewer computations in all the subsequent self-attention layers. The token filter can be realized using a very simple network, where we utilize multi-layer perceptron. Except for the uniqueness of performing token filtering only once from the very beginning prior to self-attention, the other core feature making our method different from the other token filters lies in the predictability of token impact from a feature selection point of view. The experiments show that the proposed method provides an efficient way to approach a light weighted model after optimized with a backbone by means of fine tune, which is easy to be deployed in comparison with the existing methods based on training from scratch.

Face inpainting aims at plausibly predicting missing pixels of face images within a corrupted region. Most existing methods rely on generative models learning a face image distribution from a big dataset, which produces uncontrollable results, especially with large-scale missing regions. To introduce strong control for face inpainting, we propose a novel reference-guided face inpainting method that fills the large-scale missing region with identity and texture control guided by a reference face image. However, generating high-quality results under imposing two control signals is challenging. To tackle such difficulty, we propose a dual control one-stage framework that decouples the reference image into two levels for flexible control: High-level identity information and low-level texture information, where the identity information figures out the shape of the face and the texture information depicts the component-aware texture. To synthesize high-quality results, we design two novel modules referred to as Half-AdaIN and Component-Wise Style Injector (CWSI) to inject the two kinds of control information into the inpainting processing. Our method produces realistic results with identity and texture control faithful to reference images. To the best of our knowledge, it is the first work to concurrently apply identity and component-level controls in face inpainting to promise more precise and controllable results.

Federated learning can collaboratively train AI models while protecting data privacy. In practical industry environment, non-independent and identically distributed (Non-IID) characteristics of data affect the effectiveness of federated learning. Personalized federated learning can help resolve this, but it cannot adapt to unknown data. In addition, practical applications also call for trusted training environment and remain stable when there are security threats. In this article, we propose a credible federated self-learning (CFSL), based on the idea of hypernetwork supported by blockchain to achieve secured, credible, personalized federated self-learning, especially, for unknown data in Non-IID environment. Extensive experiments on three Non-IID data sets demonstrate the capabilities on adaptive resilience for security attacks and on accuracy of recognizing unknown objects, with good performance at the same time. CFSL outperforms the existing personalized federated learning methods, with an increase in average accuracy by 4.11%.