Accurate building footprint databases are fundamental for sustainable urbanization yet face persistent updating challenges due to the rapid pace of urban change. Traditional methods rely on bi-temporal image comparison for change detection, which requires a large number of new labels to retrain the model, which is costly. We propose a passive updating paradigm that eliminates the reliance on historical imagery and leverages a lightweight adaptive strategy applied to Segment Anything Model (SAM) to minimize labeling costs. Furthermore, we propose a Cross Modal Temporal Fusion (CMTF) module that combines features from historical building footprints with those from recent imagery, alleviating the burden of small-sample training. The training process utilizes a semi-supervised approach, enabling the model to learn from both labeled and unlabeled regions, with labeled regions comprising only 0.4% of the building samples. Besides, we propose the RIO dataset, a sub-meter bi-temporal building footprint update dataset for studying building changes in rapidly developing areas. In addition, this work is validated on a range of cities worldwide, including Christchurch (post-earthquake reconstruction) and Beijing-Shanghai (megacity expansion). This work advances urban building renewal by overcoming the reliance on paired historical imagery for change detection and the need for large amounts of up-to-date labels. This approach offers a scalable solution for monitoring SDG 11 (Sustainable Cities and Communities), enabling less developed countries to use free and open product data to track urban expansion patterns with only a few labels.

Data collected by different modalities can provide a wealth of complementary information, such as hyperspectral image (HSI) to offer rich spectral-spatial properties, synthetic aperture radar (SAR) to provide structural information about the Earth's surface, and light detection and ranging (LiDAR) to cover altitude information about ground elevation. Therefore, a natural idea is to combine multimodal images for refined and accurate land-cover interpretation. Although many efforts have been attempted to achieve multi-source remote sensing image classification, there are still three issues as follows: 1) indiscriminate feature representation without sufficiently considering modal heterogeneity, 2) abundant features and complex computations associated with modeling long-range dependencies, and 3) overfitting phenomenon caused by sparsely labeled samples. To overcome the above barriers, a transformer-based heterogeneously salient graph representation (THSGR) approach is proposed in this paper. First, a multimodal heterogeneous graph encoder is presented to encode distinctively non-Euclidean structural features from heterogeneous data. Then, a self-attention-free multi-convolutional modulator is designed for effective and efficient long-term dependency modeling. Finally, a mean forward strategy is developed in order to avoid overfitting. Based on the above structures, the proposed model is able to break through modal gaps to obtain differentiated graph representation with competitive time cost, even for a small fraction of training samples. Experiments and analyses in three benchmark datasets with various state-of-the-art (SOTA) approaches show the performance of the proposed THSGR. The code will be available in https://github.com/jqyang22.

Effective first-trimester screening for congenital heart disease (CHD) remains an unmet clinical need, hindered by technical constraints and the lack of validated diagnostic tools. While artificial intelligence (AI) offers promise, its progress is restricted by data scarcity and privacy concerns surrounding data sharing. Federated learning (FL) offers a promising paradigm for collaborative model training without exposing sensitive patient data. In this study, we establish a Federated Congenital Heart Disease Learning to enable cross-hospital collaboration in early CHD diagnosis. A major challenge arises from inter-hospital heterogeneity, where variations in ultrasound devices, scanning protocols, and patient demographics lead to significant feature distribution shifts, resulting in poor performance. To address this, we introduce federated prototypes that align both clinical concept and disease subtype representations across participating sites, effectively calibrating local updates and enhancing global consistency. Experiments conducted across four tertiary hospitals demonstrate that our method achieves a 10.3% improvement in F1 score, 5.1% increase in sensitivity, and 1.0% improvement in specificity over state-of-the-art federated approaches. These results highlight our effectiveness in improving generalization under real-world clinical heterogeneity. Our implementation and benchmarking resources are publicly available at: https://github.com/WenkeHuang/FLCHD.

Text-to-image person retrieval (TIPR) aims to identify the target person using textual descriptions, facing challenge in modality heterogeneity. Prior works have attempted to address it by developing cross-modal global or local alignment strategies. However, global methods typically overlook fine-grained cross-modal differences, whereas local methods require prior information to explore explicit part alignments. Additionally, current methods are English-centric, restricting their application in multilingual contexts. To alleviate these issues, we pioneer a multilingual TIPR task by developing a multilingual TIPR benchmark, for which we leverage large language models for initial translations and refine them by integrating domain-specific knowledge. Correspondingly, we propose Bi-IRRA: a Bidirectional Implicit Relation Reasoning and Aligning framework to learn alignment across languages and modalities. Within Bi-IRRA, a bidirectional implicit relation reasoning module enables bidirectional prediction of masked image and text, implicitly enhancing the modeling of local relations across languages and modalities, a multi-dimensional global alignment module is integrated to bridge the modality heterogeneity. The proposed method achieves new state-of-the-art results on all multilingual TIPR datasets.

Due to the powerful feature extraction capabilities of deep learning, a series of deep learning-based methods for hyperspectral image (HSI) classification have been proposed and achieved satisfactory performance. However, most of these methods require a large number of labeled data, and the collection of completely accurate pixel-level labeled HSI data is difficult, resulting from the intricate label ambiguity of HSI and incomplete prior knowledge of annotators. Simultaneously, a few researchers focus on label ambiguity for HSI classification. Partial label learning (PLL) is one of the strategies to solve the problem where each training instance is assigned a candidate label set, among which only one is the ground truth label, which can essentially alleviate labeling difficulties. In this article, a max-gap loss-guided Siamese reconstruction network (MAGS) is proposed to combine PLL with HSI classification. MAGS consists of three components, including a spatial-spectral encoder, a spatial-spectral decoder, and a Siamese spatial-spectral encoder for high-quality feature representation learning to facilitate label disambiguation. In the encoding process, MAGS introduces the cross-attention and max-matching fusion strategies to obtain more representative features. In addition, to improve label disambiguation, the maximum gap loss is designed to guide the model training. Quantitative and qualitative results indicate that the MAGS outperforms several state-of-the-art methods on three HSI datasets. The code is available at https://github.com/Nemo96yu/MAGS.

Label skew is a widely discussed issue in federated learning. Common solutions often involve weighted parameter aggregation based on the scale of private data and the introduction of global signals to regularize the local model. However, under label skew conditions, local data exhibits diverse and imbalanced properties. Allocating high weight to clients with more samples can distract the global aggregation process. To make matters worse, global signals introduce an unavoidable computational burden. In this paper, we approach these issues from a novel over-confidence perspective. First, we prove that under severe label skew, local models exhibit greater over-confidence in private data, which appears as a smaller empirical loss owing to increased logits magnitudes. We leverage the local logits magnitude as a metric for private data balance to reweight parameter aggregation. Second, we point out that over-confidence in local data results in divergence in the aggregated classifier as different local classifiers bias distinct instance-rich classes. We address this by uniformly rescaling the normalized logits to detach the classifier weight norm from the optimization equation. This successfully calibrates the local model without relying on global signals. Empirical results on four datasets manifest the superiority of our proposed method. Our code is available at https://github.com/WenkeHuang/RevisitFL .

Due to the rising concerns on privacy protection, how to build machine learning models from distributed databases with privacy guarantees has gained more popularity. Vertical federated learning (VFL) trains machine learning models in a privacy-preserving way when the data features are scattered over distributed databases. We study the participant selection problem (PSP) for VFL, which chooses a given number of participants to conduct training while maximizing model accuracy. Compared to training with all participants, PSP can filter out hitch-riders that contribute marginally to model quality and reduce training time by involving fewer participants. To achieve good model accuracy, we formulate PSP as choosing a set of participants that maximizes the likelihood of the data samples. Then, utilizing the k-nearest neighbors (KNN) classifier as the proxy model, we express the likelihood as a function of the selected participants and prove that the function is sub modular. The submodular property is favorable as it can account for the feature diversity among the participants and allows to greedily select the participant with the maximum gain in each step. However, the selection process requires finding the top-k neighbors of a data sample as the basic operation, which is expensive in VFL setting as it involves encrypted communication. As such, we adapt the Fagin's algorithm, a famous top-k query algorithm, to reduce the amount of encrypted communication. We deploy our solution VFPS-SM across five distributed nodes and conduct experiments with 10 datasets and 3 models to evaluate its performance. The results show that VFPS-SM can reduce the end-to-end running time by up to $35\times$, selection time $365\times$ and improve model accuracy by 6.0% compared with state-of-the-art baselines.

Due to the lack of an effective auditing mechanism for malicious participants, federated learning (FL) framework faces serious threats of Byzantine attacks. Existing Byzantine resilience methods generally use the same batch size by default. However, this same batch size setting may not be applicable in practice, since clients may comprise a diverse array of devices with different data storage and computational capabilities. This paper mainly studies the robustness of Byzantine across different clients with different batch sizes in FL systems. Specifically, we propose a weighted aggregation framework (WAggF) to enhance the Byzantine robustness of the FL systems in case of different clients with different batch sizes. Combining WAggF with Byzantine attack resilient distributed (Byrd-) algorithms, we develop two Byzantine-robustness algorithms: Byrd-SGD with Different Batch sizes (Byrd-DBSGD) and Byrd-SAGA with Different Batch sizes (Byrd-DBSAGA). Theoretically, we prove that these algorithms exhibit superior performance compared to their original counterparts in the presence of Byzantine attacks. Additionally, we introduce a data partition method to solve the over-centralization problem caused by significantly disparate different batch sizes among different clients. Considering the communication efficiency, we utilize an unbiased compressor to ensure synchronous training of the FL system. To the best of our knowledge, this study represents the first investigation into the impact of different batch sizes across different clients on Byzantine robustness in FL. Extensive experimental results confirm that our proposed algorithms exhibit greater resilience to Byzantine attacks. Codes will be released upon publication.

Federated Learning (FL), as a privacy-preserving collaborative learning paradigm, helps alleviate privacy leakage and data silos challenges. However, in medical scenarios, it still suffers from the severe problem of concept shift, where class distributions differ significantly across clients, leading to semantic inconsistency and making it difficult for the global model to learn unified class representations. To address this, we propose FedProCL, a prototype-aware federated contrastive optimization method. By integrating local prototype construction, global prototype aggregation, and prototype-level contrastive constraints during training, the model is guided to achieve intra-class compactness and inter-class separability in the feature space, thereby enhancing semantic alignment. We conduct systematic evaluations on multiple datasets. Experimental results show that FedProCL significantly outperforms existing methods in terms of accuracy and also demonstrates advantages in clarity of feature representation.

Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality retrieval task due to the large modality gap. While numerous efforts have been devoted to the supervised setting with a large amount of labeled cross-modality correspondences, few studies have tried to mitigate the modality gap by mining cross-modality correspondences in an unsupervised manner. However, existing works failed to capture the intrinsic relations among samples across two modalities, resulting in limited performance outcomes. In this paper, we propose a novel Progressive Graph Matching (PGM) approach to globally model the cross-modality relationships and instance-level affinities. PGM formulates cross-modality correspondence mining as a graph matching procedure, aiming to integrate global information by minimizing global matching costs. Considering that samples in wrong clusters cannot find reliable cross-modality correspondences by PGM, we further introduce a robust Dual-Level Matching (DLM) mechanism, combining the cluster-level PGM and Nearest Instance-Cluster Searching (NICS) with instance-level affinity optimization. Additionally, we design an Outlier Filter Strategy (OFS) to filter out unreliable cross-modality correspondences based on the dual-level relation constraints. To mitigate false accumulation in cross-modal correspondence learning, an Alternate Cross Contrastive Learning (ACCL) module is proposed to alternately adjust the dominated matching, i.e., visible-to-infrared or infrared-to-visible matching. Empirical results demonstrate the superiority of our unsupervised solution, achieving comparable performance with supervised counterparts.

Vertical Federated Learning (VFL) is a privacy-preserving distributed learning paradigm where different parties collaboratively learn models with partitioned features of shared samples without leaking private data. Recent research has shown promising results addressing various challenges in VFL, highlighting its potential for practical applications in cross-domain collaboration. However, the corresponding research is scattered and lacks organization. To advance VFL research, this survey offers a systematic overview of recent developments. First, we provide a history and background introduction, along with a summary of the general training protocol of VFL. We then revisit the taxonomy in recent reviews and analyze limitations in-depth. For a comprehensive and structured discussion, we synthesize recent research from three fundamental perspectives: effectiveness, security, and applicability. Finally, we discuss several critical future research directions in VFL, which will facilitate the developments in this field. We provide a collection of research lists and periodically update them at https://github.com/shentt67/VFL_Survey.

Multitemporal hyperspectral images (HSIs) have been widely applied in change detection (CD) of different land covers for their rich spectral features and image details. However, alignment and labeling pairs of bitemporal HSIs are labor-intensive. In this article, we propose a single-temporal mask-based network (STMNet) for self-supervised HSI CD from a new perspective of detecting masks as changes. STMNet implements self-supervised by treating artificially constructed masks attached to single-temporal HSI as changed regions. To this end, we design a multiscale mask change simulation (MMCS) strategy to generate pseudo-second-temporal HSI closer to the real case. Meanwhile, a global-local feature aggregation network is proposed to enhance long-distance and local spatial-spectral feature extraction. To the best of our knowledge, this is the first work in the field of HSI CD that uses single-temporal HSIs and eliminates the need for labeling and pairing samples, alleviating the problem of difficult multitemporal HSI annotation. The visual and quantitative experimental results on three HSI datasets show that the proposed STMNet outperforms the compared state-of-the-art methods for HSI CD. Codes are available at https://github.com/Zhoutya/ChangeDetection-STMNet.

This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients, where the clients have different local model structures. Data corruption is unavoidable due to factors such as random noise, compression artifacts, or environmental conditions in real-world deployment, drastically crippling the entire federated system. To address these issues, this paper introduces a novel Robust Asymmetric Heterogeneous Federated Learning (RAHFL) framework. We propose a Diversity-enhanced supervised Contrastive Learning technique to enhance the resilience and adaptability of local models on various data corruption patterns. Its basic idea is to utilize complex augmented samples obtained by the mixed-data augmentation strategy for supervised contrastive learning, thereby enhancing the ability of the model to learn robust and diverse feature representations. Furthermore, we design an Asymmetric Heterogeneous Federated Learning strategy to resist corrupt feedback from external clients. The strategy allows clients to perform selective one-way learning during collaborative learning phase, enabling clients to refrain from incorporating lower-quality information from less robust or underperforming collaborators. Extensive experimental results demonstrate the effectiveness and robustness of our approach in diverse, challenging federated learning environments. Our code and models are public available at https://github.com/FangXiuwen/RAHFL .

Sketch-Based Person Re-Identification (Sketch ReID) aims to retrieve a person of interest across disjoint cameras using hand-drawn sketches as queries. A significant issue is the limited structural clues of sketch queries, which fail to convey high-level semantic retrieval intentions, such as colors and genders. Existing works typically combine sketches and texts for multi-modal retrieval, which inevitably introduces modality interference and relies heavily on expensive tri-modal datasets. In this paper, we propose, for the first time, an interactive and flexible sketch-based person retrieval framework that incorporates user feedback to refine the sketch person retrieval ranking without text training. A lightweight vision-to-text converting network is proposed to represent sketches with equivalent pseudo-word tokens, which aims to provide context assistance for interactive retrieval. Then, the sketch token can be seamlessly integrated with text feedback tokens within CLIP’s textual space for explicit sketch-text compositionality, thus achieving feedback-guided ranking refinement. Extensive experiments underscore the superiority of our InteractReID. Code will be available at https://github.com/littlexinyi/InteractReID.

Unsupervised visible-infrared person re-identification (USL-VI-ReID) aims to train a cross-modality retrieval model without labels, reducing the reliance on expensive cross-modality manual annotation. However, existing USL-VI-ReID methods rely on artificially cross-modality paired data as implicit supervision, which is also expensive for human annotation and contrary to the setting of unsupervised tasks. In addition, this full alignment of identity across modalities is inconsistent with real-world scenarios, where unpaired settings are prevalent. To this end, we study the USL-VI-ReID task under unpaired settings, which uses cross-modality unpaired and unlabeled data for training a VI-ReID model. We propose a novel Mapping and Collaborative Learning (MCL) framework. Specifically, we first design a simple yet effective Cross-modality Feature Mapping (CFM) module to map and generate fake crossmodality positive feature pairs, constructing a cross-modal pseudo-identity space for feature alignment. Then, a Static-Dynamic Collaborative (SDC) learning strategy is proposed to align cross-modality correspondences through a collaborative approach, eliminating inter-modality discrepancies across different aspects i.e., cluster-level and instance-level, in scenarios with cross-modal identity mismatches. Extensive experiments on the conducted SYSU-MM01 and RegDB benchmarks under paired and unpaired settings demonstrate that our proposed MCL significantly outperforms existing unsupervised methods, facilitating USL-VI-ReID to real-world deployment.

Person re-identification (ReID) aims to retrieve a target person across non-overlapping cameras. Due to the uncontrollable environment and the privacy concerns, the diversity and scale of real-world training data are usually limited, resulting in poor testing generalizability. To overcome these problems, we introduce a large-scale Weather Person dataset that generates synthetic images with different weather conditions, complex scenes, natural lighting changes, and various pedestrian accessories in a simulated camera network. The environment is fully controllable, supporting factor-by-factor analysis. To narrow the gap between synthetic data and real-world scenarios, this paper introduces a simple yet efficient domain generalization method via Single Query Adaptation (SQA), calibrating the statistics and transformation parameters in BatchNorm layers with only a single query image in the target domain. This significantly improves performance through a single adaptation epoch, greatly boosting the applicability of the ReID technique for intelligent surveillance systems. Abundant experiment results demonstrate that the WePerson dataset achieves superior performance under direct transfer setting without any real-world data training. In addition, the proposed SQA method shows amazing robustness in real-to-real, synthetic-to-real ReID, and various corruption settings. Dataset and code are available at https://github.com/lihe404/WePerson.

Federated learning, hailed as a privacy-preserving collaboration paradigm, has garnered significant attention in research circles. Typically, it involves multiple clients collaborating to integrate multi-party knowledge, facilitating the learning of a shared global model with decentralized local data. Despite the popularity of federated learning, the surge in approaches addressing various realistic challenges has highlighted a critical issue. The aggregated model may struggle to capture diverse domain knowledge across participants, leading to limited performance in cross-client domain scenarios. Furthermore, the incorporation of knowledge from participating parties can hinder generalization on out-of-client distributions. To comprehensively address this challenge, we dissect federated generalization into two dimensions: the participating domain and the unseen domain. In this paper, we propose a novel solution incorporating domain-specialized and invariant experts. These experts are designed to faithfully represent individual domain characteristics and different domain universality. Additionally, we introduce a pioneering test-time expert aggregation strategy that utilizes prediction consistency metrics to aggregate different experts, specifically tailored for handling agnostic testing distributions. Empirical results validate that our proposed methodology significantly enhances federated performance on both cross-client and out-of-client generalization under different scenarios and with various related methods. A comprehensive ablation study demonstrates the effectiveness of the proposed modules.

Ultrasound imaging is widely used in clinical diagnosis due to its non-invasive nature and real-time capabilities. However, traditional ultrasound diagnostics relies heavily on physician expertise and is often hampered by suboptimal image quality, leading to potential diagnostic errors. While artificial intelligence (AI) offers a promising solution to enhance clinical diagnosis by detecting abnormalities across various imaging modalities, existing AI methods for ultrasound face two major challenges. First, they typically require vast amounts of labeled medical data, raising serious concerns regarding patient privacy. Second, most models are designed for specific tasks, which restricts their broader clinical utility. To overcome these challenges, we present UltraFedFM, an innovative privacy-preserving ultrasound foundation model. UltraFedFM is collaboratively pre-trained using federated learning across 16 distributed medical institutions in 9 countries, leveraging a dataset of over 1 million ultrasound images covering 19 organs and 10 ultrasound modalities. This extensive and diverse data, combined with a secure training framework, enables UltraFedFM to exhibit strong generalization and diagnostic capabilities. It achieves an average area under the receiver operating characteristic curve (AUROC) of 0.927 for disease diagnosis and a dice similarity coefficient (DSC) of 0.878 for lesion segmentation. Notably, UltraFedFM surpasses the diagnostic accuracy of mid-level ultrasonographers (4-8 years of experience) and matches the performance of expert-level sonographers (10+ years of experience) in the joint diagnosis of 8 common systemic diseases.c These findings indicate that UltraFedFM can significantly enhance clinical diagnostics while safeguarding patient privacy, marking a significant advancement in AI-driven ultrasound imaging for future clinical applications.

Person re-identification (ReID) is the task of matching individuals across different camera views. Existing approaches typically employ neural networks to extract discriminative features, ranking gallery images based on their similarities to probe images. While effective, these methods are often enhanced through re-ranking, a post-processing step that refines initial retrieval results without requiring additional model training. However, current re-ranking methods mostly rely on k-nearest neighbor search to extract similar images that might have the same identity as the query, which is time-consuming with a high computation burden, limiting their applications in reality. We rethink the effect of the k-nearest neighbor search and introduce the Chebyshev’s Theorem-guided Graph Re-ranking (Cheb-GR) method, which adopts the adaptive neighbor search guided by Chebyshev’s Theorem over the k-nearest neighbor search for efficient neighbor selection. Our method leverages graph convolution operations to refine image features and achieve robust re-ranking, leading to enhanced retrieval performance. Furthermore, we provide a theoretical analysis based on Chebyshev’s Inequality to elucidate the factors contributing to the strong performance of the proposed method. Our method significantly reduces the computation costs while maintaining relatively strong performance. Through extensive experiments in both general and cross-domain settings, we demonstrate the effectiveness of Cheb-GR and its potential for real-world applications.

Recently, the disentanglement generative adversarial network (DisentGAN) unsupervised learning framework has made significant progress in remote sensing image cloud removal. However, most existing DisentGAN-based methods fail to effectively disentangle surface and cloud information within the latent space, leading to the suboptimal performance in cloud removal. Moreover, the limited feature representation capabilities of the existing haze and cloud removal methods frequently result in inadequate restoration of image texture details and colors. To address these issues, we propose a latent feature disentanglement bidirectional prompting network (LFDBP-Net) for unsupervised cloud removal. Specifically, we propose an unsupervised latent feature disentanglement (ULFD) framework that explicitly separates surface and cloud information by leveraging unpaired cloud-free latent features as priors to pull surface features closer and push cloud features apart, constrained by a bi-half-cycle reconstruction branch. Furthermore, the bidirectional prompt-guided codec mechanism (BPCM) enables coarse-to-fine feature reconstruction through a two-phase "look and think" process, in which the residual query state-space model (RQSSM) modulates encoder-decoder queries using residual-guided historical hidden state to fuse complementary local structures and global context. BPCM can progressively enhance feature discriminability and semantic expressiveness, thereby improving the image textures and colors. Finally, we construct an all-season covered, difficulty-graded cloud removal dataset, named CSRD-CR. Experiments on both our proposed dataset and public datasets demonstrate that our LFDBP-Net outperforms several state-of-the-art unsupervised haze and cloud removal methods. The source code and dataset are available at: https://github.com/nbhuangzhixuan/LFDBP-Net