The sparsity of solutions in certain problems typically exhibits strong correlations with the input data, which highlights the need for generalized models to facilitate effective learning. Multi-task learning rises to this challenge by incorporating auxiliary tasks, enabling the transfer of information across tasks to enhance model learning and representation. However, auxiliary tasks do not always contribute equally to the improvement of the main task. Given the nonlinear relationships between tasks, accurately capturing these interactions is challenging. Additionally, the optimal collaborative combination weights may be different as the main task changes. To address the above problems, this paper proposes an evolution-based multi-level collaborative multi-task sparse learning framework (EMCMSL), which integrates evolutionary optimization into the sparse learning process to enable both intra-model and inter-model task collaboration. At the intra-model level, auxiliary tasks are incorporated through shared network parameters to jointly enhance feature representation. At the inter-model level, a tailored evolutionary multi-task optimization mechanism is employed to search for optimal collaborative weight configurations across multiple sparse problems. This collaboration enables EMCMSL to automatically discover beneficial cooperative structures for different main tasks while mitigating negative transfer. The proposed framework is evaluated on two main tasks: sparse reconstruction and denoising. The experimental results validate the effectiveness of multi-level task collaboration and demonstrate that the proposed algorithm can achieve good results on different data.

Existing federated remote sensing scene classification methods generally assume a fixed client network architecture,disregarding the significant heterogeneity in computing power,storage,and energy supply among devices such as satellites and drones.These limitations make deploying models on resource-constrained terminals difficult,restricting the practical efficiency of federated systems.Therefore,a novel federated remote sensing paradigm that is capable of dynamically adapting to the heterogeneity of client data and computational resources must be urgently developed.While safeguarding the data privacy of all parties,this paradigm should collaboratively explore optimal network architectures that are lighter,more computationally efficient,and tailored to the specific characteristics of each client's local data.To address this bottleneck,this study proposes a federated collaborative framework with the autonomous evolution of neural network architectures.It proposes a neural architecture search mechanism based on path and data collaborative sampling to achieve lightweight,efficient,and personalized model construction that adapts to local data characteristics.During the collaborative training phase,each client dynamically screens key subnet architectures through gradient norm-guided path sampling and focuses on high-value samples through data sampling based on gradient upper bounds,significantly reducing supernetwork training and communication overhead.In the multiparty deployment phase,evolutionary algorithms are used to search for and validate the personalized subnet with the optimal verification accuracy on the basis of local data.The proposed framework's foundational breakthrough lies in its integrated approach wherein gradient norm-guided path sampling dynamically identifies and prioritizes architecturally critical subnets during federated training.Simultaneously,gradient-capped data sampling concentrates computational resources on samples with significant effect on training outcomes.These mechanisms collectively form a synergistic strategy that substantially reduces gradient variance and training overhead across the hypernetwork while respecting client resource constraints.Following this collaborative hypernetwork development,each client autonomously executes evolutionary optimization,extracting customized subnetworks that are precisely tailored to local data distributions through adaptation-driven architecture exploration.By unifying adaptive sampling during training with evolutionary personalization while on deployment,the framework achieves unprecedented efficiency in generating lightweight but high-performance models.These models are optimized for diverse edge devices and their unique remote sensing environments while remaining within strict privacy-preserving federated parameters.Experiments on four benchmark heterogeneous remote sensing classification datasets,namely,AID,NWPU-RESISC45,PatternNet,and MEET,demonstrate that this method significantly outperforms fixed architectures and mainstream pruning methods under nonindependent and identically distributed conditions.It can evolve lightweight,high-performance,and dedicated network architectures for heterogeneous clients while protecting data privacy,effectively enhancing the deployment feasibility and overall performance of federated remote sensing systems.Ablation studies have confirmed that the integrated path-data sampling strategy is pivotal to these gains,reducing gradient variance and improving subnet consistency.This work resolves key bottlenecks in federated remote sensing systems by enabling resource-constrained clients to evolve specialized architectures autonomously.The integration of gradient-guided path sampling and data sampling optimizes training efficiency,while evolutionary optimization facilitate personalized subnet deployment.The proposed framework demonstrably enhances deployment feasibility and overall system performance without compromising data privacy,establishing an effective solution for heterogeneous federated learning in remote sensing scene classification.

Fusing visible and near-infrared (NIR) images is crucial for various engineering applications. Accurate alignment of visible and NIR images relies on local features specifically tailored to the unique characteristics of infrared images. The potential of DNN-derived local features remains largely underexplored in this domain. This study aims to thoroughly investigate feature extraction methodologies for aligning visible and NIR images. An innovative DNN-based feature extraction technique is introduced, which employs a streamlined fully convolutional network to derive dense features and saliency scores. The proposed model is trained unsupervised through dual training with adaptive knowledge transfer (DT-AKT) strategies. The training process utilizes visible images, which avoids the need for numerous NIR images. The proposed approach generates a flexible number of key points and provides more robust local features for aligning visible and NIR images compared to many state-of-the-art methods, particularly regarding the stability, repeatability and distinctiveness of local features.

Graph structure learning has substantially advanced the analysis of graph data with unknown structures across numerous domains. Existing graph structure learning methods broadly follow a low-pass filtering scheme, where edges are generated only through nearest neighbor sampling and then fed into a low-pass filtering graph convolutional network to obtain smooth node representations. In this work, we present a dual-channel graph structure learning framework composed of low-pass and high-pass filtering channels. The high-pass channel is designed to capture differential node representations by constructing edges via farthest neighbor sampling and processing them through a high-pass graph convolutional network. However, our analysis reveals that the existing edge sampling strategies may not invariably be necessary from the perspective of information gain. Our solution centers on a dual-channel architecture leveraging a trainable neural Gaussian function to model similarities and connectivity, thereby mitigating the aforementioned issue. To enhance scalability, we introduce a method that transfers the large-scale graph to a transition graph, significantly reducing time and memory complexity. The effectiveness and efficiency of the proposed approach are demonstrated through experiments on multiple benchmark datasets spanning graph, text, and image data.

Humor is a unique expression in the human language system that plays a critical part in human communication. Most earlier efforts on humor detection concentrated on a single text modality, without considering the complex potential humor associations between the context and punchline of other modalities. In addition, there is no unifying paradigm for multi-modal feature fusion across the few extant multi-modal humor detection methods. To alleviate the aforementioned issues, we propose a bi-temporal collaborative fusion network (BCFNet) for multi-modal humor detection, to construct a novel collaborative paradigm in context and multi-modality. For the collaboration between punchlines and their contexts, we propose a bi-temporal collaborative classification (BTCC) framework. In the BTCC, we design two different humor classifiers for the punchline domain and the context domain, to improve the detection performance by integrating the classification results from the two domains. For the collaboration between three modalities of text, audio, and vision, we propose a multi-modal collaborative fusion (MMCF) framework. In the MMCF, we improve the multi-modal fusion process by designing audio-text collaborative attention (Audio-Text CA) and vision-text collaborative attention (Vision-Text CA) mechanisms, which respectively combine the audio and visual modalities with the text modality. Finally, the multi-modal fusion feature, which is further utilized collaboratively for the downstream classification task, is obtained by weighting and adding two bi-modal features with the text feature. BCFNet achieved significant improvements on the existing multi-modal humor detection benchmark dataset UR-FUNNY. A series of ablation experiments and case analyses have shown that each component of BCFNet is effective.

The deployment of federated learning (FL) in real-world Internet of Things (IoT) ecosystems presents intrinsic challenges stemming from hardware asymmetry and sample scarcity, the existing related approaches generally homogenize model architectures and assume abundant labeled data, resulting in an inability to achieve the fast generalization on devices with varying computational capabilities and dynamic task conditions. To address the aforementioned challenges, we propose a novel federated cross-device heterogeneous few-shot adaptation (Fed-CHFSA) method for IoT systems. In Fed-CHFSA, collaborating with other devices, each edge device obtains a personalized model that can not only adapt well to the category distribution of respective local data but also recognize unseen categories without data leakage. Specifically, we designed a fine-grained personalized aggregation (FPA) module and an information entropy-driven adaptive feature constraint (EAFC) module for the devices possessing a small amount of labeled data in the model aggregation and training phases of Fed-CHFSA, respectively. In each round of global communication, the edge device performs a certain epoch of personalized training locally under the normalization of EAFC in the feature space. Subsequently, the central server follows the FPA to finely aggregate the received model updates parameterwise, and redistribute the updated global model to participating devices. After multiple rounds of global communication, every edge device acquires an optimal model more adaptable to local data and more generalized to unseen categories. Compared with existing FL and PFL algorithms on three benchmark few-shot learning (FSL) datasets, the proposed Fed-CHFSA framework achieves the best performance. The effectiveness of FPA and EAFC is also demonstrated by extensive ablation experiments.

Energy-based models are effective for out-of-distribution (OOD) detection, but extending them to multiplex graphs remains challenging. A direct strategy performs energy propagation independently within each structure and averages the propagated scores to obtain an OOD score. This strategy neglects energy exchange between corresponding nodes across structures and may miss complementary semantic information. To address this limitation, we propose Energy Multiplexing Propagation (EMP), a framework with two coordinated mechanisms, node-topology propagation and multiplex-structure propagation. The former smooths energy within individual structures to maintain local consistency, whereas the latter captures latent inter-structure correlations for cross-structure interaction. Based on these mechanisms, we introduce two variants, EMP Seq and EMP Para. EMP Seq uses a sequential architecture to prioritize local consistency before global integration, while EMP Para uses a parallel dual-branch architecture to fuse multiplex energy. Experiments on six real-world benchmarks show that EMP improves OOD node detection over representative baselines across different scenarios. The implementation code is available at https://github.com/xiaolongo/EMP.

Extracting effective global image features is critical for detecting frame change points, a vital task in video analysis applications. DNN-derived features have gained prominence across various applications over the last two decades. However, these features often falter when detecting frame change points in videos with homogeneous consecutive frames. Therefore, rather than relying solely on individual images, integrating temporal differences between frames into DNN-derived features is essential. This study introduces a novel approach for extracting discriminative class-specific features through learning temporal differences. Based on meticulously designed loss functions, a self-supervised contrastive learning siamese framework is proposed. Furthermore, a new algorithm (Multi-KTS) based on class-specific features is proposed for frame change point detection. The effectiveness of the proposed feature learning methods was evaluated using two types of datasets. Compared to state-of-the-art deep features, the class-specific features we propose demonstrate significantly higher discriminability between homogeneous frames and semantic classes in a video.

Self-supervised learning, which leverages unlabeled data to learn useful feature representations by constructing auxiliary tasks, has been widely explored in few-shot scene classification to improve the feature representation and generalization capabilities of deep models in scarce data. However, most of the current related work adopts specific self-supervised auxiliary tasks (SSATs) for combinatorial improvement, and does not explore the intrinsic connection between different pretext tasks. In practice, the linkage of SSATs is complex, and the optimization of task-sharing parameters by minimizing linear combinations of losses can be conflicting. In addition, although a single combination of SSAT can improve certain performance on the baseline, it is not the personalized optimal solution on various remote sensing datasets with diverse properties. In this article, we propose a collaborative self-supervised evolution (so-called CSENet) framework for few-shot remote sensing scene classification to automatically search for appropriate weights in balancing the task conflicts. In contrast to most existing methods, which consider all SSATs to be equally efficacious or fixed-weighted for the few-shot main task, CSENet achieves autonomous co-evolutionary optimization by encoding arbitrary self-supervised weights. Specifically, the complex self-supervised combinations for different remote sensing data are transformed into an evolutionary optimization problem, where chromosomes with weighting variables obtain the optimal combination with genetic operators. Based on the transfer learning few-shot training paradigm, CSENet first efficiently searches for optimal self-supervised combinations with potential by the proposed automatic collaborative evolution strategy and automatically adjusts the weights without manual settings. Importantly, CSENet provides both inductive and transductive inference, and supports the embedding of arbitrary SSATs. The effectiveness of the proposed framework is demonstrated by state-of-the-art (SOTA) results on three benchmark datasets.

Oriented detectors have become the mainstream of object detection in remote-sensing images since they provide more precise bounding boxes and contain less background. However, there remain several challenges that restrict the detection performance and need to be tackled. This article focuses on the following two aspects: 1) numerous tiny objects in remote-sensing images pose a challenge for the detectors pursuing high recall and accurate localization and 2) specific categories with large aspect ratios and arbitrary angles also trouble the regression of the detectors. We attempt to alleviate the above problems by constructing a weak feature extraction network (WFEN) and a dual appearance-aware (DA) loss. Specifically, WFEN is used to extract hierarchical weight vectors for multiscale feature layers by employing a lightweight convolutional module, aiming to fuse activation features distributed in different layers and provide pure features for subsequent regression and classification. DA loss is tailored to regressions of tiny and slender objects by dynamically modulating the associated loss on objects with various appearances, which consists of two auxiliary losses, termed scale-aware loss ${\mathcal {L}}_{S}$ and aspect-ratio-aware loss ${\mathcal {L}}_{A}$ . These two components can contribute to each other, that is, the former provides more accurate features for detection tasks, while the latter can reciprocate the former by imposing constraints on crucial objects, and together constitute an appearance sensitivity detector (ASDet). Extensive experiments on three public datasets demonstrate that our ASDet outperforms all refine-stage detectors in terms of accuracy while maintaining the superior inference speed of single-stage counterparts.

Structurally similar data exist in most practical semantic segmentation applications. For example, objects can appear identical or positionally similar in many images, such as video frames. Objects with structural similarity in data samples can confuse deep neural networks (DNNs) in semantic segmentation applications. These challenges often lead to lower pixel classification accuracy of natural object segmentation. This study proposes a novel approach (S2-GCN) that enhances CNN-based semantic segmentation for structurally similar data using a contrastive graph convolutional network (GCN). By selecting specific label pairs and developing a customized GCN branch parallel to an encoder-decoder backbone, our method significantly improves accuracy, IoU, and F1-score, by up to 8%, as demonstrated through an extensive evaluation of five datasets. Our findings show that our proposed method effectively addresses the structural similarity problem of CNN-based semantic segmentation and can be applied to a wide range of practical applications.

Graph Structure Learning (GSL) has demonstrated considerable potential in the analysis of graph-unknown non-Euclidean data across a wide range of domains. However, constructing an end-to-end graph structure learning model poses a challenge due to the impediment of gradient flow caused by the nearest neighbor sampling strategy. In this paper, we construct a differential graph structure learning model by replacing the non-differentiable nearest neighbor sampling with a differentiable sampling using the reparameterization trick. Under this framework, we argue that the act of sampling nearest neighbors may not invariably be essential, particularly in instances where node features exhibit a significant degree of similarity. To alleviate this issue, the bell-shaped Gaussian Similarity (GauSim) modeling is proposed to sample non-nearest neighbors. To adaptively model the similarity, we further propose Neural Gaussian Similarity (NeuralGauSim) with learnable parameters featuring flexible sampling behaviors. In addition, we develop a scalable method by transferring the large-scale graph to the transition graph to significantly reduce the complexity. Experimental results demonstrate the effectiveness of the proposed methods.

Point cloud registration (PCR) is a popular research topic in computer vision. Recently, the registration method in an evolutionary way has received continuous attention because of its robustness to the initial pose and flexibility in objective function design. However, most evolving registration methods cannot tackle the local optimum well and they have rarely investigated the success ratio, which implies the probability of not falling into local optima and is closely related to the practicality of the algorithm. Evolutionary multi-task optimization (EMTO) is a widely used paradigm, which can boost exploration capability through knowledge transfer among related tasks. Inspired by this concept, this study proposes a novel evolving registration algorithm via EMTO, where the multi-task configuration is based on the idea of solution space cutting. Concretely, one task searching in cut space assists another task with complex function landscape in escaping from local optima and enhancing successful registration ratio. To reduce unnecessary computational cost, a sparse-to-dense strategy is proposed. In addition, a novel fitness function robust to various overlap rates as well as a problem-specific metric of computational cost is introduced. Compared with 8 evolving approaches, 4 traditional approaches and 3 deep learning approaches on the object-scale and scene-scale registration datasets, experimental results demonstrate that the proposed method has superior performances in terms of precision and tackling local optima.

Multi-view point cloud registration is fundamental in 3D reconstruction. Since there are close connections between point clouds captured from different viewpoints, registration performance can be enhanced if these connections be harnessed properly. Therefore, this article models the registration problem as multi-task optimization, and proposes a novel bi-channel knowledge sharing mechanism for effective and efficient problem solving. The modeling of multi-view point cloud registration as multi-task optimization are twofold. By simultaneously considering the local accuracy of two point clouds as well as the global consistency posed by all the point clouds involved, a fitness function with an adaptive threshold is derived. Also a framework of the co-evolutionary search process is defined for the concurrent optimization of multiple fitness functions belonging to related tasks. To enhance solution quality and convergence speed, the proposed bi-channel knowledge sharing mechanism plays its role. The intra-task knowledge sharing introduces aiding tasks that are much simpler to solve, and useful information is shared across aiding tasks and the original tasks, accelerating the search process. The inter-task knowledge sharing explores commonalities buried among the original tasks, aiming to prevent tasks from getting stuck to local optima. Comprehensive experiments conducted on model object as well as scene point clouds show the efficacy of the proposed method.

The searching ability of the population-based search algorithms strongly relies on the coordinate system on which they are implemented. However, the widely used coordinate systems in the existing multifactorial optimization (MFO) algorithms are still fixed and might not be suitable for various function landscapes with differential modalities, rotations, and dimensions; thus, the intertask knowledge transfer might not be efficient. Therefore, this article proposes a novel intertask knowledge transfer strategy for MFOs implemented upon an active coordinate system that is established on a common subspace of two search spaces. The proper coordinate system might identify some common modality in a proper subspace to some extent. In this article, to seek the intermediate subspace, we innovatively introduce the geodesic flow that starts from a subspace, reaching another subspace in unit time. A low-dimension intermediate subspace is drawn from a uniform distribution defined on the geodesic flow, and the corresponding coordinate system is given. The intertask trial generation method is applied to the individuals by first projecting them on the low-dimension subspace, which reveals the important invariant features of the multiple function landscapes. Since intermediate subspace is generated from the major eigenvectors of tasks' spaces, this model turns out to be intrinsically regularized by neglecting the minor and small eigenvalues. Therefore, the transfer strategy can alleviate the influence of noise led by redundant dimensions. The proposed method exhibits promising performance in the experiments.

Deep learning-based image inpainting has been widely studied, leading to great success. However, many methods adopt convolution and normalization operations, which will bring up some issues to affect the performance. The vanilla normalization cannot distinguish the pixels in corrupted regions from the other valid pixels, resulting in the mean and variance shifts. In addition, the limited receptive field of convolution makes it unable to capture long-range valid information directly. In order to tackle these challenges, we propose a novel deep generative model for image inpainting with two key modules, namely, the channel and spatially adaptive batch normalization (CSA-BN) module, and the selective latent-space-mapping-based contextual attention (SLSM-CA) layer. We replace the vanilla normalization with the CSA-BN module. By channel and spatially adaptive denormalization, the CSA-BN module can mitigate the spatial mean and variance shifts in each channel in a targeted way. In addition, we also integrate the SLSM-CA layer into our model to capture the long-range correlations explicitly. By introducing dual-branch attention and a feature selection module, the SLSM-CA layer can selectively utilize the multi-scale background information to improve prediction quality. What’s more, it introduces the latent spaces to achieve the low-rank approximations of attention matrices and to reduce computational costs. Extensive quantitative and qualitative evaluations demonstrate the superiority of the proposed method compared with state-of-the-art methods.

Network embedding has been widely used to solve the network analytics problem. Existing methods mainly focus on networks with single-layered homogeneous or heterogeneous networks. However, many real-world complex systems can be naturally represented by multilayer networks, which is another term of heterogeneous networks with multiple edge/relation types. The problem of how to capture and utilize rich interaction information of multi-type relations causes a major challenge of multilayer network embedding. To address this problem, we propose a fast and scalable multilayer network embedding model, called HMNE, to efficiently preserve and learn information of multi-type relations into a unified embedding space. We develop a heuristic 3D interactive walk technique dedicated for multilayer networks, which can leverage rich interactions among distinct layers and effectively capture important information contained in the layered structure. We evaluate our proposed model HMNE on two downstream analytic applications: node classification and link prediction. Experimental results on seven social and biological multilayer network datasets demonstrate that the proposed model outperforms existing competitive baselines with reduced time and memory occupations.

As a unified framework for graph neural networks, message passing-based neural network (MPNN) has attracted a lot of research interest and has been shown successfully in a number of domains in recent years. However, because of over-smoothing and vanishing gradients, deep MPNNs are still difficult to train. To alleviate these issues, we first introduce a deep hierarchical layer aggregation (DHLA) strategy, which utilizes a block-based layer aggregation to aggregate representations from different layers and transfers the output of the previous block to the subsequent block, so that deeper MPNNs can be easily trained. Additionally, to stabilize the training process, we also develop a novel normalization strategy, neighbor normalization (NeighborNorm), which normalizes the neighbor of each node to further address the training issue in deep MPNNs. Our analysis reveals that NeighborNorm can smooth the gradient of the loss function, i.e., adding NeighborNorm makes the optimization landscape much easier to navigate. Experimental results on two typical graph pattern-recognition tasks, including node classification and graph classification, demonstrate the necessity and effectiveness of the proposed strategies for graph message-passing neural networks.

Existing multifactorial particle swarm optimization algorithms treat all particles equally with a consistent inter-task exemplar selection and generation strategy. This may lead to poor performance when the algorithm searches partial optimal areas belonging to different tasks at the later stage. In pedagogy, teachers teach students in different levels distinctively under their cognitive and learning abilities. Inspired by this idea, in this work, we devise a novel level-based inter-task learning strategy upon a dynamic local topology of inter-task particles. The proposed method separates particles into several levels and assigns particles to different levels with distinct inter-task learning methods. Specifically, we propose a level-based inter-task learning strategy to transfer sharing information among the cross-task neighborhood. By assigning the particles with diverse search preferences, the algorithm is able to explore the search space by using the cross-task knowledge, while reserving an ability to refine the search area. In addition, to address the issue of inter-task neighbor selection, we reform dynamically the local topology structure across the inter-task particles by methodical sampling, evaluating and selecting processes. Experimental results on the benchmark problems demonstrate that the proposed method enables the efficient cross-domain information transfer via the level-based inter-task learning.

The bottleneck of computation burden limits the widespread use of the 2nd order optimization algorithms for training deep neural networks. In this paper, we present a computationally efficient approximation for natural gradient descent, named Swift Kronecker-Factored Approximate Curvature (SKFAC), which combines Kronecker factorization and a fast low-rank matrix inversion technique. Our research aims at both fully connected and convolutional layers. For the fully connected layers, by utilizing the low-rank property of Kronecker factors of Fisher information matrix, our method only requires inverting a small matrix to approximate the curvature with desirable accuracy. For convolutional layers, we propose a way with two strategies to save computational efforts without affecting the empirical performance by reducing across the spatial dimension or receptive fields of feature maps. Specifically, we propose two effective dimension reduction methods for this purpose: Spatial Subsampling and Reduce Sum. Experimental results of training several deep neural networks on Cifar-10 and ImageNet-1k datasets demonstrate that SKFAC can capture the main curvature and yield comparative performance to K-FAC. The proposed method bridges the wall-clock time gap between the 1st and 2nd order algorithms.