Multimodal remote sensing imagery classification (MRSIC) aims to synergistically leverage complementary information from heterogeneous data sources, enabling precise land-cover classification. Existing MRSIC approaches predominantly rely on abundant annotated samples, facing critical performance degradation under data-scarce scenarios that are particularly exacerbated by the inherent complexity of heterogeneous multi-modal data. Furthermore, effectively extracting spatial-spectral information of multimodal data and fusing the cross-modal heterogeneous features persists as a significant challenge. To address these obstacles, we propose a pioneering few-shot MRSIC network, Two-timer-KAN, which integrates modality-specific feature extraction for spectral- and spatial-dominant data. Specifically, leveraging the nonlinear power of Kolmogorov-Arnold Networks (KANs), we develop the Dual-Exclusive Fourier KAN (DEF-KAN) encoder, which captures modality-specific global features in the frequency domain, bridging spectral and spatial gaps across various datasets. Following this, a Multivariate-Gaussian-based Cross-KAN (MG-Cross-KAN) is dedicated to enhancing the robustness of cross-modality fusion by capturing modality-shared features in a distribution-based manner. Finally, to further tackle classification ambiguity under limited annotated samples, we present a visual-textual bidirectional alignment strategy, which leverages textual descriptions as supplementary semantical knowledge to clarify class feature centers. Extensive experiments demonstrate that the proposed Two-timer-KAN achieves superior performance, outperforming the state-of-the-art methods in both accuracy and robustness.
Current spectral reconstruction (SR) methods predominantly employ empirically crafted deep architectures to recover hyperspectral images (HSIs) from low-dimensional observations. However, the limited spectral cues and the high correlation in low-dimensional embeddings make single-modality reconstruction fundamentally underconstrained, hindering the recovery of fine-scale spectral variations. To address this issue, we introduce a multi-modal SR framework that incorporates text-derived linguistic guidance encoding task-specific cues and spectral descriptors, which are efficiently processed without significant computational overhead into the reconstruction pipeline. Based on this idea, we develop the Progressive Text Interaction Reconstruction Network (PTIR). PTIR employs a hierarchical interaction paradigm to inject text-derived semantics, such as task-specific information, at multiple processing stages, effectively mitigating unimodal limitations. In particular, the dedicated Focus-Weighted Attention (FWA) synergizes with Kolmogorov-Arnold Networks (KAN) to dynamically modulate the focus scope of attention heads, refining the granularity of interactions between features at each level and accomplishing the modulation of features. Meanwhile, a novel diagonal Core-Correlation perception module (DCCP) is proposed to preserve the essential correlations while discarding redundancy via diagonal compression, enhancing fine-grained pattern recognition and yielding a compact, semantically enriched latent space. In addition, this kind of design significantly reduces computational complexity and memory footprint. Extensive experiments across multiple benchmarks demonstrate that the proposed PTIR consistently surpasses existing state-of-the-art methods.
Existing deep learning-based multimodal remote sensing imagery (MRSI) classification (MRSIC) models rely on fixed-category paradigms and struggle to adapt to novel categories, primarily due to the catastrophic forgetting of vintage knowledge, which substantially impedes their practical applications. To overcome this challenge, we propose a novel dual-flow prompt-expert Mamba (DF-PEM) network for MRSI incremental classification (IC), which incorporates dedicated parallel branches for spectral and spatial feature extraction, synergized with a cross-modal fusion module to integrate modality-specific representations. The primary objective of DF-PEM is to capture discriminative MRSI features while preserving knowledge of previously seen categories during the incremental learning process. Specifically, DF-PEM integrates the prompt-driven assistant-expert (PAE) mechanism within the Mamba architecture to dynamically activate category-specific prompt experts per input, enabling seamless adaptation to emerging categories while mitigating knowledge erosion of prior classes. Furthermore, a spherical Gaussian sift (SGS) strategy is developed in the initial stage, which models the complex feature distributions of historical categories and neutralizes noise and outlier effects, yielding robust representative prototypes. Finally, a prototype-aware regularization classifier (PRC) that adaptively balances prediction bias between incremental and old categories by learning regularization coefficients, which strategically penalize parameter updates that deviate the model from historical knowledge, further enhancing knowledge retention and generalization. Extensive experiments on benchmark datasets demonstrate that the proposed DF-PEM significantly outperforms existing methods, validating its effectiveness for MRSI classification tasks.
Spectral reconstruction (SR) aims to recover high-quality hyperspectral images (HSIs) from more readily available RGB or multispectral images (MSIs). While supervised SR has shown promising results, it is hindered by the difficulty of collecting abundant, well-registered RGB-HSI or MSI-HSI pairs. Semi-supervised SR (Semi-SR) offers a more practical solution by exploiting plentiful RGBs/MSIs together with limited HSIs. However, existing Semi-SR approaches still suffer from cross-domain discrepancies, cross-modality inconsistency, and unreliable pseudo-labels. To tackle these challenges, we propose a Manifold-aware Teacher-Student Semi-SR (MTSSR) framework, which seamlessly integrates labeled and unlabeled domains through a teacher-student paradigm and memory-efficient consistency learning. At its core, a Flexible Cross-attention Spectral Reconstruction (FCSR) network extracts scene-related spatial cues via customized self-attention and models scene-agnostic priors through dynamic quantization, thereby enhancing spectral fidelity. Furthermore, a manifold-aware dimensionality analysis derives a latent space that jointly captures spatial and spectral structures across modalities. This enables a manifold-aware alignment loss to enforce cross-modality consistency and a manifold-aware contrastive loss to progressively refine pseudo-label reliability. In addition, we develop a Threshold-adjusted Memory Bank Update (TMBU) strategy, which generates reliable negative samples by storing network-driven representations instead of memory-consuming HSIs, significantly reducing memory consumption. Extensive experiments on three visual and two remote sensing benchmarks demonstrate that MTSSR consistently outperforms state-of-the-art SR methods, achieving robust and memory-efficient spectral reconstruction.
We present DeSOPE, a large-scale dataset designed for Deformed Six-DoF Object Pose Estimation. Most existing 6D object pose approaches assume rigid or articulated objects, leaving deformed daily objects largely unexplored. This gap limits the realism and robustness of current pose estimation methods, which often fail when objects deviate from their canonical shapes due to wear, collision, or deformation. To address this issue, we present DeSOPE, a large-scale real-world dataset specifically designed for deformed object pose estimation. DeSOPE contains two major components: (1) a collection of high-fidelity 3D scans of 26 common object categories, each captured in one canonical and three deformed states using a non-rigid alignment framework; and (2) a real-scene RGB-D dataset comprising 133K frames and 665K pose annotations across 104 deformed instances, recorded in both static and dynamic scenarios. The varying degrees of deformation introduce substantial geometric and textural changes, presenting new challenges for existing methods. We benchmark several state-of-the-art algorithms on DeSOPE and demonstrate significant performance degradation as deformation increases, highlighting the limitations of current pose estimators. As the first large-scale dataset designed for systematic study of deformed object pose estimation, DeSOPE lays the groundwork for developing 6D pose estimators capable of handling real-world deformation and variability.
Cross-domain few-shot learning hyperspectral image classification (CDFSL HSIC) presents significant challenges, primarily due to spectral variability across domains and the scarcity of labeled target samples. To tackle these issues, we propose a graph relation aggregated spectral perception network (GRASP-Net), which progressively models spectral and structural information from the sample level to the category level. First, an enhanced spectral perception (ESP) module is developed to refine sample-level features by enhancing discriminative spectral cues while suppressing redundancy. On this basis, a prototype-guided graph construction (PGC) module establishes a sample-prototype bipartite graph, integrating geometric neighborhood relations with nongeometric priors to generate compact and robust category-aware embeddings. Furthermore, a gradient-aware metric function interprets gradients as directional vectors to adaptively refine sample-prototype relations, thereby overcoming the limitations of conventional scalar distance metrics. Extensive experiments on multiple cross-domain benchmarks demonstrate that GRASP-Net achieves superior accuracy and robustness compared with state-of-the-art methods, particularly in scenarios with limited labeled samples and large domain shifts.
Survival analysis is a widely used statistical framework for modeling time-to-event data under censoring. Classical methods, such as the Cox proportional hazards (Cox PH) model, offer a semiparametric approach to estimating the effects of covariates on the hazard function. Despite its importance, survival analysis has been largely unexplored in online settings, particularly within the bandit framework, where decisions must be made sequentially to optimize treatments as new data arrive over time. In this work, we take an initial step toward integrating survival analysis into a purely online learning setting under the Cox PH model, addressing key challenges including staggered entry, delayed feedback, and right censoring. We adapt three canonical bandit algorithms to balance exploration and exploitation, with theoretical guarantees of sublinear regret bounds. Extensive simulations and semi-real experiments using SEER cancer data demonstrate that our approach enables rapid and effective learning of near-optimal treatment policies.
Spectral super-resolution (SSR) aims to reconstruct the hyperspectral images (HSIs) from RGB or multispectral images. Most existing SSR methods operate under the close-world setting, where models are trained and evaluated within the same scene distribution. Although this yields high accuracy, such approaches struggle to generalize when deployed to unseen scenes. To address this gap, we move toward the open-world setting and propose a decomposition-based framework, termed DecSSR, which explicitly enhances generalization across diverse scenes. Then, Frequency-aware Contrastive Learning in Orthogonal Space (FCO) is customized to narrow the amplitude consistency between different spaces while amplifying the phase difference between different scenes, which urges DecSSR to simultaneously maintain interscene priors and share cross-scene spectral correlation. Next, a fractional Fourier transform guided sparse spectral-wise transformer (FRFT-SST) is proposed to reconstruct the null with learnable sparsity constraints and emphasized region distribution. To protect salient characteristics in null, a degradation-aware null-space purification module (DNPM) is tailored to dynamically accommodate and counteract noise perturbations with degradation parameters. We evaluate our method on four benchmarks across both visual and remote sensing domains, showing superior reconstruction accuracy and improved robustness under distribution shifts compared with state-of-the-art baselines.
During clinical trials evaluating a drug's effect on a survival endpoint, intermediate events often occur in addition to the primary event. The treatment can exert its effect on the primary endpoint along multiple pathways through intermediate events. Assumptions for identifying mediation effects, such as sequential ignorability in natural effects or the dismissible components condition in separable effects, fail because intermediate events act as treatment-induced confounding. To understand the effect along each pathway, we consider hypothetical interventions in transitions between event statuses to mimic the treatment mechanism. The hypothetical interventions adjust for effects through intermediate events and marginalize over unobserved treatment-induced confounding, if any. Based on the derived efficient influence functions for the counterfactual cumulative incidences under hypothetical interventions, we construct multiply robust and semiparametrically efficient estimators for pathway treatment effects. Our proposed framework enables the examination of treatment effects through each transition, on each event, and along each path. By analyzing data from the LEADER Trial, we find that liraglutide significantly reduces the risk of cardiovascular and microvascular events. The reduction in all-cause mortality is primarily mediated by its effects on expanded major adverse cardiovascular events.
Recently, domain alignment and metric-based few-shot learning (FSL) have been introduced into hyperspectral image classification (HSIC) to solve the issues of uneven data distribution and scarcity of annotated data faced in practical applications. However, existing cross-domain few-shot methods ignore pivotal frequency priors of the complex field, which contribute to better category discrimination and knowledge transfer. To address this issue, we propose a novel physics-guided time-interactive-frequency network (PTFNet) for cross-domain few-shot HSIC, enabling the extraction of both frequency priors and spatial features (termed "time domain" following Fourier convention) simultaneously through a lightweight time-interactive-frequency module (TiF-Module) as a pioneering effort. Meanwhile, a spectral Fourier-based augmentation module (SFA-Module) is designed to decouple the frequency priors and enhance the diversity of distribution of physical attributes to imitate the domain shift. Then, the physics consistency loss is introduced to regularize the diverse embeddings to approximate the center of each category's physical attributes, guiding the network to excavate more transferable knowledge of source domain (SD). Furthermore, to fully exploit the discriminant time-frequency information and further improve the accuracy of boundary pixels, a set of multiorientation homogeneous prototypes is adopted to represent each class comprehensively, and an intuitive and flexible uncertainty-rectified bidirectional random walk strategy is applied to replace the Euclidean metric for more reliable classification. The experimental results on four public datasets demonstrate the prominent performance of the proposed PTFNet.
Deep learning has significantly advanced oriented object detection in remote sensing imagery (RSI), yet its success heavily hinges on large-scale orientation-aware annotations, which are prohibitively costly to acquire. While Semi-Supervised Oriented Object Detection (SOOD) offers a promising alternative, the dense agglomerations, arbitrary orientations, and extreme scale variations inherent to remote sensing scenes undermine the reliability of the standard teacher-student paradigm. Specifically, these characteristics induce severe model instability through three coupled mechanisms: semantic ambiguity in complex backgrounds leading to classification uncertainty; geometric sensitivity of high-aspect-ratio objects resulting in regression uncertainty; and selection bias from rigid filtering policies causing assignment uncertainty. To systematically address these challenges, we propose a holistic uncertainty-oriented teacher-student framework, termed Min-Uncertainty Teacher (MUT), that orchestrates representation alignment, dynamic selection, and geometric calibration. We first introduce a Category-response Pixel-level Contrastive Learning (CPCL) strategy, which performs dense contrastive regularization on classification-branch category-response maps to mitigate teacher-student semantic inconsistency. Building on these refined representations, a Dual-stream Adaptive Pseudo-label Filtering (DAPF) mechanism is employed to resolve assignment uncertainty by dynamically assessing pseudo-label quality to reduce selection bias. Complementarily, a Shape-aware Adaptive Weighting (SAW) module calibrates the regression objective, thereby tackling regression uncertainty arising from geometric sensitivity. Extensive experiments on the DOTA-v1.5 and DIOR-R benchmarks demonstrate the effectiveness of the proposed approach.
Hyperspectral image classification (HSIC) has witnessed remarkable progress with the rise of deep learning. Nevertheless, its real-world deployment remains substantially constrained by two fundamental challenges: substantial domain discrepancies across heterogeneous sensors and the scarcity of labeled samples in the real scene (target domain). These issues motivate the task of cross-domain zero-shot HSIC (CDZS HSIC), where the objective is to recognize unseen target categories without any labeled supervision. To address this challenge, we develop a Hilbert-Schmidt independence learning guided Kolmogorov-Arnold Network (HSIL-KAN). Specifically, we first construct a Cross-Domain Kolmogorov-Arnold Network (CD-KAN) as the core backbone, enabling expressive spectral-spatial representation learning with compact parameterization. Building upon this backbone, we design a Hilbert-Schmidt Independence Criterion guided Domain Feature Decomposition (H-DFD) module to explicitly factorize latent features into domain-invariant and domain-specific subspaces, thereby substantially enhancing cross-domain transferability. Furthermore, a Prototype-Regularized Equiangular Tight Frame (PR-ETF) classifier is proposed to maintain geometric class separability and alleviate class imbalance in the zero-shot condition. Comprehensive experiments on three benchmark hyperspectral datasets demonstrate that HSIL-KAN consistently achieves state-of-the-art performance while also exhibiting notable parameter efficiency and computational scalability. The code will be available online at https://github.com/jojolee6513.
Large Language Model (LLM) agents commonly condition actions on full action-observation histories, which introduce task-irrelevant information that easily leads to redundant actions and higher inference cost. We propose Progress-Aware Belief Update (PABU), a belief-state framework that compactly represents an agent's state by explicitly modeling task progress and selectively retaining past actions and observations. At each step, the agent predicts its relative progress since the previous round and decides whether the newly encountered interaction should be stored, conditioning future decisions only on the retained subset. Across eight environments in the AgentGym benchmark, and using identical training trajectories, PABU achieves an 81.0
We present DeSOPE, a large-scale dataset for 6DoF deformed objects. Most 6D object pose methods assume rigid or articulated objects, an assumption that fails in practice as objects deviate from their canonical shapes due to wear, impact, or deformation. To model this, we introduce the DeSOPE dataset, which features high-fidelity 3D scans of 26 common object categories, each captured in one canonical state and three deformed configurations, with accurate 3D registration to the canonical mesh. Additionally, it features an RGB-D dataset with 133K frames across diverse scenarios and 665K pose annotations produced via a semi-automatic pipeline. We begin by annotating 2D masks for each instance, then compute initial poses using an object pose method, refine them through an object-level SLAM system, and finally perform manual verification to produce the final annotations. We evaluate several object pose methods and find that performance drops sharply with increasing deformation, suggesting that robust handling of such deformations is critical for practical applications. The project page and dataset are available at https://desope-6d.github.io/}{https://desope-6d.github.io/.
Curriculum learning improves reinforcement learning (RL) efficiency by sequencing tasks from simple to complex. However, many self-paced curriculum methods rely on computationally expensive inner-loop optimizations, limiting their scalability in high-dimensional context spaces. In this paper, we propose Self-Paced Gaussian Curriculum Learning (SPGL), a novel approach that avoids costly numerical procedures by leveraging a closed-form update rule for Gaussian context distributions. SPGL maintains the sample efficiency and adaptability of traditional self-paced methods while substantially reducing computational overhead. We provide theoretical guarantees on convergence and validate our method across several contextual RL benchmarks, including the Point Mass, Lunar Lander, and Ball Catching environments. Experimental results show that SPGL matches or outperforms existing curriculum methods, especially in hidden context scenarios, and achieves more stable context distribution convergence. Our method offers a scalable, principled alternative for curriculum generation in challenging continuous and partially observable domains.
Pre-trained Large Language Model (LLM) exhibits broad capabilities, yet, for specific tasks or domains their attainment of higher accuracy and more reliable reasoning generally depends on post-training through Supervised Fine-Tuning (SFT) or Reinforcement Learning (RL). Although often treated as distinct methodologies, recent theoretical and empirical developments demonstrate that SFT and RL are closely connected. This study presents a comprehensive and unified perspective on LLM post-training with SFT and RL. We first provide an in-depth overview of both techniques, examining their objectives, algorithmic structures, and data requirements. We then systematically analyze their interplay, highlighting frameworks that integrate SFT and RL, hybrid training pipelines, and methods that leverage their complementary strengths. Drawing on a representative set of recent application studies from 2023 to 2025, we identify emerging trends, characterize the rapid shift toward hybrid post-training paradigms, and distill key takeaways that clarify when and why each method is most effective. By synthesizing theoretical insights, practical methodologies, and empirical evidence, this study establishes a coherent understanding of SFT and RL within a unified framework and outlines promising directions for future research in scalable, efficient, and generalizable LLM post-training.
Hyperspectral image reconstruction from RGB inputs provides a cost-effective and portable alternative to conventional HSI acquisition systems, which are often hindered by high costs, bulky hardware, and limited imaging speeds. Although deep learning has significantly advanced this task, existing state-of-the-art (SOTA) approaches tend to adopt either computationally intensive architectures with millions of parameters, which impedes deployment on edge devices, or lightweight designs that overlook critical spectral structures, resulting in suboptimal trade-offs between accuracy and efficiency. To overcome these challenges, we propose a novel lightweight framework, FWHNet, integrating the Co-Evolving Stream Harmonizer (CESH) (with its core Iterative Frequency-domain Focused Refiner (IFFR)), Wavelet-based Sampling (WS), and Dynamic Channel Pilot (DCP) to achieve superior spectral fidelity with minimal computational cost. Comprehensive experiments conducted on the NTIRE2020, CAVE, and Harvard datasets demonstrate that our model achieves competitive or superior reconstruction accuracy against recent SOTA methods across quantitative metrics and visual quality. In comparison to high-accuracy models such as MST++, our method reduces parameters by 90.2%, FLOPs by 84.2%, and inference time by 42.4% on the CAVE dataset, while maintaining or improving reconstruction quality, highlighting its practicality for real-time spectral reconstruction (SR) in mobile and embedded multimedia applications.