Accurate gland segmentation in colorectal cancer histopathology is crucial, but the scarcity of pixel-level annotations limits robust model development. This study aims to develop a highly accurate gland segmentation method that leverages weakly labeled data, specifically image-level labels, to overcome the need for extensive pixel-level annotations. We propose a novel three-stage framework that uniquely combines self-supervised fine-tuning of the DINOv2 vision transformer, attention-based pseudo-label generation, and a boundary-aware loss function. Initially, an off-the-shelf DINOv2 encoder is fine-tuned on a large unlabeled dataset of histopathology images. This fine-tuned encoder is then integrated into a classification network equipped with an attention mechanism, which is trained using image-level labels to generate initial pseudo-labels via attention maps. These maps are refined through blending, thresholding, and Conditional Random Field (CRF) post-processing. Finally, a segmentation network, employing the same fine-tuned encoder and a lightweight decoder, is trained using these refined pseudo-labels and a boundary-aware loss. Ablation studies demonstrated the significant benefit of the fine-tuned encoder and the comprehensive post-processing steps for pseudo-label generation. Further studies confirmed the effectiveness of the boundary-aware loss in improving segmentation accuracy. Our method achieved superior performance on the GlaS dataset compared to several state-of-the-art methods, including both fully supervised and weakly supervised approaches, demonstrating higher F1-score, Object Dice, and lower Object Hausdorff distance. This approach effectively addresses the challenge of limited pixel-level annotations by utilizing more readily available image-level data, offering a promising solution for improved colorectal cancer diagnosis. The proposed framework shows potential for generalization to other histopathology image analysis tasks.
Background Pathological images, particularly whole slide images (WSIs), serve as a cornerstone for clinical studies and diagnostics. However, the presence of permanent ink markings used by pathologists poses a significant challenge for the computational analysis of these images. Existing methods for ink removal have limitations, such as being computationally expensive, struggling with complex ink markings and staining differences, or relying on extensive annotated data. Purpose To address these limitations, we propose a novel two-stage method for robust and efficient ink removal from pathological images. Our method combines pixel clustering and deep learning classification to capture the structural features of ink markings, making it more generalizable and less dependent on annotated data. Method The first stage involves pixel clustering using a novel distance metric that incorporates spatial information and multiple color spaces. This separates ink markings from tissue regions based on structural characteristics. In the second stage, a pretrained ResNet50 model classifies the clustered pixels as ink markings or tissue by leveraging the structural features. This two-stage approach is designed to be robust to staining variations and cancer cell differences while enabling real-time processing. Results Evaluated on a large cancer dataset, our method outperformed the existing tools and methods, achieving an Obj-Dice of 0.92, Obj-HD of 84.2 mu m, and Obj-HD95 of 46.5 mu m. Furthermore, it generalized well to two other clinical datasets with multiple cancer types, achieving a success rate of 98.8% on both datasets. Implications Our method's ability to capture structural features of ink markings makes it robust and generalizable, suitable for widespread clinical application. The structural approach simplifies the classification task, reducing labeling effort and enabling real-time processing. These promising results suggest the potential of our method for improving computational analysis of pathological images in clinical settings.
Background and objective: Automated prediction of microsatellite instability from colorectal histopathology slides is clinically valuable, but stain variation can reduce model robustness across institutions and scanners. We aimed to develop a stain-robust normalization method that works directly in the feature space of pathology foundation models rather than at the image level.Methods: We trained top-k sparse autoencoders on pathology foundation model embeddings and used a controlled synthetic stain-variation probe set to identify latent features that were highly sensitive to stain changes. During inference, these stain-sensitive features were suppressed before slide-level classification. The method was evaluated for microsatellite instability prediction on an in-house cohort of 2,403 colorectal cancer slides and an external public cohort of 582 slides, and was compared with image-level normalization and augmentation baselines across multiple slide aggregation models.Results: The proposed method improved robustness to stain variation and reduced stain-related false positive predictions. On the in-house cohort, the area under the receiver operating characteristic curve increased from 0.885 without normalization to 0.918 with feature-space normalization. On the external cohort, the same metric increased from 0.871 to 0.905. The method also produced consistent gains across bag-of-features, attention-based multiple instance learning, and transformer-based multiple instance learning slide classifiers.Conclusions: Feature-space stain normalization using sparse autoencoders improved colorectal microsatellite instability prediction across heterogeneous cohorts and outperformed conventional image-level normalization strategies. The results support feature-space normalization as a practical approach for improving the robustness of pathology foundation models in computational pathology.
LiDAR-camera fusion has significantly advanced 3D object detection by enriching sparse LiDAR point clouds with dense pseudo point clouds derived from images. However, pseudo point clouds often introduce noise, which can compromise localization accuracy if not properly handled. To address this problem, we propose a simple and effective LiDAR-pseudo point cloud refinement framework (LPRFusion) that improves detection accuracy through a two-step asymmetric cascade RoI refinement strategy. In the first step, region proposals are refined using only LiDAR RoI features to ensure precise and stable localization. In the second step, pseudo point clouds are selectively incorporated to further refine structurally ambiguous or distant proposals. To better exploit pseudo points, we introduce a lightweight pseudo point geometric feature aggregator (PGFA) to encode local context via 2D-guided neighborhood features. A dual point gated RoI fusion (DP-GRF) module is then employed in the second step to adaptively fuse LiDAR and pseudo RoI features, enabling robust and effective multimodal integration. Experiments on the KITTI and nuScenes datasets demonstrate that our LPRFusion achieves mAP improvements of 1.14% and 0.80%, respectively, validating the effectiveness of our design. Our code is available at: https: //github.com/Bradgers/LPRFusion.git.
Vision-Language Models (VLMs) have demonstrated remarkable capabilities in visual grounding, yet their performance degrades severely under adverse weather conditions such as fog, rain, and low light. To address this critical robustness gap, we introduce GRIT (Grounding through Reasoning and Iterative Thinking), a novel, training-free framework that recasts a pre-trained VLM as an autonomous agent. GRIT empowers the VLM to engage in an iterative reasoning loop, where it strategically utilizes a visual toolbox to progressively mitigate weather-induced degradations and refine its spatial focus. Guided by a meticulously engineered prompt, the agent can perform actions like dehazing, deraining, and targeted zooming to actively clarify the visual scene. Extensive experiments on challenging datasets demonstrate that GRIT significantly enhances the grounding accuracy and robustness of state-of-the-art VLMs, establishing a new paradigm for augmenting multi-modal models by transforming them from passive observers into active problem-solvers.
Event cameras provide high temporal resolution, low latency, and high dynamic range, making them suitable for semantic segmentation in challenging visual conditions. However, the lack of densely annotated event data limits supervised learning and motivates unsupervised domain adaptation (UDA) from labeled images to unlabeled event streams. Existing cross-modal UDA methods are often affected by weak target-domain feature discrimination and unreliable pseudo-label supervision, especially around ambiguous semantic boundaries. To address these issues, we propose a feature–label co-refinement framework for event semantic segmentation under cross-modal UDA. Built upon a dual-path target adaptation pipeline, the proposed method refines target representations and pseudo-label supervision from two complementary perspectives. Semantic Prototype Alignment Learning (SPAL) enhances class-wise compactness and separability in the target embedding space with semantic prototypes. Boundary-guided Pseudo-label Refinement (BPR) improves pseudo-label reliability by combining prediction confidence with boundary-aware soft weighting. Experiments on DDD17 and DSEC-Semantic show that the proposed method improves the reconstruction-based adaptation baseline, achieves competitive performance on DDD17, and brings stable gains on several challenging categories of DSEC-Semantic.
The sparsity and quality of point clouds significantly constrain the development of LiDAR-based 3D object de tectors. Previous approaches supplemented point clouds through depth completion or upsampling. However, the former suffers from inconsistencies caused by differences in multimodal data, resulting in uneven point cloud quality. Meanwhile, previous upsampling methods convert point clouds into range images which results in a loss of point accuracy. In this paper, we present DAPU, a novel real-time point cloud upsampling method designed to address these challenges. This method consists of three key components: (1) the Ground Points Recognizer (GPR), which analyzes the height difference distribution between coplanar and non-coplanar points within patches to identify ground points, establishes a sparse-to-dense index matrix for fast large-scale point cloud queries; (2) the Distribution-Aware Patch KNN (DAPKNN), which dynamically adjusts the sampling radius threshold based on distribution to reduce computation and ensure sufficient neighbor sampling for distant points; (3) the Neighbors Upsampling, which linearly upsamples between each pair of neighbors to preserve all point features. KITTI experiments show gains of up to +1.2% AP 3D and +1.4% AP BEV. Additional evaluations on mini-nuScenes and Waymo further demonstrate consistent improvements across Vehicle, Pedestrian, and Cyclist detection, confirming DAPU's robustness under diverse LiDAR settings and real-time suitability.
Extracting valuable visual cues for downstream vision tasks poses a particular challenge under unknown degradations. A straightforward solution is to preprocess images using image restoration methods, but their high computational complexity renders them unsuitable for real-time tasks. Recent efforts have aimed to enhance image representations optimized by downstream task losses. However, these approaches are confined to known single degradation scenarios, constraining their practical applicability in unpredictable environments. Therefore, we propose a Chain-of-thought Prompted Adaptive Enhancer, CPA-Enhancer, for enhancing important features crucial for downstream vision tasks under unknown degradations. Specifically, CPA-Enhancer progressively adapts its enhancement strategies under the step-by-step guidance of CoT prompts that encode degradation-related information. Overall, CPA-Enhancer is a plug-and-play lightweight enhancement model that can be integrated into any vision pipeline and trained with task-related losses, without any prior knowledge of the degradation type. Extensive experiments demonstrate that CPA-Enhancer significantly improves performance across various vision tasks under unknown degradations. The codes is available at https://github.com/zyw-stu/CPA-Enhancer.
In recent years, multimodal 3D object detectors have attracted substantial attention in autonomous driving systems for their remarkable detection capabilities. Existing approaches mainly transform LiDAR and camera modalities into a unified BEV (Bird’s Eye View) plane for fusion. However, these methods primarily adopt mono-scale BEV feature interaction, failing to fully exploit the multi-scale spatial details offered by different modalities. This work proposes a novel multi-sensor fusion framework to enhance the accuracy of 3D object detection in diverse conditions. Specifically, we develop a convolution-based generation module to obtain dual-scale BEV features for two modalities and a cross-modal fusion mechanism is employed to effectively leverage these features. Besides, Temporal information is incorporated as a prior to guide the view transformation process to gain more precise camera representations. Lastly, to mitigate the issue of performance degradation at night, a plug-in image enhancement module is used to dynamically optimize image brightness. Experiments conducted on the widely utilized Nuscenes dataset demonstrate the efficacy of our proposed framework. Evaluation results show that our method achieves a 1.4
Existing LiDAR-Camera fusion methods have achieved strong results in 3D object detection. To address the sparsity of point clouds, previous approaches typically construct spatial pseudo point clouds via depth completion as auxiliary input and adopts a proposal-refinement framework to generate detection results. However, introducing pseudo points inevitably brings noise, potentially resulting in inaccurate predictions. Considering the differing roles and reliability levels of each modality, we propose LDRFusion, a novel Lidar-dominant two-stage refinement framework for multi-sensor fusion. The first stage soley relies on LiDAR to produce accurately localized proposals, followed by a second stage where pseudo point clouds are incorporated to detect challenging instances. The instance-level results from both stages are subsequently merged. To further enhance the representation of local structures in pseudo point clouds, we present a hierarchical pseudo point residual encoding module, which encodes neighborhood sets using both feature and positional residuals. Experiments on the KITTI dataset demonstrate that our framework consistently achieves strong performance across multiple categories and difficulty levels.
Some knowledge bases (KBs) extracted from Wikipedia articles can achieve very high average precision values (over 95% in DBpedia). However, subtle mistakes including inconsistencies, outliers, and erroneous relations are usually ignored in the construction of KBs by extraction rules. Automatic detection and correction of these subtle errors is important for improving the quality of KBs. In this paper, an inductive logic programming with exceptional information (EILP) is proposed to automatically detect errors in large knowledge bases (KBs). EILP leverages the exceptional information problems that are ignored in conventional rule-learning algorithms such as inductive logic programming (ILP). Furthermore, an inductive logical correction method with exceptional features (EILC) is proposed to automatically correct these mistakes by learning a set of correction rules with exceptional features, in which respective metrics are provided to validate the revised triples. The experimental results demonstrate the effectiveness of EILP and EILC in detecting and repairing large knowledge bases, respectively.
All-in-one image restoration aims to recover various degraded images using a unified model. To adaptively reconstruct high-quality images, recent prevalent CNN and Transformer based models incorporate learnable prompts to dynamically acquire degradation-specific knowledge for different degraded images, achieving state-of-the-art restoration performance. However, existing methods exhibit limitations, including high computational burden and inadequate modeling of long-range dependencies. To address these issues, we propose a reasoning and action prompt-driven Mamba-based image restoration model, namely RamIR. Specifically, RamIR employs the Mamba block for long-range dependencies modeling with linear computational complexity relative to the feature map size. Inspired by Chain-of-Thought (CoT) prompting, we integrate Reasoning and Action (ReAct) prompts within the Mamba block. Hence, we utilize the capability of pretrained vision language (PVL) models to generate textual reasoning prompts describing the type and severity of degradations. Simultaneously, another output from PVL acts as action prompt representing the clean image caption. These prompts, employed in a CoT manner, enhance the network’s sensitivity to degradation and elicit targeted recovery actions tailored to different reasoning prompts. Additionally, we explore the seamless interaction between Mamba blocks and prompts, introducing a novel prompt-driven module (PDM) to facilitate prompt utilization. Extensive experimental results demonstrate the superior performance of RamIR, highlighting its advantages in terms of input scaling efficiency over existing benchmark models for all-in-one image restoration.
Recently, several studies have explored the use of high-definition (HD) maps to enhance 3D object detection in LiDAR point clouds. However, existing LiDAR-map fusion methods typically treat HD maps as additional input features or perform feature-level fusion through independent branches, leaving a wealth of valuable prior information within the maps underutilized. To address these limitations, we introduce a GT-Paste strategy driven by grid positions during data augmentation, which effectively enhances the ability of our modal to recognize valid objects under realistic spatial distributions. In addition, we propose a novel multi-stage fusion paradigm that performs map and point cloud fusion at both the data input stage and the feature fusion stage, fully leveraging the spatial and semantic priors inherent in HD maps. Specifically, our modal GSMNet extracts the height and point quantity difference between neighbor pillars in the Bird’s Eye View (BEV), capturing the spatial distribution priors of point clouds in input-stage. Simultaneously, GSMNet incorporates semantic features from the HD map, enabling the model to better perceive scene structures and road semantics. Furthermore, GSMNet leverages a hybrid attention mechanism, combining sparse channel attention and spatial attention to refine high-level feature fusion, enhancing the interaction and expressiveness of informative features. The experimental results validate the effectiveness of GSMNet and achieve considerable detection performance on the NuScenes dataset.
Recent advances in foundation models have opened up new possibilities for enhancing 3D perception. In particular, DepthAnything offers dense and reliable geometric priors from monocular RGB images, which can complement sparse LiDAR data in autonomous driving scenarios. However, such priors remain underutilized in LiDAR-based 3D object detection. In this paper, we address the limited expressiveness of raw LiDAR point features, especially the weak discriminative capability of the reflectance attribute, by introducing depth priors predicted by DepthAnything. These priors are fused with the original LiDAR attributes to enrich each point's representation. To leverage the enhanced point features, we propose a point-wise feature extraction module. Then, a Dual-Path RoI feature extraction framework is employed, comprising a voxel-based branch for global semantic context and a point-based branch for fine-grained structural details. To effectively integrate the complementary RoI features, we introduce a bidirectional gated RoI feature fusion module that balances global and local cues. Extensive experiments on the KITTI benchmark show that our method consistently improves detection accuracy, demonstrating the value of incorporating visual foundation model priors into LiDAR-based 3D object detection.
Pulmonary adenocarcinoma is the primary cause of cancer-related death worldwide and pathological diagnosis is the "golden standard" based on the regional distribution of cells. Thus, regional cell segmentation is a key step while it is challenging due to the following reasons: 1) It is hard for pure semantic and instance segmentation methods to obtain a high-quality regional cell segmentation result; 2) Since the spatial appearances of pulmonary cells are very similar which even confuse pathologists, annotation errors are usually inevitable. Considering these challenges, we propose a two-stage 3D adaptive joint training framework (TAJ-Net) to segment-then-classify cells with extra spectral information as the supplementary information of spatial information. Firstly, we propose to leverage a few-shot method with limited data for cell mask acquisition to avoid the disturbance of cluttered backgrounds. Secondly, we introduce an adaptive joint training strategy to remove noisy samples through two 3D networks and one 1D network for cell type classification rather than segmentation. Subsequently, we propose a patch mapping method to map classification results to the original images to obtain regional segmentation results. In order to verify the effectiveness of TAJ-Net, we build two 3D hyperspectral datasets, i.e., pulmonary adenocarcinoma (3,660 images) and thyroid carcinoma (4623 images) with 40 bands. The first dataset will be released for further research. Experiments show that TAJ-Net achieves much better performance in clustered cell segmentation, and it can regionally segment different kinds of cells with high overlap and blurred edges, which is a difficult task for the state-of-the-art methods. Compared to 2D models, the hyperspectral image-based 3D model reports a significant improvement of up to 11.5% in terms of the Dice similarity coefficient in the pulmonary adenocarcinoma dataset.
Dynamic networks, such as social networks and recommendation systems, are widespread in the real world. Graph representation learning has emerged as an effective strategy for analyzing such networks. However, the problem lies in the fact that many current techniques treat dynamic networks as static or discrete structures, while continuous-time approaches often lack the capabilities to effectively handle networks with low node repetition behavior. To alleviate these problems, we first treat dynamic networks as continuous-time interactions, and then propose a novel method for the transductive interaction prediction task. Our approach incorporates two key aspects: evolutionary expectation learning and temporal dynamic learning. The former imparts guidance to the network’s evolution and endows node embedding with a more profound wealth of information, while the latter provides detailed insight into the intricate process of network evolution. Together, these two components provide a comprehensive understanding of network behavior and can efficiently handle dynamic networks with low node repetition behavior. Specifically, we utilize an asynchronous training process, starting with the training of a multi-event embedding module that captures information about the evolutionary expectation of dynamic networks. Based on this foundation, we train a temporal multi-event embedding module to map the network’s dynamic evolution onto node embedding representations. Furthermore, we design a temporal single-event module that effectively captures implicit long-term interaction dependencies of nodes. To evaluate the effectiveness of our proposed method, we evaluate its performance on four datasets and demonstrate its superior performance compared to the baselines.
Representation learning over graph networks has recently gained popularity, with many models showing promising results. However, several challenges remain: (1) most methods are designed for static or discrete-time dynamic graphs; (2) existing continuous-time dynamic graph algorithms focus on a single evolving perspective; and (3) many continuous-time dynamic graph approaches necessitate numerous temporal neighbors to capture long-term dependencies. In response, this paper introduces a Multi-Perspective Feedback-Attention Coupling (MPFA) model. MPFA incorporates information from both evolving and original perspectives to effectively learn the complex dynamics of dynamic graph evolution processes. The evolving perspective considers the current state of historical interaction events of nodes and uses a temporal attention module to aggregate current state information. This perspective also makes it possible to capture long-term dependencies of nodes using a small number of temporal neighbors. Meanwhile, the original perspective utilizes a feedback attention module with growth characteristic coefficients to aggregate the original state information of node interactions. Experimental results on one dataset organized by ourselves and seven public datasets validate the effectiveness and competitiveness of our proposed model.
Current LiDAR-only 3D detection methods are limited by the sparsity of point clouds. The previous method used pseudo points generated by depth completion to supplement the LiDAR point cloud, but the pseudo points sampling process was complex, and the distribution of pseudo points was uneven. Meanwhile, due to the imprecision of depth completion, the pseudo points suffer from noise and local structural ambiguity, which limit the further improvement of detection accuracy. This paper presents SQDNet, a novel framework designed to address these challenges. SQDNet incorporates two key components: the SQD, which achieves sparse-to-dense matching via grid position indices, allowing for rapid sampling of large-scale pseudo points on the dense depth map directly, thus streamlining the data preprocessing pipeline. And use the density of LiDAR points within these grids to alleviate the uneven distribution and noise problems of pseudo points. Meanwhile, the sparse 3D Backbone is designed to capture long-distance dependencies, thereby improving voxel feature extraction and mitigating local structural blur in pseudo points. The experimental results validate the effectiveness of SQD and achieve considerable detection performance for difficult-to-detect instances on the KITTI test.
Modeling temporal networks for dynamic link prediction of new nodes has many real-world applications, such as providing relevant item recommendations to new customers in recommender systems and suggesting appropriate posts to new users on social platforms. Unlike old nodes, new nodes have few historical links, which poses a challenge to the dynamic link prediction task. Most existing dynamic models treat all nodes equally and are not specialized for new nodes, resulting in suboptimal performance. In this paper, we consider the dynamic link prediction of new nodes as a few-shot problem and propose a novel model based on the meta-learning principle to effectively mitigate this problem. Specifically, we devise a meta-learning-specific temporal graph neural network module for node-level dynamic link prediction, featuring an encoder with node-wise span memory and a predictor. To overcome the fewshot problem, we design an adaptive meta-learner with spanwise adaptation and node-wise adaptation to extract two types of implicit information behind this problem. The acquired implicit information can serve as model initialization and facilitate rapid adaptation to new nodes through a fine-tuning process on just a few links. Experiments on three publicly available datasets demonstrate the superior performance of our model compared to existing state-of-the-art methods.