Monocular depth estimation degrades in low texture, poor illumination, reflections, and occlusions, while acoustic echoes provide complementary geometric cues but are sparse and misaligned with images. We present EchoFormer, a multi-modal framework that fuses RGB images with echo spectrograms for robust dense depth prediction. EchoFormer addresses cross-modal gaps with (i) a Semantic–Geometric Conditional Decoder (SGCD) that conditions echo decoding on high-resolution visual semantics, and (ii) a Dynamic Uncertainty-aware Gated Fusion (DUGF) module that adaptively balances modalities using pixel-wise confidence. Experiments on Replica and Matterport3D show consistent improvements over echo-only, monocular, and prior fusion methods, yielding sharper boundaries and more coherent structures in degraded scenes.

The integration of Large Language Models (LLMs) into clinical decision support is critically obstructed by their opaque and often unreliable reasoning. In the high-stakes domain of healthcare, correct answers alone are insufficient; clinical practice demands full transparency to ensure patient safety and enable professional accountability. A pervasive and dangerous weakness of current LLMs is their tendency to produce "correct answers through flawed reasoning." This issue is far more than a minor academic flaw; such process errors signal a fundamental lack of robust understanding, making the model prone to broader hallucinations and unpredictable failures when faced with real-world clinical complexity. In this paper, we establish a framework for trustworthy clinical argumentation by adapting the Toulmin model to the diagnostic process. We propose a novel training pipeline: Curriculum Goal-Conditioned Learning (CGCL), designed to progressively train LLM to generate diagnostic arguments that explicitly follow this Toulmin structure. CGCL's progressive three-stage curriculum systematically builds a solid clinical argument: (1) extracting facts and generating differential diagnoses; (2) justifying a core hypothesis while rebutting alternatives; and (3) synthesizing the analysis into a final, qualified conclusion. We validate CGCL using T-Eval, a quantitative framework measuring the integrity of the diagnosis reasoning. Experiments show that our method achieves diagnostic accuracy and reasoning quality comparable to resource-intensive Reinforcement Learning (RL) methods, while offering a more stable and efficient training pipeline.

Depth estimation has been widely applied in the field of computer vision, primarily using unsupervised deep neural networks, which often rely on deeper neural networks. However, the addition of layers can result in slower convergence and suboptimal performance. To overcome these issues, we introduce a novel architecture employing model distillation, wherein a teacher network enhances the learning process of a preceding student network. To improve network speed, we integrate an ordinal module in the decoder of the teacher network for weight normalization. This module can classify weights and filter out those with the lowest information content. After the weight classification is completed, as the category value increases, the necessity of useful information decreases accordingly. Furthermore, we incorporate a residual stratification module, which adapts 2D image feature extraction methods to 3D depth, facilitating finer, multi-scale feature representation, to expand the receptive field size at each layer of the network, thereby enhancing the accuracy and robustness of depth estimation. Experimental results using the publicly available KITTI dataset demonstrate that the proposed method accelerates network training compared to the benchmark algorithm, reducing the relative squared error by 2.3% and the root-mean-square error by 3.3%, thus validating the effectiveness of our approach.

Timely identification of patients who meet clinical trial eligibility criteria is a persistent bottleneck in trial recruitment because the criteria are written in flexible natural language, while hospital EHRs are stored in structured schemas. To bridge this gap, we propose EC2Seq2Sql, an end-to-end, two-stage framework that automatically converts narrative eligibility criteria into executable SQL queries for EHR-based patient screening. In the first stage, a BART-based semantic parser transforms free-text trial criteria into lightweight structured pattern sequences defined over seven common clinical domains. In the second stage, an LLM-based agent, guided by system- and human-designed prompts, grounds these structured patterns to the target database schema and generates syntactically valid and logically coherent SQL statements. We evaluated the framework on the ClinicalTrials.gov eligibility-criteria dataset and further validated it on a de-identified real-world hepatocellular carcinoma EHR cohort from Zhongshan Hospital, Fudan University. The BART parser outperformed representative Seq2Seq baselines, achieving ROUGE_L 0.8067 and BLEU 0.8427, while the SQL generation stage reached an exact-match accuracy of 0.84 and an execution accuracy of 0.91 after SQL normalization. On the real-world cohort, the generated queries achieved a clinical match accuracy of 0.88 after expert review, indicating that the proposed pipeline can retrieve trial-eligible patients from operational EHR data. These results suggest that EC2Seq2Sql can substantially reduce manual screening effort and provide a reproducible path from narrative criteria to database-level cohort identification, although broader multi-center validation and ontology-based normalization will be needed for large-scale deployment.

The Segment Anything Model (SAM), a foundational vision model, struggles with fully automatic segmentation of specific objects. Its “segment everything” mode, reliant on a grid-based prompt strategy, suffers from localization blindness and computational redundancy, leading to poor performance on tasks like Dichotomous Image Segmentation (DIS). To address this, we propose PMG-SAM, a framework that introduces a Pre-Mask Guided paradigm for automatic targeted segmentation. Our method employs a dual-branch encoder to generate a coarse global Pre-Mask, which then acts as a dense internal prompt to guide the segmentation decoder. A key component, our proposed Dense Residual Fusion Module (DRFM), iteratively co-refines multi-scale features to significantly enhance the Pre-Mask’s quality. Extensive experiments on challenging DIS and Camouflaged Object Segmentation (COS) tasks validate our approach. On the DIS-TE2 benchmark, PMG-SAM boosts the maximal F-measure from SAM’s 0.283 to 0.815. Notably, our fully automatic model’s performance surpasses even the ground-truth bounding box prompted modes of SAM and SAM2, while using only 22.9 M trainable parameters (58.8% of SAM2-Tiny). PMG-SAM thus presents an efficient and accurate paradigm for resolving the localization bottleneck of large vision models in prompt-free scenarios.

Automatic crack segmentation is a cornerstone technology for intelligent visual perception modules in road safety maintenance and structural integrity systems. Existing deep learning models and “pre-training + fine-tuning” paradigms often face challenges of limited adaptability in resource-constrained environments and inadequate scalability across diverse data domains. To overcome these limitations, we propose FlexiCrackNet, a novel pipeline that seamlessly integrates traditional deep learning paradigms with the strengths of large-scale pre-trained models. At its core, FlexiCrackNet employs an encoder-decoder architecture to extract task-specific features. The lightweight EdgeSAM's CNN-based encoder is exclusively used as a generic feature extractor, decoupled from the fixed input size requirements of EdgeSAM. To harmonize general and domain-specific features, we introduce the information-Interaction gated attention mechanism (IGAM), which adaptively fuses multi-level features to enhance segmentation performance while mitigating irrelevant noise. This design enables the efficient transfer of general knowledge to crack segmentation tasks while ensuring adaptability to diverse input resolutions and resource-constrained environments. Experiments show that FlexiCrackNet outperforms state-of-the-art methods, excels in zero-shot generalization, computational efficiency, and segmentation robustness under challenging scenarios such as blurry inputs, complex backgrounds, and visually ambiguous artifacts. These advancements underscore the potential of FlexiCrackNet for real-world applications in automated crack detection and comprehensive structural health monitoring systems.

Tuberculosis (TB) remains a significant global health challenge, with millions of new cases and deaths reported annually. Early and accurate detection is crucial for controlling its spread and reducing morbidity. In this paper, we propose a novel Computer-Aided Diagnosis (CAD) system, SAM-LCA, for TB classification using chest X-ray (CXR) images. Built upon the Segment Anything Model (SAM-Med2D), our approach leverages its pre-trained image encoder for effective global representation learning. Additionally, we introduce two lightweight attention mechanisms, Improved Linear Attention (ILA) and Chunk-Wise Attention Block (CWAB), which significantly reduce computational complexity without compromising accuracy. The SAM-LCA model was evaluated on two public datasets, and demonstrated state-of-the-art performance in both binary and multi-class classification tasks. Extensive experiments, including ablation studies, show that SAM-LCA achieves near-perfect classification results with improved computational efficiency, making it well-suited for resource-constrained environments such as healthcare settings.

Most existing RGB-D semantic segmentation methods focus on the feature level fusion, including complex cross-modality and cross-scale fusion modules. However, these methods may cause misalignment problem in the feature fusion process and counter-intuitive patches in the segmentation results. Inspired by the popular pixel-node-pixel pipeline, we propose to 1) fuse features from two modalities in a late fusion style, during which the geometric feature injection is guided by texture feature prior; 2) employ Graph Neural Networks (GNNs) on the fused feature to alleviate the emergence of irregular patches by inferring patch relationship. At the 3D feature extraction stage, we argue that traditional CNNs are not efficient enough for depth maps. So, we encode depth map into normal map, after which CNNs can easily extract object surface tendencies.At projection matrix generation stage, we find the existence of Biased-Assignment and Ambiguous-Locality issues in the original pipeline. Therefore, we propose to 1) adopt the Kullback-Leibler Loss to ensure no missing important pixel features, which can be viewed as hard pixel mining process; 2) connect regions that are close to each other in the Euclidean space as well as in the semantic space with larger edge weights so that location informations can been considered. Extensive experiments on two public datasets, NYU-DepthV2 and SUN RGB-D, have shown that our approach can consistently boost the performance of RGB-D semantic segmentation task.

Integrating textual data with imaging in liver tumor segmentation is essential for enhancing diagnostic accuracy. However, current multi-modal medical datasets offer only general text annotations, lacking lesion-specific details critical for extracting nuanced features, especially for fine-grained segmentation of tumor boundaries and small lesions. To address these limitations, we developed datasets with lesion-specific text annotations for liver tumors and introduced the TexLiverNet model. TexLiverNet employs an agent-based cross-attention module that integrates text features efficiently with visual features, significantly reducing computational costs. Additionally, enhanced spatial and adaptive frequency domain perception is proposed to precisely delineate lesion boundaries, reduce background interference, and recover fine details in small lesions. Comprehensive evaluations on public and private datasets demonstrate that TexLiverNet achieves superior performance compared to current state-of-the-art methods. Code is available at https://github.com/skyvisionX/TexLiverNet.

Unsupervised domain adaptive (UDA) person re-identification (re-ID) models are of significant importance in practice since they can both utilizing annotated source domain and unlabeled target domain. However, many existing UDA person re-ID models train the source data and the target data independently. Moreover, since the intra-domain variance caused by appearance ambiguity across camera views and distinct source-target domain variations in different multi-camera configurations are common, existing UDA person re-ID models make assumptions, which constraints the model’s practical usage. Hence, we propose a novel joint training UDA framework for person re-ID without knowing the number of target class labels or camera viewing indices a priori. The new loss function jointly considers: 1) supervised learning of labeled source domain to learn discriminative invariant feature agent for individuals. 2) soft multi-label, instead of pseudo-label, assignment for target images by comparing with the source feature agent, hard negative pairs are then mined by evaluating the corresponding multi-label similarity consistency. 3) feature memory, is adopted to save up-to-date exemplar-invariance features for target domain images and to mine intra-domain invariance characteristics. The goal is to push away features of samples with similar appearance from distinct persons, and meanwhile, to pull closer the representation features of identical persons in the target domain. The loss function, which takes into account both invariant and variant characteristics for each person, encodes these three aspects to optimize the network efficiently. Abundant ablation studies show the effectiveness of the proposed framework. Comparing with the state-of-the-art methods on three challenging large-scale datasets, the proposed framework shows significant improvement in performance.

IntroductionLiver cancer is a leading cause of cancer-related mortality worldwide, necessitating advanced tools for diagnosis and management. Knowledge graphs (KGs) are crucial for advancing smart healthcare, but existing liver cancer-specific KGs are mostly derived from literature or public databases, lacking integration with real-world clinical data [e.g., Electronic Medical Records (EMRs)], creating a critical gap. Furthermore, there is currently no publicly available KGs specifically for liver cancer, creating a significant gap in structured clinical knowledge resources.MethodsThis study proposes a novel framework to construct the first Chinese liver cancer KG from Real-World Liver Cancer Electronic Medical Records (RLC-EMRs). A new named entity recognition (NER) model, DERM-RoBERTa-wwm-large-BiLSTM-CRF was developed that uses a Dynamic Entity Replacement and Masking (DERM) strategy to address data scarcity. Knowledge fusion was performed using the TF-IDF algorithm to standardize and integrate entities from clinical records, the professional medical website www.XYWY.com, and the CCMT-2019 terminology standard.ResultsThe final constructed liver cancer KG contained 46,364 entities and 296,655 semantic relationships. The proposed NER model achieved a state-of-the-art F1 score of 68.84% on the public CMeEE-v2 dataset. On the proprietary RLC-EMRs dataset, the model demonstrated high effectiveness with a precision of 93.23%, recall of 94.69%, and an F1 score of 93.96%. In addition, a KG-based retrieval system was successfully developed to query for complications, medications, and other related information.DiscussionThe findings demonstrated the effectiveness of the proposed framework in constructing a comprehensive and clinically relevant liver cancer KG. The novel DERM-based NER model significantly improved entity extraction from complex medical texts. By successfully integrating real-world clinical data, this study addresses a critical gap in existing liver cancer-specific KGs, which are mostly derived from literature or public databases and lack integration with real-world clinical information.

Accurate spatial perception is indispensable for robotic locomotion, with depth estimation playing a crucial role in enhancing spatial awareness. Recent advances have seen the integration of multimodal data becoming prevalent in robotic vision tasks, significantly improving precision and robustness in spatial perception. Despite the demonstrated potential of improving depth estimation in indoor settings through the fusion of visual and echoes data, aligning the transformed features from these modalities effectively continues to pose a challenge. In this paper, we propose a novel multimodal fusion approach that integrates visual and echo representations using a combination of self-attention and crossattention mechanisms, enabling better alignment between features from both modalities. To further enhance the fusion of global semantic information, we introduce an innovative global multimodal fusion framework, which incorporates 3D scene context into the feature extraction layers across modalities, leading to improved depth estimation results. Comprehensive experiments conducted on the Replica and Matterport3D datasets, along with comparisons to state-of-the-art methods, demonstrate the effectiveness of the proposed approach.

The Human-Object Interaction (HOI) detection task aims to locate humans and objects, find their matching relationships, and infer their interactions. While existing HOI methods have leveraged the CLIP model, a pre-trained visual-language model capable of understanding both images and text, to improve performance, they still fall short in fully capturing the complexity and fine-grained details of human-object interactions. As a result, their ability to reason about interactions accurately and in-depth remains limited. Therefore, we propose a knowledge-guided interaction perception module that combines multiple relationship information with CLIP’s visual feature information. Then, we utilize prior interaction knowledge from intersection regions to guide the process, resulting in more accurate human-object interaction detection. Moreover, we find that the potential interaction of images relies on subtle visual cues but is masked by other irrelevant information, making it difficult for algorithms to capture the basic features of interaction accurately. To address this, we have designed a human-object salient region enhancement module to enhance the feature information of humans and objects and enable better interaction pairing. Experimental results demonstrate that our method with knowledge guided (KGRE) achieves state-of-the-art performance on both the HICO-DET and V-COCO benchmark datasets.

The diversity and combinatorial complexity of real-world human-object interactions (HOI) make it impractical for any single dataset to encompass all variations. This requires models with robust zero-shot learning capabilities to infer unseen interactions. A novel approach for HOI detection is proposed, leveraging adaptive contrastive learning and class-specific feature enhancement. This method improves detection accuracy by assigning adaptive weights to challenging samples and incorporating additional human, object, and interaction-specific features. Adaptive contrastive learning computes dynamic weights based on the similarity between each HOI instance and its label, allowing the model to focus more on challenging and long-tail samples. Class-specific feature enhancement improves the matcher by separately calculating losses for humans, objects, and interactions. Experimental results on the HICO-DET and SWIG-HOI datasets demonstrate the effectiveness of the proposed method in detecting seen and unseen interactions. The code is available at https://github.com/small-code-cat/ACL-CFE (Adaptive Contrastive Learning-Class-specific Feature Enhancement).

Human-object interaction (HOI) detection is an important computer vision task for recognizing the interaction between humans and surrounding objects in an image or video. The HOI datasets have a serious long-tailed data distribution problem because it is challenging to have a dataset that contains all potential interactions. Many HOI detectors have addressed this issue by utilizing visual-language models. However, due to the calculation mechanism of the Transformer, the visual-language model is not good at extracting the local features of input samples. Therefore, we propose a novel local feature enhanced Transformer to motivate encoders to extract multi-modal features that contain more information. Moreover, it is worth noting that the application of prompt learning in HOI detection is still in preliminary stages. Consequently, we propose a multi-modal adaptive prompt module, which uses an adaptive learning strategy to facilitate the interaction of language and visual prompts. In the HICO-DET and SWIG-HOI datasets, the proposed model achieves full interaction with 24.21 https://github.com/small-code-cat/AMP-HOI .

Lightweight yet reliable depth estimation models that can deployed on edge devices are crucial for the practical application of fields such as autonomous driving, robot navigation, and augmented reality. However, previous research often fails to balance accuracy and complexity well. Existing lightweight models still have relatively high error rates in specific scenarios, which makes them unsuitable for industrial applications. Hence, we propose LiDUT-Depth, a lightweight architecture for self-supervised monocular depth estimation that is enhanced through dynamic upsampling and triplet loss optimization, which achieves competitive results with smaller model sizes and lower computational complexity. Specifically, an efficient dynamic upsampling module (EDU Module) is employed to preserve detailed local features, thereby obtaining more accurate depth estimation results. In addition, an improved semantic-aware triplet loss (SaTri Loss) is integrated into the training loss, significantly improving depth estimation accuracy in gradient-rich regions. Experiments show that our architecture achieves a considerably enhanced accuracy compared with previous works with a much lower model size. Our codes and models are available here.

3D perception of depth and ego-motion is of vital importance in intelligent agent and Human Computer Interaction (HCI) tasks, such as robotics and autonomous driving. There are different kinds of sensors that can directly obtain 3D depth information. However, the commonly used Lidar sensor is expensive, and the effective range of RGB-D cameras is limited. In the field of computer vision, researchers have done a lot of work on 3D perception. While traditional geometric algorithms require a lot of manual features for depth estimation, Deep Learning methods have achieved great success in this field. In this work, we proposed a novel self-supervised method based on Vision Transformer (ViT) with Convolutional Neural Network (CNN) architecture, which is referred to as ViT-Depth . The image reconstruction losses computed by the estimated depth and motion between adjacent frames are treated as supervision signal to establish a self-supervised learning pipeline. This is an effective solution for tasks that need accurate and low-cost 3D perception, such as autonomous driving, robotic navigation, 3D reconstruction, and so on. Our method could leverage both the ability of CNN and Transformer to extract deep features and capture global contextual information. In addition, we propose a cross-frame loss that could constrain photometric error and scale consistency among multi-frames, which lead the training process to be more stable and improve the performance. Extensive experimental results on autonomous driving dataset demonstrate the proposed approach is competitive with the state-of-the-art depth and motion estimation methods.

Breast cancer has become the most common cancer in the world. Early diagnosis and treatment can greatly improve the survival rate of breast cancer patients. Computer diagnostic technology based on convolutional neural networks (CNNs) can assist in detecting breast cancer based on medical images, effectively improving detection accuracy. Hyperparameters in CNN will affect model performance, so hyperparameter tuning is necessary for model training. However, traditional tuning methods can get stuck in local minimums. Therefore, the weights and biases of artificial neural networks are usually trained using global optimization algorithms. Our research introduces cat swarm optimization (CSO) to construct a cat swarm optimization-guided convolutional neural network (CSO-CNN). The model can quickly obtain the optimal combination of hyperparameters and stably get closer to the global optimal. The statistical results of CSO-CNN obtained a sensitivity of 93.50

Current convolution neural network (CNN)-based multiperson pose estimators have achieved great progress, however, they pay no or less attention to “hard” samples, such as occluded keypoints, small and nearly invisible keypoints, and ambiguous keypoints. In this article, we explicitly deal with these “hard” samples by proposing a novel multiscale coarse-to-fine human pose estimation network (HM $^{2}$ PN), which includes two sequential subnetworks: CoarseNet and FineNet. CoarseNet conducts a coarse prediction to locate “simple” keypoints like hands and ankles with a multiscale fusion module, which is integrated with bottleneck, resulting in a novel module called multiscale bottleneck. The new module improves the multiscale representation ability of the network in a fine-grained level, while marginally reducing the computation cost because of group convolution. FineNet further infers “hard” keypoints and refines “simple” keypoints simultaneously with a hard keypoint mining loss. Distinct from the previous works, the proposed loss deals with “hard” keypoints differentially and prevents “simple” keypoints from dominating the computed gradients during training. Experiments on the COCO keypoint benchmark show that our approach achieves superior pose estimation performance compared with other state-of-the-art methods. Source code is available for further research: https://github.com/sues-vision/C2F-HumanPoseEstimation.

The zero-shot classification performance of large-scale vision-language pre-training models (e.g., CLIP, BLIP and ALIGN) can be enhanced by incorporating a prompt (e.g., “a photo of a [CLASS]”) before the class words. Modifying the prompt slightly can have significant effect on the classification outcomes of these models. Thus, it is crucial to include an appropriate prompt tailored to the classes. However, manual prompt design is labor-intensive and necessitates domain-specific expertise. The CoOp (Context Optimization) converts hand-crafted prompt templates into learnable word vectors to automatically generate prompts, resulting in substantial improvements for CLIP. However, CoOp exhibited significant variation in classification performance across different classes. Although CoOp-CSC (Class-Specific Context) has a separate prompt for each class, only shows some advantages on fine-grained datasets. In this paper, we propose a novel automatic prompt generation method called F-SCP (Filter-based Specific Class Prompt), which distinguishes itself from the CoOp-UC (Unified Context) model and the CoOp-CSC model. Our approach focuses on prompt generation for low-accuracy classes and similar classes. We add the Filter and SCP modules to the prompt generation architecture. The Filter module selects the poorly classified classes, and then reproduce the prompts through the SCP (Specific Class Prompt) module to replace the prompts of specific classes. Experimental results on six multi-domain datasets shows the superiority of our approach over the state-of-the-art methods. Particularly, the improvement in accuracy for the specific classes mentioned above is significant. For instance, compared with CoOp-UC on the OxfordPets dataset, the low-accuracy classes, such as, Class21 and Class26, are improved by 18% and 12%, respectively.