In recent years, CNNs have demonstrated remarkable success in medical image processing due to their powerful feature extraction capabilities. However, their inherent limitation hampers their ability to learn global features. In contrast, Transformers excel at modeling long-range dependencies, offering a solution to this limitation. Yet, the computational demands of self-attention make high-resolution medical image processing-particularly for 3D data-extremely resource-intensive. To address these challenges, we introduce DeTransNet, a hybrid segmentation method that harnesses the strengths of both CNNs and deformable Transformer. First, we introduce a multi-channel feature fusion module (MCFF) that efficiently integrates multi-scale features from CNNs to enhance the input for the Transformer network. Additionally, we propose a deformable attention mechanism to improve feature extraction, enabling the model to capture global information while reducing self-attention complexity. Extensive experiments on the public BCV dataset demonstrate that the proposed method outperforms existing methods.
ChatGPT faces significant challenges in realising the dynamic construction of reality, cognition and language emphasised in Embodied-cognitive translatology. This study first examines the translation application of ChatGPT, defines hallucination phenomena, and reviews related research. It then explores the hallucination phenomena in ChatGPT's translation tasks, categorizing them into semantic, pragmatic, syntactic, and cultural hallucinations. To quantify these manifestations, we manually annotate ChatGPT's English-Chinese translations on the Flores-101 dataset, providing statistical insights into the frequency and severity of each type of hallucination. This analysis offers clarity on the prevalence of various hallucinations and suggests directions for improvement. Finally, this study summarises the current research on hallucination mitigation strategies in a targeted way, including the improvement of corpus quality, model architecture development, and prompt engineering techniques, offering novel insights for advancing machine translation technology towards greater intelligence.
Alzheimer's disease (AD) is a common and dangerous disorder that primarily impacts older adults, early detection is crucial, and diagnostic tools like PET and MRI have an impact on offering detailed anatomical and metabolic insights, respectively. However, traditional methods usually directly concatenate the two modal data by channel, which fails to fully utilize the complementary information provided by MRI and PET data. Hence, this paper introduces a new multi-modal dynamic information selection framework to enhance the accuracy of AD classification. It includes three main modules: a dual ResNet50-based feature extraction module; a modal fusion module containing a feature pyramid for finer detail extraction and a cross-attention mechanism to integrate modal information; a dynamic information selection module that evaluates the data content across modalities to optimize decision-making. The outcome on the ADNI dataset confirm the efficiency of the proposed approach.
Research on adversarial attacks for facial recognition identifies and mitigates vulnerabilities in facial recognition systems, enhancing their security. Current adversarial attacks targeting facial attributes lack precision in controlling specific attributes, often altering multiple facial features instead of a single target attribute. Such imprecise modifications reduce the attack's effectiveness and stealthiness. To address these issues, this paper proposes a method for generating adversarial examples based on multi-layer feature map fusion. First, we design a framework called StarAdv, which uses fused multi-layer feature maps with high concealment to achieve realistic facial image transformations under different attributes. This approach generates adversarial images visually similar to the originals while possessing specific misleading properties, enabling transferable attacks on facial recognition systems. Second, we introduce a Multi-layer Fusion module that captures weight information from each layer of the network's residual blocks to adaptively fuse feature maps from different layers, producing high-concealment feature maps. Finally, we propose a Bilinear Feature Map Interpolation (BFMI) algorithm to interpolate the high-concealment features with the original image features, ensuring the final decoded adversarial samples maintain a natural appearance, further reducing the likelihood of detection. We conduct comprehensive experiments on the CelebA dataset, demonstrating that the adversarial facial images generated by our method possess semantic plausibility and authenticity in appearance and achieve a high attack success rate in both white-box and black-box settings.
In response to the current challenges of large parameter size and low detection accuracy in SAR ship detection models, this paper proposes an improved YOLOv8 model. The model integrates the Swin-Transformer architecture into YOLOv8 through an adaptive feature fusion method, enhancing the model’s global information perception and detection accuracy. Additionally, it employs a top-down unidirectional semantic pyramid and a lightweight detection head structure to achieve model lightweighting. Experimental comparisons on the HRSID dataset show that, compared to the benchmark algorithm, the proposed algorithm increases mAP50 by 1.2
Addressing the domain shift challenge between datasets is critical to maintaining model performance. However, existing methods rarely consider the behavior of the human brain when dealing with the domain shift, resulting in poor model segmentation results. Brain-inspired strategies can enhance the migration ability and generalization performance of deep learning models across different data sets by simulating the mechanisms of the human brain in processing complex visual information. Inspired by this, we propose a Brain-inspired Domain Adaptation Network (BDANet) to solve the domain shift problem. Specifically, we adopt the teacher-student architecture for mutual learning and adversarial learning. Both the teacher and student models contain a dual-branch encoder and a brain-inspired decoder for object segmentation. Inspired by the cognitive process of the human brain, we propose a brain-inspired fusion module in the dual-branch encoder to effectively fuse the dual-branch features to obtain complete object information. In the brain-inspired decoder, we propose a brain-inspired refinement module to gradually refine the initial segmentation result and accurately segment the edges of objects. Extensive experiments demonstrate that the proposed method significantly outperforms the existing competitors on the challenging benchmark dataset under evaluation metrics.
Automatically parsing and reassembling fragmented 3D models is a critical challenge in the field of 3D modeling. However, existing methods often rely heavily on manual feature engineering, limiting their flexibility and performance. In this work, we present a novel and compact approach called ReassemblingNet that directly parses and reassembles instance-level 3D models from fragmented pieces. ReassemblingNet leverages a deep neural network architecture that efficiently analyzes cluttered 3D fragments and predicts the necessary transformation matrices to enable seamless reassembly. To support the training of this model, we curated a large-scale dataset named PvBreaks, which contains 2,800 fragmented pieces obtained by dissecting 20 pot and vase models from the Stanford Shape Benchmark. Through extensive experimentation, we demonstrate that our ReassemblingNet approach can effectively capture the intricate features present in both virtual and real-world 3D fragments, leading to successful reassembly.
Memory limitation of a single GPU is an urgent problem for efficient training of deep learning models on multi-GPU clusters, as a large number of model parameters, intermediate states, and activation values need to be stored during training. Efficient management and allocation of GPU memory is the key to improving memory resource utilization and model training efficiency. The existing methods mainly optimize the layout of memory space when training data is allocated and released to reduce memory waste. These methods do not consider the structural differences between models and the memory access feature of different data (model parameters, intermediate data, etc.) during the model training, so there are still problems of memory fragmentation and low utilization. To address these problems, this paper proposes an efficient GPU-memory allocation and management method, TMManager. It proposes an access feature analyzer, sampling and analyzing the structure of the model and the access features of memory for different data in the model training. Then, we design a dual-level memory partition management method with block and chunk, and a time-sharing deque memory allocation method, to reduce memory fragmentation and improve memory utilization. The experiments demonstrate that TMManager can save up to 23.5
3D point cloud-based object detection is a crucial topic in the field of computer vision. This paper addresses the issue of low detection accuracy in 3D object detection using LiDAR during autonomous driving by proposing an improved 3D object detection method based on PointPillars. Firstly, in the feature extraction stage of the PointPillars model, the CA attention mechanism is embedded in the backbone network to learn similarity information and focus on important features. By calculating attention in two spatial dimensions (height and width) separately, the spatial distribution characteristics in the image are captured more accurately, thereby more comprehensively capturing the dependency relationships between features. Secondly, when performing feature fusion on the extracted information, instead of simply adding or concatenating the feature maps directly, suitable weights are learned for each position to select features of different scales, enabling more effective fusion. This allows the model to more efficiently detect feature information at different scales. To verify the performance of the improved algorithm, experiments were conducted on the publicly available KITTI dataset. The experimental results show that the proposed optimization algorithm achieves certain improvements over other publicly available algorithms. Compared to the baseline algorithm PointPillars, the average precision was improved by 4.48%.
Shadow removal has long been a challenging task in the field of computer vision. Despite the significant progress since the advent of deep learning, existing approaches still fall short of robustness in generating high-quality shadow-free images due to the scarcity of training data and the complexity of shadow images. In this paper, we approach the task as an image-denoising process and present a deep neural network based on the ControlNet-driven stable diffusion model, whose rich prior knowledge compensates for the data shortage and better facilitates the task modeling of shadow removal. Moreover, we simulate multi-exposure shadow images as conditional inputs to regularize the denoising process during training. An intensity modulation block is also integrated to boost the intensity of the recovered scene. Experiments on two benchmark datasets, ISTD+ and SRD, witness the superior performance of the proposed approach.
Recently, point cloud semantic segmentation technology has made significant progress, boosting the development of autonomous driving, robotic navigation, and urban modeling. However, most current approaches rely on training data of all categories at once. This limitation makes it difficult for models to adapt to dynamic environments, leading to repetitive retraining and high computational costs. To enable continuous learning of new categories by leveraging previous knowledge, and inspired by the brain's ability to learn new knowledge through comparison and association, we introduce a cross-generational contrastive continual learning approach for 3D point cloud semantic segmentation. To mitigate catastrophic forgetting, we contrast representations of old classes and new classes across different generations of encoders. Further, we propose a refined labels guided contrastive loss, which comprehensively accounts for the semantic dependencies between points and leverages previous knowledge. Additionally, we propose a refined label estimation strategy to boost the confidence of all classes while retaining previous knowledge. Extensive experiments on two public 3D point cloud semantic segmentation benchmarks demonstrate the effectiveness of our proposed approach.
Rib segmentation is one of the most challenging tasks in the field of medical image segmentation, and also an important tool to assist doctors in diagnosis. However, the challenge of this task is that ribs overlap each other and have low contrast and blurred edges. To address these issues, we propose a novel Disentanglement Enhancement Network called DENet, which disentangles and enhances appearance representations of ribs via the segmentation difficulties and location priors respectively, for robust rib segmentation. In particular, we design a difficulty-guided representation disentanglement module to focus on the most challenging ribs by using the separate decoders for these challenging ribs. To leverage the relations among ribs, we design a location-aware mutual enhancement module, which enables the information exchange and enhancement among different ribs according to the location priors. The experimental results show that mDice of our method is improved by 2
In order to alleviate the problem of person re-identification task's dependence on labeled data and low accuracy in occlusion scenarios, an unsupervised person re-identification method combining random occlusion and ContrastiveCrop is proposed. Firstly, the input image is randomly occluded with various patterns according to the real scene, and the contrast cropping method is used to generate pedestrian image samples with greater differences while ensuring the semantic consistency of the positive sample pair, so as to alleviate the impact of occlusion and complex background on the network. Then, the SC-CAResNet network was designed in the contrastive learning model for multi-granularity feature extraction, so that it can pay more attention to the important areas of the image; finally, the network was trained by combining multiple loss functions. Experimental results show that this method outperforms traditional classical methods on two public datasets of person re-identification, Market-1501 and DukeMTMC-reID, has stronger robustness, and significantly improves the ability of person re-identification in model occlusion scenes.
Palmprint recognition, as a biometric technology, is highly valued for its uniqueness and stability. This paper integrates traditional palmprint recognition techniques with deep learning-based methods, proposing a palmprint texture fusion ViT (PTF-ViT), aimed at enhancing the accuracy and robustness of palmprint recognition. We leverage the advantages of the Vision Transformer (ViT) to fuse the texture features of palmprint with original image information through an attention mechanism, thereby enhancing the model's discriminative ability. Experimental results indicate that PTF-ViT performs exceptionally well across various palmprint datasets, exhibiting high Average Recognition Rate (ARR) and low Equal Error Rate (EER), especially demonstrating strong robustness on cross-device datasets. Compared to traditional methods and existing deep learning approaches, PTF-ViT holds significant advantages in the field of palmprint recognition, proving its effectiveness and advancement. This study not only fills the void of ViT in palmprint recognition but also paves a new direction for future research in biometric technology.
Constructing the Brain Rhythm Entropy Matrix (BREM) as a feature for emotion recognition is feasible due to the correlation between brain rhythm activities and arousal/valence levels, as well as the effectiveness of entropy in quantifying such activities. Besides, similarity, a vital metric in bioinformatics, measures the degree of resemblance between internal elements. Building on these foundations, this paper introduces an emotion recognition method that leverages the similarity measures of BREM derived from EEG signals. The results using the DEAP database indicate that the proposed method achieves an accuracy range of 75
Facial acne is a prevalent skin disease worldwide. Misjudging the severity of its condition can lead to permanent facial damage or negatively impact patients' self-esteem. With the rapid advancements in computer vision techniques, automated methods for detecting or grading facial acne in digital images are attracting growing attention. However, in clinical practice, multi-view (left, front, and right) face images used for accurate diagnosis pose a significant challenge due to redundant acne detection on the overlapped image regions that can mislead diagnostic results. In this paper, we propose a novel facial acne detection framework to detect various kinds of acne for accurate acne diagnosis or prognosis prediction. Specifically, multi-view facial images are first stitched by aligning key facial feature points, and then the proposed acne detection network YoloV5-Acne is then applied to the stitched images for patient-level acne detection. Experimental results on a collected multi-view face image dataset (ACNE-Shanghai) and a public acne dataset (ACNE04) validate the effectiveness of the proposed method on various evaluation metrics. The code and the dataset will be released for the research purpose once the paper gets published.
Decoding information related to language cognition from Electroencephalogram (EEG) signals can reveal learners' genuine responses to knowledge. Existing language learning research using EEG experimental paradigms often focuses on abstract stimuli, which undermines the accuracy of reflecting real experiences in the actual learning process. This study presents a more natural EEG experimental paradigm for second language(L2) learners, recording EEG signals in video-based language learning scenarios. Data analysis from 12 participants confirms the paradigm's validity, allowing us to establish a mapping model between EEG signals and learners' states of language acquisition, achieving an accuracy of 0.95. These findings demonstrate the feasibility of decoding cognitive states from real-world scenarios, highlighting the proposed paradigm's significant research potential in second language learning and offering insights into the brain's linguistic cognitive processes.
In recent years, Action Quality Assessment technology has been introduced to evaluate hand hygiene. However, current two-stage methods, which separate segmentation and Action Quality Assessment, suffer from unstable evaluation due to the reliance on the models for each respective task. Additionally, single-stage methods perform poorly in long-duration videos due to extensive background noise interference. Moreover, existing hand hygiene datasets have low annotation density and limited samples. To alleviate these limitations in methodology and data, this paper proposes a Single-stage Dual-task Joint Learning (SDJL) framework and a hand hygiene evaluation dataset (HHA1009). First, this model utilizes the vision transformer with the fixed token as the backbone, significantly improving computational efficiency and prediction accuracy by jointly learning Action Segmentation and quality assessment tasks. Additionally, the HHA1009 dataset captures video sequences in various scenarios and the individual action in each video is scored by multiple raters. The experimental results on the HHA1009 show the superior performance of the proposed SDJL, ablation studies further confirm the effectiveness of each module.
For distributed computing, there is an urgent need to solve the problem of device placement, so we propose an innovative graph neural network framework NEAGMamba. It employs a decoupling and importance score evaluation mechanism to prevent excessive smoothing of node encodings during the aggregation process. In addition, a new Graph Mamba Network (Gmamba) is designed to replace the traditional GNN, capable of extracting node characteristics and capturing remote dependencies. Specifically, the features of each node are decoupled into two parts: discriminant features and aggregation features. At the same time, the proposed GMamba is used to obtain the aggregation features. Finally, the importance of adjacent nodes is evaluated by the importance score to realize the integration of adjacent information. The experimental results showed that the execution time was improved by 13.55
This paper evaluates several representative algorithms on real datasets. It analyzes Vertex Component Analysis (VCA), Total Variation Regularized Reweighted Sparse Nonnegative Matrix Factorization (RSNMF), Sparse Hyperspectral Unmixing (HU) with Mixed Norms, and Hapke Model with Convolutional Neural Network (HapkeCNN). Results indicate that VCA achieves high accuracy in endmember extraction and is widely applicable, RSNMF perform well with fewer, distinct endmembers, Sparse HU with Mixed Norms estimates abundances effectively without high-precision endmembers, addressing endmember variability, and HapkeCNN excels in nonlinear fitting and noise resistance, validating the effectiveness of cognitive models in unmixing tasks.
