The NerveStitcher demonstrated that a set of in vivo confocal microscopy (IVCM) images can be merged under a framework of graph convolutional neural network. However, the high similarity of the nerval structure on IVCM image results in mis-stitching when the internal images in sequence happen to drop. Particularly on large gap between a pair of adjacent images, NerveStitcher sometimes cannot detected. In this paper, we advance the concept of global optical flow of IVCM image pair and intergrate it to the existing framework. The large improvements in algorithm robustness are caused by that we make a trade of global displacement of the images sequence according to optical flow, and the local corresponding of key points. We firstly analyze the global shifts in pixel intensity and position between consecutive images, which allows us to determine the motion of each pixel. The displacement vector in the optical flow is then used to calculate the displacement distance of the image. After obtaining the displacement distance of each image using the optical flow, we then use NerveStitcher to stitch the same image sequence. By using a feature point matching algorithm, we can calculate the displacement distance of matching feature points in the two images. We then subtract the displacement distances obtained by the two methods and locate and modify the incorrect stitching results based on the difference value range. Experimental results showed that the improved algorithm, named NerveStitcher2.0, decreases the estimation error by more than 25%. The implemented code is available at https://github.com/better-77/nervestitcher2.0.
Lung cancer is a significant cause of cancer-related deaths globally. X-ray image has been widely used for first-stage screening as it is affordable and widely available. Recently, with the development of the gas sensor IC chip, low-cost enose sensing exhaled breath from patients can potentially be used for the first-stage screening in the near future. We propose a share-embedding model combining x-ray images and enose sensory signals to diagnose lung cancer. Our model contains two branches: the image branch and the enose branch. Since the lack of the enose data, we try to use the pretrained image model to guide the enose branch to align toward the embedding space that the image model learned. Our share-embedding model is designed to be robust to domain shifts across devices and environments. To further improve performance, we use semi-supervised learning with instance weighting to transfer the model to the unlabeled target domain. To train and evaluate the performance, we collect the first paired X-ray images and enose data across multiple devices and clinical environments. In the experiments, our method outperforms each individual branch and a feature concatenation fusion method. In the cross-device setting, our method leveraging semi-supervised learning achieves the best performance.
Gibbs-ringing artifact is a common artifact in medical image processing. It is usually caused by image enhancement procedures, like image deblurring, or physically caused in magnetic resonance imaging (MRI), where the k-space is a finite sampling of underlying signals. Under-sampling in Fourier space is often employed to reduce imaging time for real-time applications. In addition, ringing effect is regularly accompanied by noise boosting due to a low-grade sensor, resulting in lowering image quality to a large extent. In this work we propose hybrid approach for suppressing the stated artifacts, which utilizes recently proposed deep learning Fourier Neural Operator over the classical solvers: Perona-Malik diffusion and Kellner algorithm.
Driver distraction has been one of the primary causes of traffic accidents. Electroencephalography (EEG), a record of the electric potential from the scalp, is considered as a reliable indicator of brain activities. It has been widely used to detect driver distraction. Previous studies have analyzed driver distraction based on time and frequency domain features of EEG. However, challenges still exist in manifesting the distraction information of EEG which contains a large amount of complex information about driver distraction in realistic driving scenarios from the perspective of complexity. In this paper, we propose a driver distraction detection framework using Random Forest (RF) based on the complexity feature fusion of EEG in real driving environment. Five entropy-based features of EEG are firstly extracted with a sliding window. Then, an RF classifier is trained with the extracted features to detect driver distraction. Our results show that differential entropy (DE) with an accuracy of 72.9% achieves the best result while single type feature is applied to detect distraction. The classifier's accuracy is further increased by about 7% using fused multiple features compared with the highest accuracy obtained by single type feature. In terms of feature contribution, we found that the feature with the best distraction detection result by using single type features may not contribute the most when using fused multiple features.
Vision transformer models began gaining recognition alongside NLP. However, their performance compared to Convolutional Neural Network (CNN) models in this domain still requires more significant investigations. Hence, this article comprehensively analyzes their impact and effectiveness against CNN models in medical imaging. We conducted experiments using ViT, MobileViT, and Swin transformers against a pure CNN ConvNeXt trained on Magnetic Resonance Image (MRI) scans. While our findings show promising advancements in imaging with transformers, we observed challenges in their scalability and deployment due to their cost and complexity. We also noticed that they require more medical data, specifically higher-quality MRI scans, when considering better reliability. Nonetheless, comparing their performance in terms of accuracy, even with such limitations, these visual transformer models have shown better detection and diagnosis of brain tumors in MRI scans compared to pure CNN models selected in this study.
