Recent advancements in 3D reconstruction helped endoscopy doctors analyze the patients' gastrointestinal surfaces and abnormality detections. In this work, we expand this development further with a reconstruction method based on both classic techniques like structure from motion and recent advanced techniques like neural radiation fields and Gaussian splatting with new Gaussian encoding-decoding modules. In addition, an unique dataset was collected with some videos from daily endoscopy examinations. This development helped us achieve better reconstruction results and lower training time compared to existing methods.
In this paper, we address critical challenges in medical image generation using deep learning techniques. While convolutional neural networks and generative adversarial networks (GANs) have achieved remarkable results in various image generation tasks, their application to medical imaging faces unique obstacles. These include the complexity and diversity of medical images, limitations in discriminator network structures, and the risk of model collapse and gradient vanishing in multi-scale discriminators. To overcome these issues, we propose a novel deterministic guided progressive GAN that specifically targets regions of interest (ROI) in medical images. Our approach progressively integrates adversarial generative networks, evolving from single-scale to multi-scale discriminators, to produce higher quality images. We demonstrate the efficacy of our model in generating high-precision cross-modal medical images through four comprehensive evaluation criteria, providing both quantitative and qualitative evidence of its performance compared to real images. This innovative method promises to significantly advance the field of medical image generation, potentially enhancing diagnostic accuracy and research capabilities in healthcare.
Automated annotation of circuit structures can generate hierarchical representations of analog circuit networks, thereby advancing the development of automated analog circuit design tasks. This paper introduces a graph attention network-based model that transforms circuit netlists into graph structures, proposes a feature extraction strategy to learn and predict the circuit structures composed of nodes in the netlists, and presents a method for quickly generating a large number of SPICE circuit netlists to provide ample data for training the graph model. Experiments compared the recognition effects of graph convolutional networks, graph isomorphism networks, and GraphSAGE on the same dataset. The results show that the GAT model outperforms the other models in accuracy, precision, and mean average precision, achieving 90.9
Semantic segmentation plays a crucial role in computer-aided medical image analysis by achieving important and useful regions, which are vital for various diagnostic tasks. Recently, vision transformers (ViTs) have emerged as the leading approach in medical image segmentation, outperforming traditional convolutional neural networks (CNNs). The incorporation strategies of the ViTs for medical segmentation are dominated to leverage the widely used U-shape like architecture (U-Net) while replace the convolution blocks in both encoder and decoder paths using transformer blocks. It remains uncertain which components of the incorporated transformer block contribute most significantly to segmentation results in the medical field. This study presents a hierarchical feature aggregation method based on hierarchical Transformer features to enhance the performance of ViT-based architecture in data-constrained medical image segmentation. Specifically, our approach employs the hierarchical vision Transformer to configure the main encoder path for extracting multi-scale semantic features, and leverages several residual blocks to achieve local representation with detail spatial information. Then, we introduce a hierarchical feature aggregation module (HFAM) to serve as the decoder path for fusing multi-scale semantic features and residual spatial features. Compared with the existing transformer-based U-Net, the explored HFAM can not only effectively combine the diverse contexts but also potentially reduce the computational complexity. Experiments on 3 different medical image segmentation benchmarks have demonstrated our proposed method consistently outperformers the conventional U-Net, and various Transformer-based U-Net.
Urban land use maps at the building instance level are crucial geo-information for many applications, yet they are challenging to obtain. Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the last few decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. However, for many applications, such as urban population density estimation or urban utility mapping, a classification map based on individual buildings (residential, commercial, mixed-type, and religious) is much more informative. Nonetheless, this type of semantic classification still poses fundamental challenges, such as retrieving fine boundaries of individual buildings. Street view images (SVI) are highly suited for predicting building functions because building facades provide clear hints. Although SVIs are used in many studies, their application in generating building usage maps is limited. Furthermore, their application to Indian cities remains void. In this paper, we propose a comprehensive framework for classifying the functionality of individual buildings. Our method leverages the YOLOs model and utilizes SVIs, including those from Google Street View and OpenStreetMap. Geographic information is employed to mask individual buildings and associate them with the corresponding SVIs. We created our own dataset in Indian cities for training and evaluating our model.
