This article presents a general Bayesian learning framework for multi-modal groupwise image registration. The method builds on probabilistic modelling of the image generative process, where the underlying common anatomy and geometric variations of the observed images are explicitly disentangled as latent variables. Therefore, groupwise image registration is achieved via hierarchical Bayesian inference. We propose a novel hierarchical variational auto-encoding architecture to realise the inference procedure of the latent variables, where the registration parameters can be explicitly estimated in a mathematically interpretable fashion. Remarkably, this new paradigm learns groupwise image registration in an unsupervised closed-loop self-reconstruction process, sparing the burden of designing complex image-based similarity measures. The computationally efficient disentangled network architecture is also inherently scalable and flexible, allowing for groupwise registration on large-scale image groups with variable sizes. Furthermore, the inferred structural representations from multi-modal images via disentanglement learning are capable of capturing the latent anatomy of the observations with visual semantics. Extensive experiments were conducted to validate the proposed framework, including four different datasets from cardiac, brain, and abdominal medical images. The results have demonstrated the superiority of our method over conventional similarity-based approaches in terms of accuracy, efficiency, scalability, and interpretability.

Cross-modality synthesis (CMS), super-resolution (SR), and their combination (CMSR) have been extensively studied for magnetic resonance imaging (MRI). Their primary goals are to enhance the imaging quality by synthesizing the desired modality and reducing the slice thickness. Despite the promising synthetic results, these techniques are often tailored to specific tasks, thereby limiting their adaptability to complex clinical scenarios. Therefore, it is crucial to build a unified network that can handle various image synthesis tasks with arbitrary requirements of modality and resolution settings, so that the resources for training and deploying the models can be greatly reduced. However, none of the previous works is capable of performing CMS, SR, and CMSR using a unified network. Moreover, these MRI reconstruction methods often treat alias frequencies improperly, resulting in suboptimal detail restoration. In this paper, we propose a Unified Co-Modulated Alias-free framework (Uni-COAL) to accomplish the aforementioned tasks with a single network. The co-modulation design of the image-conditioned and stochastic attribute representations ensures the consistency between CMS and SR, while simultaneously accommodating arbitrary combinations of input/output modalities and thickness. The generator of Uni-COAL is also designed to be alias-free based on the Shannon-Nyquist signal processing framework, ensuring effective suppression of alias frequencies. Additionally, we leverage the semantic prior of Segment Anything Model (SAM) to guide Uni-COAL, ensuring a more authentic preservation of anatomical structures during synthesis. Experiments on three datasets demonstrate that Uni-COAL outperforms the alternatives in CMS, SR, and CMSR tasks for MR images, which highlights its generalizability to wide-range applications.

Resting-state fMRI (rs-fMRI) is an effective tool for quantifying functional connectivity (FC), which plays a crucial role in exploring various brain diseases. Due to the high dimensionality of fMRI data, FC is typically computed based on the region of interest (ROI), whose parcellation relies on a pre-defined atlas. However, utilizing the brain atlas poses several challenges including 1) subjective selection bias in choosing from various brain atlases, 2) parcellation of each subject's brain with the same atlas yet disregarding individual specificity; 3) lack of interaction between brain region parcellation and downstream ROI-based FC analysis. To address these limitations, we propose a novel randomizing strategy for generating brain function representation to facilitate neural disease diagnosis. Specifically, we randomly sample brain patches, thus avoiding ROI parcellations of the brain atlas. Then, we introduce a new brain function representation framework for the sampled patches. Each patch has its function description by referring to anchor patches, as well as the position description. Furthermore, we design an adaptive-selection-assisted Transformer network to optimize and integrate the function representations of all sampled patches within each brain for neural disease diagnosis. To validate our framework, we conduct extensive evaluations on three datasets, and the experimental results establish the effectiveness and generality of our proposed method, offering a promising avenue for advancing neural disease diagnosis beyond the confines of traditional atlas-based methods. Our code is available at https://github.com/mjliu2020/RandomFR.

Magnetic resonance (MR) images collected in 2D clinical protocols typically have large inter-slice spacing, resulting in high in-plane resolution and reduced through-plane resolution. Super-resolution technique can enhance the through-plane resolution of MR images to facilitate downstream visualization and computer-aided diagnosis. However, most existing works train the super-resolution network at a fixed scaling factor, which is not friendly to clinical scenes of varying inter-slice spacing in MR scanning. Inspired by the recent progress in implicit neural representation, we propose a Spatial Attention-based Implicit Neural Representation (SA-INR) network for arbitrary reduction of MR inter-slice spacing. The SA-INR aims to represent an MR image as a continuous implicit function of 3D coordinates. In this way, the SA-INR can reconstruct the MR image with arbitrary inter-slice spacing by continuously sampling the coordinates in 3D space. In particular, a local-aware spatial attention operation is introduced to model nearby voxels and their affinity more accurately in a larger receptive field. Meanwhile, to improve the computational efficiency, a gradient-guided gating mask is proposed for applying the local-aware spatial attention to selected areas only. We evaluate our method on the public HCP-1200 dataset and the clinical knee MR dataset to demonstrate its superiority over other existing methods.

Automated detection of cervical abnormal cells from Thin-prep cytologic test (TCT) images is crucial for efficient cervical abnormal screening using computer-aided diagnosis systems. However, the construction of the detection model is hindered by the preparation of the training images, which usually suffers from issues of class imbalance and incomplete annotations. Additionally, existing methods often overlook the visual feature correlations among cells, which are crucial in cervical lesion cell detection as pathologists commonly rely on surrounding cells for identification. In this paper, we propose a distillation framework that utilizes a patch-level pre-training network to guide the training of an image-level detection network, which can be applied to various detectors without changing their architectures during inference. The main contribution is three-fold: (1) We propose the Balanced Pre-training Model (BPM) as the patch-level cervical cell classification model, which employs an image synthesis model to construct a class-balanced patch dataset for pre-training. (2) We design the Score Correction Loss (SCL) to enable the detection network to distill knowledge from the BPM model, thereby mitigating the impact of incomplete annotations. (3) We design the Patch Correlation Consistency (PCC) strategy to exploit the correlation information of extracted cells, consistent with the behavior of cytopathologists. Experiments on public and private datasets demonstrate the superior performance of the proposed distillation method, as well as its adaptability to various detection architectures.

Automatic thin-prep cytologic test (TCT) screening can assist pathologists in finding cervical abnormality towards accurate and efficient cervical cancer diagnosis. Current automatic TCT screening systems mostly involve abnormal cervical cell detection, which generally requires large-scale and diverse training data with high-quality annotations to achieve promising performance. Pathological image synthesis is naturally raised to minimize the efforts in data collection and annotation. However, it is challenging to generate realistic large-size cytopathological images while simultaneously synthesizing visually plausible appearances for small-size abnormal cervical cells. In this paper, we propose a two-stage image synthesis framework to create synthetic data for augmenting cervical abnormality screening. In the first Global Image Generation stage, a Normal Image Generator is designed to generate cytopathological images full of normal cervical cells. In the second Local Cell Editing stage, normal cells are randomly selected from the generated images and then are converted to different types of abnormal cells using the proposed Abnormal Cell Synthesizer. Both Normal Image Generator and Abnormal Cell Synthesizer are built upon the pre-trained Stable Diffusion via parameter-efficient fine-tuning methods for customizing cytopathological image contents and extending spatial layout controllability, respectively. Our experiments demonstrate the synthetic image quality, diversity, and controllability of the proposed synthesis framework, and validate its data augmentation effectiveness in enhancing the performance of abnormal cervical cell detection.

In clinical practice, 2D magnetic resonance (MR) sequences are widely adopted. While individual 2D slices can be stacked to form a 3D volume, the relatively large slice spacing can pose challenges for both image visualization and subsequent analysis tasks, which often require isotropic voxel spacing. To reduce slice spacing, deep-learning-based super-resolution techniques are widely investigated. However, most current solutions require a substantial number of paired high-resolution and low-resolution images for supervised training, which are typically unavailable in real-world scenarios. In this work, we propose a self-supervised super-resolution framework for inter-slice super-resolution of MR images. Our framework is first featured by pre-training on video dataset, as temporal correlation of videos is found beneficial for modeling the spatial relation among MR slices. Then, we use public high-quality MR dataset to fine-tune our pre-trained model, for enhancing awareness of our model to medical data. Finally, given a target dataset at hand, we utilize self-supervised fine-tuning to further ensure our model works well with user-specific super-resolution tasks. The proposed method demonstrates superior performance compared to other self-supervised methods and also holds the potential to benefit various downstream applications.

Magnetic resonance (MR) images collected in 2D scanning protocols typically have large inter-slice spacing, resulting in high in-plane resolution but reduced through-plane resolution. Super-resolution techniques can reduce the inter-slice spacing of 2D scanned MR images, facilitating the downstream visual experience and computer-aided diagnosis. However, most existing super-resolution methods are trained at a fixed scaling ratio, which is inconvenient in clinical settings where MR scanning may have varying inter-slice spacings. To solve this issue, we propose Hierarchical Feature Conditional Diffusion (HiFi-Diff) for arbitrary reduction of MR inter-slice spacing. Given two adjacent MR slices and the relative positional offset, HiFi-Diff can iteratively convert a Gaussian noise map into any desired in-between MR slice. Furthermore, to enable fine-grained conditioning, the Hierarchical Feature Extraction (HiFE) module is proposed to hierarchically extract conditional features and conduct element-wise modulation. Our experimental results on the publicly available HCP-1200 dataset demonstrate the high-fidelity super-resolution capability of HiFi-Diff and its efficacy in enhancing downstream segmentation performance.

Using a single three-component geophone to estimate the direction of arrival of a near sound source in the air is a challenging study. Through experimental data, we found that the azimuth angle obtained by using the polarization characteristics of the most energetic part of the signal is different from the true value. The research first analyzes the reasons for this phenomenon, and it is thought that the multipath effect of acoustic-seismic coupling may cause most signals to become interference, which makes it diffi-cult to detect useful Rayleigh waves. Then we propose a method to detect Rayleigh waves by combining polarization characteristics and frequency domain peaks. The simulation and actual data shows that the method proposed in this study is still effective in estimating the azimuth of different sound sources in the air in a low signal-to-noise ratio environment. This study also has important implications for the appli-cation of long-distance acoustic-seismic coupling and hydroacoustics.(c) 2023 Elsevier Ltd. All rights reserved.

Motion capture is a long-standing research problem. Although it has been studied for decades, the majority of research focus on ground-based movements such as walking, sitting, dancing, etc. Off- grounded actions such as climbing are largely overlooked. As an important type of action in sports and firefighting field, the climbing movements is challenging to capture because of its complex back poses, intricate human-scene interactions, and difficult global localization. The research community does not have an indepth understanding of the climbing action due to the lack of specific datasets. To address this limitation, we collect CIMI4D, a large rock Climbing Motion dataset from 12 persons climbing 13 different climbing walls. The dataset consists of around 180,000 frames of pose inertial measurements, LiDAR point clouds, RGB videos, high-precision static point cloud scenes, and reconstructed scene meshes. Moreover, we frame-wise annotate touch rock holds to facilitate a detailed exploration of human-scene interaction. The core of this dataset is a blending optimization process, which corrects for the pose as it drifts and is affected by the magnetic conditions. To evaluate the merit of CIMI4D, we perform four tasks which include human pose estimations (with/without scene constraints), pose prediction, and pose generation. The experimental results demonstrate that CIMI4D presents great challenges to existing methods and enables extensive research opportunities. We share the dataset with the research community in http://www.lidarhumanmotion.net/cimi4d/.

Purposes The density of organic matter in oil shale is much lower than that of other rock matrix, so the gray value of organic matter in CT images is often close that of pore fractures, which results in problems such as inconspicuous difference in gray value and blurred boundary between organic matter and rock in the images. In order to accurately identify the organic matter in the segmented oil shale CT images, the image segmentation methods in the field of deep learning are studied, and the OM-Unet semantic segmentation network architectures describing the organic matter segmentation is built independently. Methods By introducing a hybrid null convolution module, a coarse-to-fine deployment strategy, and a lightweight adaptive feature fusion module into the traditional Unet model, the convolutional neural network is used to identify and segment organic matter in oil shale CT images, and its segmentation effect is evaluated by combining MIoU and other evaluation indexes. Findings The MIoU of the OM-Unet model is 80.66%, which is higher than that of the three-phase segmentation methods, Unet, CBAM-Unet, DeepLabV3, HDC-Unet, and LAFF-Unet models by 8.01%, 17.68%, 9.5%, 2.54%, 2.83%, and 9.13%, respectively. The MPA of OM-Unet model is 89.16%, which is higher than that of the three-phase segmentation method, Unet, CBAM-Unet, DeepLabV3, HDC-Unet, and LAFF-Unet models by 12.85%, 20.62%, 15.82%, 8.81%, 9.55%, and 15.34%, respectively. Conclusions The results demonstrate that the OM-Unet model can effectively improve the accuracy of oil shale organic matter partitioning, more accurately determine the variation patterns of organic matter volume percentage and organic matter cluster number with temperature or pyrolysis conditions, and provide basic theoretical data for in situ oil shale development.

Magnetic Resonance Imaging (MRI) has become an essential tool for clinical knee examinations. In clinical practice, knee scans are acquired from multiple views with stacked 2D slices, ensuring diagnosis accuracy while saving scanning time. However, obtaining fine 3D knee segmentation from multi-view 2D scans is challenging, which is yet necessary for morphological analysis. Moreover, radiologists need to annotate the knee segmentation in multiple 2D scans for medical studies, bringing additional labor. In this paper, we propose the Cross-view Aligned Segmentation Network (CAS-Net) to produce 3D knee segmentation from multi-view 2D MRI scans and annotations of sagittal views only. Specifically, a knee graph representation is firstly built in a 3D isotropic space after the super-resolution of multi-view 2D scans. Then, we utilize a graph-based network to segment individual multi-view patches along the knee surface, and piece together these patch segmentations into a complete knee segmentation with help of the knee graph. Experiments conducted on the Osteoarthritis Initiative (OAI) dataset demonstrate the validity of the CAS-Net to generate accurate 3D segmentation.

Acoustic detection technology has disadvantages, such as short forecasting time in monitoring low-altitude flying targets. Based on the principle that the speed of precursor waves is greater than the speed of sound, this article proposes a complete method for forecasting low-altitude flying targets using precursor seismic waves. First, ray-tracing models of 3-D single-frequency helicopters and broadband-signal aircraft were established based on the attenuation law of precursor wave propagation. Then, the Doppler frequency shift equation of precursor wave is derived and found that the frequency width of precursor wave varies less. In addition, with the increase of distance, the precursor wave correlation detection ability shows a pattern of increasing and then decreasing. Then, an arrival-pickup algorithm is proposed for precursor waves based on multitemporal window determination. Under the experimental conditions, the use of properly developed precursor waves with other trace correlation detection is able to forecast low-altitude flying targets of at least about 6 km. The simulation results illustrate the feasibility and validity of this study.

Although various wired or wireless communication facilities play a significant role in underground mine production monitoring and disaster warning, some communication nodes will be damaged when an accident occurs, resulting in unreliable emergency communications. Due to the excellent transmission characteristics of seismic waves in the earth channel, there is no need for the seismic wave communication system to set up relay nodes. Since seismic waves are not easily disturbed by the external environment, they can be used as a new type of emergency communication carrier to realize information transmission. However, under strong background noise interference in a real environment, the feasibility of seismic wave communication using an electromagnetic vibrator is not yet known. Therefore, a field experiment research on electromagnetic vibrator seismic wave communication was conducted. First, the model of electromagnetic vibrator seismic wave communication was established. In addition, the on-off keying modulation method, the cross-correlation segment threshold demodulation method, and the noise suppression method based on the time-frequency characteristics of the carrier were introduced. Finally, an electromagnetic vibrator seismic wave communication system was built, and binary data communication up to 72 m without deploying additional relay nodes was achieved successfully. The experimental results illustrate the feasibility of the electromagnetic vibrator seismic wave communication under strong background noise interference in a real environment. Furthermore, combined with the characteristics of the seismic wave, the time-frequency slice filtering can significantly improve the signal-to-noise ratio of the seismic wave and can further increase the communication distance with respect to the traditional methods.

Multimodal groupwise registration aligns internal structures in a group of medical images. Current approaches to this problem involve developing similarity measures over the joint intensity profile of all images, which may be computationally prohibitive for large image groups and unstable under various conditions. To tackle these issues, we propose BInGo, a general unsupervised hierarchical Bayesian framework based on deep learning, to learn intrinsic structural representations to measure the similarity of multimodal images. Particularly, a variational auto-encoder with a novel posterior is proposed, which facilitates the disentanglement learning of structural representations and spatial transformations, and characterizes the imaging process from the common structure with shape transition and appearance variation. Notably, BInGo is scalable to learn from small groups, whereas being tested for large-scale groupwise registration, thus significantly reducing computational costs. We compared BInGo with five iterative or deep learning methods on three public intrasubject and intersubject datasets, i.e. BraTS, MS-CMR of the heart, and Learn2Reg abdomen MR-CT, and demonstrated its superior accuracy and computational efficiency, even for very large group sizes (e.g., over 1300 2D images from MS-CMR in each group).

This paper points out a critical issue in the study of estimating the azimuth of ground sources by using the polarization characteristics of Rayleigh waves: even if the signal quality is good, the degree of polarization of Rayleigh waves varies across different frequency bands, and the band with the strongest energy is not the one with the lowest azimuth error. A direction of arrival estimation method for ground sources based on optimally polarized Rayleigh waves using a single three-component geophone is presented in this paper. First, the reciprocal ellipse rate, flatness coefficient, and the angle between the semi-minor axes and the horizontal plane are selected as the polarization parameters of this method according to two quantitative principles. Then the frequency band range of the optimal polarization Rayleigh wave is determined by analyzing the sum of the weights of the three polarization parameters in different frequency bands. After filtering and combining with the existing surface wave analysis method, the actual data bearing estimation result with an average error of only 4.95 degrees and a standard deviation of only 1.82 degrees is obtained. It is also found that the signal-to-noise ratio approximates the exponential decay of the direction of arrival error obtained by this method.

Cross-modality synthesis (CMS) and super-resolution (SR) have both been extensively studied with learning-based methods, which aim to synthesize desired modality images and reduce slice thickness for magnetic resonance imaging (MRI), respectively. It is also desirable to build a network for simultaneous cross-modality and super-resolution (CMSR) so as to further bridge the gap between clinical scenarios and research studies. However, these works are limited to specific fields. None of them can flexibly adapt to various combinations of resolution and modality, and perform CMS, SR, and CMSR with a single network. Moreover, alias frequencies are often treated carelessly in these works, leading to inferior detail-restoration ability. In this paper, we propose Alias-Free Co-Modulated network ( AFCM ) to accomplish all the tasks with a single network design. To this end, we propose to perform CMS and SR consistently with co-modulation, which also provides the flexibility to reduce slice thickness to various, non-integer values for SR. Furthermore, the network is redesigned to be alias-free under the Shannon-Nyquist signal processing framework, ensuring efficient suppression of alias frequencies. Experiments on three datasets demonstrate that AFCM outperforms the alternatives in CMS, SR, and CMSR of MR images. Our codes are available at https://github.com/zhiyuns/AFCM .

Matrix Factorization for collaborative filtering approach has important theoretical significance and application value in large-scale systems and practical applications. In this paper, the traditional Singular value decomposition (SVD) collaborative filter recommendation algorithm is sensitive to the sparseness of the data, the accuracy is not high, and the number of iterations is too long. This paper proposes a gradient descent matrix decomposition cooperation based on SVD Filter recommendation algorithm. Firstly,this algorithm proposes the row mean, column mean, and global mean pre-filling method to pre-fill the original scoring matrix, and then performs SVD decomposition to reduce the impact of data sparsity on the recommendation accuracy. Second, the matrix after SVD decomposition the relationship with the original score is used to remodel the prediction score to improve the accuracy of the final prediction score. Finally,an optimized gradient descent(OGD) method is proposed. Specifically,the pre-filled improved SVD decomposition matrix is used as the initial value of gradient descent to perform gradient descent, thereby reducing the number of iterations of gradient descent. Compared with the traditional SVD-based gradient descent matrix factorization collaborative filtering recommendation algorithm, the proposed algorithm can effectively recommend accuracy and speed up the prediction score,data sparsity.

This work is a technical report concerning the Large Scale Vertebrae Segmentation Challenge (VerSe) organised in conjunction with the MICCAI 2019. The challenge set-up consisting of two tasks, vertebrae labelling and vertebrae segmentation, is detailed. A total of 160 multidetector CT scans closely resembling a typical spine-centreed clinical setting were prepared and annotated at voxel-level by a human-machine hybrid algorithm. Both the annotation protocol and the algorithm that aided the medical experts in this annotation process are presented. More importantly, eleven fully automated algorithms of the participating teams were submitted to be benchmarked on the VerSe data. This work presents a detailed performance analysis of these algorithms with the best performing algorithm achieving a vertebrae identification rate of 95% and a Dice coefficient of 90%. VerSe'19 is an open-call challenge and its image data along with the annotations and evaluation tools will continue to be publicly accessible through its online portal.