There is a growing consensus in neuroscience that pathological proteins accumulate and spread along specific large-scale brain networks, indicating the mechanistic role of connectome architecture in the progression of neurodegenerative diseases. Although mounting evidence shows that pathology spreading is a dynamic biological process shaped by the complex interplay between the wiring mechanism of neuronal fibers and the self-organized synchronization of functional fluctuations, current computational methods model the propagation of pathology burden through either structural connectivity (SC) or functional connectivity (FC). To address this limitation, we present a multilayer transport model to capture the SC/FC-specific propagation of neuropathological burdens and their interactions from longitudinal imaging data. Furthermore, we propose to parameterize the spreading pathways using a physics-informed neural network, enabling the prediction of the progression of pathological events at the baseline. We have evaluated the prediction accuracy of tau aggregates in Alzheimer's disease (AD), where our method achieves a significantly higher accuracy compared to existing approaches. In addition, the physics principle in our deep model allows us to explore the biological underpinning of how SC-FC interaction contributes to pathology propagation in AD. Taken together, enhanced prediction accuracy and model interpretability suggest the great potential of our deep model in uncovering the pathophysiological mechanism in neurodegenerative diseases through data-driven approaches.
Deep learning has achieved significant success in diffusion MRI (dMRI) computing. However, most deep learning-based dMRI methods heavily rely on supervised learning, which requires large-scale labeled datasets that are difficult to obtain. Recently, Masked AutoEncoder (MAE) has gained popularity in computer vision for its ability to leverage unlabeled data, yet its potential has not yet been fully explored in dMRI. To fill this gap, we propose the first MAE-based selfsupervised learning framework, called DMAE, for representation learning of dMRI data. In DMAE, we first create dMRI patches using a deliberately designed tube q-space masking strategy, which adapts well to the unique q-space characteristics of dMRI data. We then encode the patches into a latent space for pre-training to learn high-level semantic representations. During the fine-tuning stage, we design a task-specific decoder and incorporate the decoder with the pre-trained encoder to achieve superior performance in downstream tasks. Extensive quantitative and qualitative results demonstrate the superiority of our framework over the state-of-the-art self-supervised learning approaches.
Computed Tomography Report Generation (CTRG) aims to automate the clinical radiology reporting process, thereby reducing the workload of report writing and facilitating patient care. While deep learning approaches have achieved remarkable advances in X-ray report generation, their effectiveness may be limited in CTRG due to larger data volumes of CT images and more intricate details required to describe them. This work introduces a novel two-stage (structureand report-learning) framework tailored for CTRG featuring effective structure-wise image-text contrasting. In the first stage, a set of learnable structure-specific visual queries "observe" corresponding structures in a CT image. The resulting observation tokens are contrasted with structure-specific textual features extracted from the accompanying radiology report with a structure-wise image-text contrastive loss. In addition, text-text similarity-based soft pseudo targets are proposed to mitigate the impact of false negatives, i.e., semantically identical image structures and texts from non-paired images and reports. Thus, the model learns structure-level semantic correspondences between CT images and reports. Further, a dynamic, diversity-enhanced negative queue is proposed to guide the network in learning to discriminate various abnormalities. In the second stage, the visual structure queries are frozen and used to select the critical image patch embeddings depicting each anatomical structure, minimizing distractions from irrelevant areas while reducing memory consumption. Also, a text decoder is added and trained for report generation. Our extensive experiments on two public datasets demonstrate that our framework establishes new state-of-the-art performance for CTRG in clinical efficiency, and its components are effective.
In light of the inherent entailment relations between images and text, hyperbolic point vector embeddings, which leverage the hierarchical structure of hyperbolic space, have been used for visual-semantic representation, showing significant advantages in zero-shot learning tasks. However, unlike general image-text alignment tasks, the availability of high-quality paired image-text data in the medical domain is limited. This scarcity presents challenges for visual language models, hindering their ability to effectively comprehend free-text medical reports and the associated images. Moreover, many medical terms are complex, specialized, and abstract, and embeddings derived solely from raw imaging report texts often fail to generalize effectively. To address these challenges, we propose MkCLIPH, a hyperbolic space image-text alignment pre-training method that incorporates medical domain knowledge. MkCLIPH models the visual-semantic hierarchical partial order relationship through hierarchical entailment angle modeling and integrates medical domain knowledge as a prior to enhance the representation of medical image-text data. This improves generalization and interpretability. Experimental results demonstrate that our method outperforms baseline approaches in terms of interpretability and performance across a range of zero-shot tasks and datasets.
Pixel and voxel-basis are commonly used for designing Xray CT projectors that underly existing projection and back-projection methods that are widely used in practice. We introduce a bilinear projector for modeling X-ray optics and provide experimental evidence showing its impact in reducing discretization artifacts in Model-Based Iterative Reconstruction (MBIR). We demonstrate the advantages of using this bilinear model over the conventional pixel-based projectors (piecewise constant) for MBIR in realistic simulations in presence of nonlinearity of the Lambert-Beer law. We derive efficient piecewise polynomial forms for footprint and detector blur computations for this new projector and demonstrate its efficient GPU implementation for high resolution fanbeam geometry.
