We introduce MedIL, a first-of-its-kind autoencoder built for medical images with heterogeneous sizes and resolutions for image generation. Medical images are often large and heterogeneous, where fine details are of vital clinical importance. Image properties change drastically between different acquisition equipment, scan parameters, and pathologies, making realistic medical image generation challenging. Recent work in latent diffusion models (LDMs) has shown success in generating images resampled to a fixed size. However, this is a narrow subset of the native image resolutions, and resampling discards fine anatomical details. MedIL utilizes implicit neural representations to treat images as continuous signals, where encoding and decoding can be performed at arbitrary resolutions without prior resampling. We show how MedIL compresses and preserves fine details over large and multi-resolution datasets of T1w brain MRIs and lung CTs. We further demonstrate how MedIL improves the quality of images generated with a diffusion model and discuss how generative models may be improved to resemble raw clinical acquisitions.

In this work, we introduce a new approach to model group actions in autoencoders. Diverging from prior research in this domain, we propose to learn the group actions on the latent space rather than strictly on the data space. This adaptation enhances the versatility of our model, enabling it to learn a broader range of scenarios prevalent in the real world, where groups can act on latent factors. Our method allows a wide flexibility in the encoder and decoder architectures and does not require group-specific layers. In addition, we show that our model theoretically serves as a superset of methods that learn group actions on the data space. We test our approach on five image datasets with diverse groups acting on them and demonstrate superior performance to recently proposed methods for modeling group actions.

Environmental enteric dysfunction (EED) is a subclinical enteropathy challenging to diagnose due to an overlap of tissue features with other inflammatory enteropathies. EED subjects (n=52) from Pakistan, controls (n=25), and a validation EED cohort (n=30) from Zambia were used to develop a machine-learning-based image analysis classification model. We extracted histologic feature representations from the Pakistan EED model and correlated them to transcriptomics and clinical biomarkers. In-silico metabolic network modeling was used to characterize alterations in metabolic flux between EED and controls and validated using untargeted lipidomics. Genes encoding beta-ureidopropionase, CYP4F3, and epoxide hydrolase 1 correlated to numerous tissue feature representations. Fatty acid and glycerophospholipid metabolism-related reactions showed altered flux. Increased phosphatidylcholine, lysophosphatidylcholine (LPC), and ether-linked LPCs, and decreased ester-linked LPCs were observed in the duodenal lipidome of Pakistan EED subjects, while plasma levels of glycine-conjugated bile acids were significantly increased. Together, these findings elucidate a multi-omic signature of EED.

Crohn's disease (CD) is a chronic inflammatory disease of the gastrointestinal tract. A clear gap in our existing CD diagnostics and current disease management approaches is the lack of highly specific biomarkers that can be used to streamline or personalize disease management. Comprehensive profiling of metabolites holds promise; however, these high-dimensional profiles need to be reduced to have relevance in the context of CD. Machine learning approaches are optimally suited to bridge this gap in knowledge by contextualizing the metabolic alterations in CD using genome-scale metabolic network reconstructions. Our work presents a framework for studying altered metabolic reactions between patients with CD and controls using publicly available transcriptomic data and existing gene-driven metabolic network reconstructions. Additionally, we apply the same methods to patient-derived ileal enteroids to explore the utility of using this experimental in vitro platform for studying CD. Furthermore, we have piloted an untargeted metabolomics approach as a proof-of-concept validation strategy in human ileal mucosal tissue. These findings suggest that in silico metabolic modeling can potentially identify pathways of clinical relevance in CD, paving the way for the future discovery of novel diagnostic biomarkers and therapeutic targets.

In this work, we propose Mutual Information Maximization Knowledge Distillation (MIMKD). Our method uses a contrastive objective to simultaneously estimate and maximize a lower bound on the mutual information of local and global feature representations between a teacher and a student network. We demonstrate through extensive experiments that this can be used to improve the performance of low capacity models by transferring knowledge from more performant but computationally expensive models. This can be used to produce better models that can be run on devices with low computational resources. Our method is flexible, we can distill knowledge from teachers with arbitrary network architectures to arbitrary student networks. Our empirical results show that MIMKD outperforms competing approaches across a wide range of student-teacher pairs with different capacities, with different architectures, and when student networks are with extremely low capacity. We are able to obtain 74.55% accuracy on CIFAR100 with a ShufflenetV2 from a baseline accuracy of 69.8% by distilling knowledge from ResNet-50. On Imagenet we improve a ResNet-18 network from 68.88% to 70.32% accuracy (1.44%+) using a ResNet-34 teacher network.

Skin disorders are more frequent than other types of illnesses. The key necessity for pigmentation lesion analysis is automated intelligence systems for impartial diagnosis. It has piqued the interest of scientists in recent decades. These systems go through several stages, including pre-processing, feature recognition, classification, and post processing. It is critical to appropriately diagnose and divide the skin lesion. It has been found that latest developments in ML algorithms & dermoscopy images techniques lowered the errors in predictions, and so the emphasis on computer assisted systems has expanded significantly in recent years. Although computer-aided diagnostic systems are a credible source for dermatologists to assess the kind of cancer, it is commonly known that even more precision is required for computer assisted systems to be used realistically in the diagnosis of fatal illnesses. This suggested approach may notify clinicians and lab technicians of the 3 most probable diagnoses for a specific skin lesion. It might benefit them in promptly identifying high risk patients and speeding up their operations. The program will deliver a response in under three seconds. To preserve user privacy, pictures provided by users are pre-processed and evaluated locally before being posted to an open network. Artificial Intelligence drives this app. Users may upload a snapshot of a skin infection and obtain an immediate prognosis.

In recent years, computational pathology has seen tremendous progress driven by deep learning methods in segmentation and classification tasks aiding prognostic and diagnostic settings. Nuclei segmentation, for instance, is an important task for diagnosing different cancers. However, training deep learning models for nuclei segmentation requires large amounts of annotated data, which is expensive to collect and label. This necessitates explorations into generative modeling of histopathological images. In this work, we use recent advances in conditional diffusion modeling to formulate a first-of-its-kind nuclei-aware semantic tissue generation framework (NASDM) which can synthesize realistic tissue samples given a semantic instance mask of up to six different nuclei types, enabling pixel-perfect nuclei localization in generated samples. These synthetic images are useful in applications in pathology pedagogy, validation of models, and supplementation of existing nuclei segmentation datasets. We demonstrate that NASDM is able to synthesize high-quality histopathology images of the colon with superior quality and semantic controllability over existing generative methods. Implementation: https://github.com/4m4n5/NASDM .

Recent NLP literature has seen growing interest in improving model interpretability. Along this direction, we propose a trainable neural network layer that learns a global interaction graph between words and then selects more informative words using the learned word interactions. Our layer, we call WIGRAPH, can plug into any neural network-based NLP text classifiers right after its word embedding layer. Across multiple SOTA NLP models and various NLP datasets, we demonstrate that adding the WIGRAPH layer substantially improves NLP models' interpretability and enhances models' prediction performance at the same time.

We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations. Compared to the previously proposed CLIP model, CLIP-Lite requires only one negative image-text sample pair for every positive image-text sample during the optimization of its contrastive learning objective. We accomplish this by taking advantage of an information efficient lower-bound to maximize the mutual information between the two input modalities. This allows CLIP-Lite to be trained with significantly reduced amounts of data and batch sizes while obtaining better performance than CLIP at the same scale. We evaluate CLIP-Lite by pretraining on the COCO-Captions dataset and testing transfer learning to other datasets. CLIP-Lite obtains a +14.0% mAP absolute gain in performance on Pascal VOC classification, and a +22.1% top-1 accuracy gain on ImageNet, while being comparable or superior to other, more complex, text-supervised models. CLIP-Lite is also superior to CLIP on image and text retrieval, zero-shot classification, and visual grounding. Finally, we show that CLIP-Lite can leverage language semantics to encourage bias-free visual representations that can be used in downstream tasks. Implementation: https://github.com/4m4n5/CLIP-Lite

The relationship between environmental factors and child health is not well understood in rural Pakistan. This study characterized the environmental factors related to the morbidity of acute respiratory infections (ARIs), diarrhea, and growth using geographical information systems (GIS) technology. Anthropometric, address and disease prevalence data were collected through the SEEM (Study of Environmental Enteropathy and Malnutrition) study in Matiari, Pakistan. Publicly available map data were used to compile coordinates of healthcare facilities. A Pearson correlation coefficient (r) was used to calculate the correlation between distance from healthcare facilities and participant growth and morbidity. Other continuous variables influencing these outcomes were analyzed using a random forest regression model. In this study of 416 children, we found that participants living closer to secondary hospitals had a lower prevalence of ARI (r = 0.154, p < 0.010) and diarrhea (r = 0.228, p < 0.001) as well as participants living closer to Maternal Health Centers (MHCs): ARI (r = 0.185, p < 0.002) and diarrhea (r = 0.223, p < 0.001) compared to those living near primary facilities. Our random forest model showed that distance has high variable importance in the context of disease prevalence. Our results indicated that participants closer to more basic healthcare facilities reported a higher prevalence of both diarrhea and ARI than those near more urban facilities, highlighting potential public policy gaps in ameliorating rural health.

Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utilized for biopsy visualization-based diagnostic and prognostic assessment of diseases. Variation in the H&E staining process across different lab sites can lead to significant variations in biopsy image appearance. These variations introduce an undesirable bias when the slides are examined by pathologists or used for training deep learning models. To reduce this bias, slides need to be translated to a common domain of stain appearance before analysis. We propose a Self-Attentive Adversarial Stain Normalization (SAASN) approach for the normalization of multiple stain appearances to a common domain. This unsupervised generative adversarial approach includes self-attention mechanism for synthesizing images with finer detail while preserving the structural consistency of the biopsy features during translation. SAASN demonstrates consistent and superior performance compared to other popular stain normalization techniques on H&E stained duodenal biopsy image data.

Candidate markers for Crohn’s Disease (CD) may be identified via gene expression-based construction of metabolic networks (MN). These can computationally describe gene-protein-reaction associations for entire tissues and also predict the flux of reactions (rate of turnover of specific molecules via a metabolic pathway). Recon3D is the most comprehensive human MN to date. We used publicly available CD transcriptomic data along with Recon3D to identify metabolites as potential diagnostic and prognostic biomarkers. Terminal ileal gene expression profiles (36,372 genes; 218 CD. 42 controls) from the RISK cohort (Risk Stratification and Identification of Immunogenetic and Microbial Markers of Rapid Disease Progression in Children with Crohn’s Disease) and their transcriptomic abundances were used. Recon3D was pruned to only include RISK dataset transcripts which determined metabolic reaction linkage with transcriptionally active genes. Flux balance analysis (FBA) was then run using RiPTiDe with context specific transcriptomic data to further constrain genes (Figure 1). RiPTiDe was independently run on transcriptomic data from both CD and controls. From the pruned and constricted MN obtained, reactions were extracted for further analysis. After applying the necessary constraints to modify Recon3D, 527 CD and 537 control reactions were obtained. Reaction comparison with a publicly available list of healthy small intestinal epithelial reactions (n=1282) showed an overlap of 80 CD and 84 control reactions. These were then further grouped based on their metabolic pathways. RiPTiDe identified context specific metabolic pathway activity without supervision and the percentage of forward, backward, and balanced reactions for each metabolic pathway (Figure 2). The metabolite concentrations in the small intestine was altered among CD patients. Notably, the citric acid cycle and malate-aspartate shuttle were affected, highlighting changes in mitochondrial metabolic pathways. This is illustrated by changes in the number of reactions at equilibrium between CD and control. The results are relevant as cytosolic acetyl-CoA is needed for fatty acid synthesis and is obtained by removing citrate from the citric acid cycle. An intermediate removal from the cycle has significant cataplerotic effects. The malate-aspartate shuttle also allows electrons to move across the impermeable membrane in the mitochondria (fatty acid synthesis location). These findings are reported by previously published studies where gene expression for fatty acid synthesis is altered in CD patients along with mitochondrial metabolic pathway changes, resulting in altered cell homeostasis. In-depth analysis is currently underway with our work supporting the utility of potential metabolic biomarkers for CD diagnosis, management and improved care.

We propose CLIP-Lite, an information efficient method for visual representation learning by feature alignment with textual annotations. Compared to the previously proposed CLIP model, CLIP-Lite requires only one negative image-text sample pair for every positive image-text sample during the optimization of its contrastive learning objective. We accomplish this by taking advantage of an information efficient lower-bound to maximize the mutual information between the two input modalities. This allows CLIP-Lite to be trained with significantly reduced amounts of data and batch sizes while obtaining better performance than CLIP. We evaluate CLIP-Lite by pretraining on the COCO-Captions dataset and testing transfer learning to other datasets. CLIP-Lite obtains a +15.4% mAP absolute gain in performance on Pascal VOC classification, and a +22.1% top-1 accuracy gain on ImageNet, while being comparable or superior to other, more complex, text-supervised models. CLIP-Lite is also superior to CLIP on image and text retrieval, zero-shot classification, and visual grounding. Finally, by performing explicit image-text alignment during representation learning, we show that CLIP-Lite can leverage language semantics to encourage bias-free visual representations that can be used in downstream tasks.

In recent years, the availability of digitized Whole Slide Images (WSIs) has enabled the use of deep learning-based computer vision techniques for automated disease diagnosis. However, WSIs present unique computational and algorithmic challenges. WSIs are gigapixel-sized ($\sim$100K pixels), making them infeasible to be used directly for training deep neural networks. Also, often only slide-level labels are available for training as detailed annotations are tedious and can be time-consuming for experts. Approaches using multiple-instance learning (MIL) frameworks have been shown to overcome these challenges. Current state-of-the-art approaches divide the learning framework into two decoupled parts: a convolutional neural network (CNN) for encoding the patches followed by an independent aggregation approach for slide-level prediction. In this approach, the aggregation step has no bearing on the representations learned by the CNN encoder. We have proposed an end-to-end framework that clusters the patches from a WSI into ${k}$-groups, samples ${k}'$ patches from each group for training, and uses an adaptive attention mechanism for slide level prediction; Cluster-to-Conquer (C2C). We have demonstrated that dividing a WSI into clusters can improve the model training by exposing it to diverse discriminative features extracted from the patches. We regularized the clustering mechanism by introducing a KL-divergence loss between the attention weights of patches in a cluster and the uniform distribution. The framework is optimized end-to-end on slide-level cross-entropy, patch-level cross-entropy, and KL-divergence loss (Implementation: https://github.com/YashSharma/C2C).

OBJECTIVES:Striking histopathological overlap between distinct but related conditions poses a disease diagnostic challenge. There is a major clinical need to develop computational methods enabling clinicians to translate heterogeneous biomedical images into accurate and quantitative diagnostics. This need is particularly salient with small bowel enteropathies; environmental enteropathy (EE) and celiac disease (CD). We built upon our preliminary analysis by developing an artificial intelligence (AI)-based image analysis platform utilizing deep learning convolutional neural networks (CNNs) for these enteropathies. METHODS:Data for the secondary analysis was obtained from three primary studies at different sites. The image analysis platform for EE and CD was developed using CNNs including one with multizoom architecture. Gradient-weighted class activation mappings (Grad-CAMs) were used to visualize the models' decision-making process for classifying each disease. A team of medical experts simultaneously reviewed the stain color normalized images done for bias reduction and Grad-CAMs to confirm structural preservation and biomedical relevance, respectively. RESULTS:Four hundred and sixty-one high-resolution biopsy images from 150 children were acquired. Median age (interquartile range) was 37.5 (19.0-121.5) months with a roughly equal sex distribution; 77 males (51.3%). ResNet50 and shallow CNN demonstrated 98% and 96% case-detection accuracy, respectively, which increased to 98.3% with an ensemble. Grad-CAMs demonstrated models' ability to learn different microscopic morphological features for EE, CD, and controls. CONCLUSIONS:Our AI-based image analysis platform demonstrated high classification accuracy for small bowel enteropathies which was capable of identifying biologically relevant microscopic features and emulating human pathologist decision-making process. Grad-CAMs illuminated the otherwise "black box" of deep learning in medicine, allowing for increased physician confidence in adopting these new technologies in clinical practice.

Infectious disease is the leading cause of mortality in children under five. This study has investigated environmental factors related to the morbidity of acute respiratory infections (ARIs), diarrhea, and growth using geographical information systems (GIS) technology. Anthropometric, address and disease prevalence data were collected through the SEEM study in Matiari, Pakistan. Publicly available map data was used to compile coordinates of healthcare facilities. A Pearson correlation coefficient (r) was used to calculate the correlation between distance from healthcare facilities and participant growth and morbidity. Other continuous variables influencing these outcomes were analyzed using a random forest regression model. In this study of 416 children, we found participants living closer to secondary hospitals had lower prevalence of ARI (r=0.154, p&lt;0.010) and diarrhea (r=0.228, p&lt;0.001) as well as participants living closer to Maternal Health Centers (MHCs): ARI (r=0.185, p&lt;0.002) and diarrhea (r=0.223, p&lt;0.001) compared to those living near primary facilities. Our random forest model showed distance to have high variable importance in the context of disease prevalence. Our results indicated that participants closer to more basic healthcare facilities reported a higher prevalence of both diarrhea and ARI than those near more urban facilities, highlighting potential public policy gaps in ameliorating rural health.

Artificial intelligence (AI), a discipline encompassed by data science, has seen recent rapid growth in its application to healthcare and beyond, and is now an integral part of daily life. Uses of AI in gastroenterology include the automated detection of disease and differentiation of pathology subtypes and disease severity. Although a majority of AI research in gastroenterology focuses on adult applications, there are a number of pediatric pathologies that could benefit from more research. As new and improved diagnostic tools become available and more information is retrieved from them, AI could provide physicians a method to distill enormous amounts of data into enhanced decision-making and cost saving for children with digestive disorders. This review provides a broad overview of AI and examples of its possible applications in pediatric gastroenterology.

Physicians use biopsies to distinguish between different but histologically similar enteropathies. The range of syndromes and pathologies that could cause different gastrointestinal conditions makes this a difficult problem. Recently, deep learning has been used successfully in helping diagnose cancerous tissues in histopathological images. These successes motivated the research presented in this paper, which describes a deep learning approach that distinguishes between Celiac Disease (CD) and Environmental Enteropathy (EE) and normal tissue from digitized duodenal biopsies. Experimental results show accuracies of over 90% for this approach. We also look into interpreting the neural network model using Gradient-weighted Class Activation Mappings and filter activations on input images to understand the visual explanations for the decisions made by the model.