Cell-cell proximity influences tissue homeostasis and disease progression, yet robust quantification across varying cell abundances remains challenging. We introduce a Monte Carlo simulation framework using the G-function as a spatial randomness reference to detect proximity differences between case groups independent of cell count. Three metrics, G-area, G-difference, and G-ratio, were evaluated for summarizing G-function outputs, alongside established approaches such as the Morisita-Horn Index and likelihood ratio. G-area most accurately captured group-level proximity changes. To demonstrate generalizability, we validated G-area in two external multiplex imaging datasets from colorectal and prostate cancer. This framework provides a cell-count-robust method for spatial analysis, enabling more reliable detection of microenvironmental changes across diseases and imaging platforms.

In renal histopathology, the routine clinical use of several histological stains presents challenges for the direct application of stain-specific deep learning-based analysis tools to whole-slide images. We present an approach to the in silico histological staining of kidney tissue where samples stained with hematoxylin and eosin (H&E) are virtually restained with periodic acid-Schiff (PAS). Our approach is underpinned by cycle-consistent generative adversarial neural networks trained on the National Unified Renal Translational Research Enterprise data set-the first UK-wide Biobank for chronic kidney disease-which features diverse data from 16 nephrology centers. Our work is divided into the following 4 main components: (1) we developed a virtual staining model, which infers PAS staining from H&E; (2) 2 board-certified pathologists assessed the virtual staining by attempting to distinguish it from real examples; (3) we trained a glomerular segmentation model using 3 independent renal segmentation data sets (Kidney Precision Medicine Project, Human BioMolecular Atlas Program [Kidney], and data by Jayapandian et al); and (4) we demonstrated the utility of virtual staining by inferring PAS staining from previously unseen H&E test images and applying our PAS-specific glomerular segmentation model. Each pathologist was able to identify 52.5% and 75.8% of the virtually stained images, respectively, showing an overlap in the variability of the authentic and synthetic staining. We discussed the utility of virtual staining in digital pathology, the need for pathology-specific testing with respect to chronic damage, and minimal changes and steps for incorporating more stains. Furthermore, alongside this article, we included complete glomerular annotations for 20 Kidney Precision Medicine Project H&E-stained slides.

Accurate segmentation of glomerulus instances attains high clinical significance in the automated analysis of renal biopsies to aid in diagnosing and monitoring kidney disease. Analyzing real-world histopathology images often encompasses inter-observer variability and requires a labor-intensive process of data annotation. Therefore, conventional supervised learning approaches generally achieve sub-optimal performance when applied to external datasets. Considering these challenges, we present a semi-supervised learning approach for glomeruli segmentation based on the weak-to-strong consistency framework validated on multiple real-world datasets. Our experimental results on 3 independent datasets indicate superior performance of our approach as compared with existing supervised baseline models such as U-Net and SegFormer.

Abstract Background Interleukin23 (IL-23) is a cytokine that plays a crucial role in the pathogenesis of inflammatory bowel disease (IBD), making it a highly validated therapeutic target. Understanding the role of IL-23 in IBD at the histopathological level is crucial for determining effective treatment strategies, providing insights into IBD patients who fail to respond to targeted therapies, or predicting those who are likely to lose response. In this context, there is a surge in utilizing artificial intelligence (AI) for histopathological data in IBD and other disease indications. Here, we present an automated computer vision approach to predict IL-23 signalling activity directly from routinely stained Hematoxylin and Eosin (H&E) images Methods A total of 1502 samples with matched clinical data and H&E biopsy images were included from 991 Crohn’s disease (CD) and 511 ulcerative colitis (UC) samples. IL-23 signalling activity was calculated using gene set variation analysis on RNA-seq data collected from the same tissue biopsies. The data were obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation. The proposed approach is based on vision transformers (ViTs) which is a type of deep learning model. ViTs divide the input image into fixed-size patches, transform it into linear embedding, and analyze it with the self-attention mechanism. This enables ViTs to incorporate relationships between different patches of the input image to identify regions predictive of IL-23. Our approach was trained in a weakly supervised manner to automatically identify tissue regions that correlate with IL-23 signaling activity. The model produces interpretable heatmaps to interrogate model predictions and allow clinicians to visualize and interpret the significance of different tissue regions predictive of IL-23 Results We performed 5-fold cross-validation on the splits obtained at the patient level, retaining the data distribution of IL-23 signaling activity, biopsy location, and diagnosis. We separately validated the performance of the proposed model on both disease categories, including CD and UC. The proposed approach achieved an area under the curve (AUC) of 0.82 ± 0.04 on unseen data from CD and an AUC of 0.80 ± 0.02 for UC. The 5-fold results for both disease categories are shown below Conclusion The presented results highlight the significance of computational pathology algorithms to identify IL-23 signalling activity from H&E images. Pathological interpretation from the heatmaps may help understand disease pathomechanism and optimize the treatment options for IBD patients by timely identification of IL-23 status. We are further validating the clinical utility of such heatmaps and expanding the use of H&E to predict other patient-centric endpoints

Abstract Background Histopathological endpoints are evolving as a treatment target in Inflammatory Bowel Disease (IBD). Use of histology to screen entrants could add value in IBD clinical trials; for example, by refining eligibility criteria to ensure studies recruit patients with definitive active inflammation at the microscopic level. Several histopathological indices have been developed, but the relative complexity of available scores hinders the development of an AI algorithm without large-scale labour-intensive annotation by a pathologist. We aimed to develop computer vision tools to assist decoding the complex clinical disease features at the histological level for both Crohn’s Disease (CD) and Ulcerative Colitis (UC). This will inform understanding of disease pathology and patient stratification to support clinical trial development strategies. Methods A total of 1397 clinically annotated Haematoxylin & Eosin (H&E) images were included from 418 CD and 218 UC patients enrolled in a multicentred longitudinal Study of a Prospective Adult Research Cohort with IBD (SPARC IBD) obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation. We developed an image quality control (QC) algorithm to automatically identify image and tissue processing/staining artefacts negatively impacting analysis (e.g., out-of-focus, tissue folds, overstained regions) and excluded these regions (Fig. 1). Next, a self-supervised learning (SSL) deep learning computer vison model was developed and trained to predict disease relevant features including disease diagnosis and lesional macroscopic appearance (inflammation, erosions and ulcers). Finally, to better understand the model’s predictions, we generated heatmap overlays on the tissue that show regions which the model considers to be most predictive (Fig. 2) and shared these with pathologists for qualitative evaluation. Results We find that the SSL model performs well on different downstream classification tasks such as UC vs CD (area under curve (AUC) = 0.79) and normal vs lesional tissue (AUC = 0.76). Specialist pathologist collaboration further confirmed that the heatmap overlays identified clinically relevant tissue features, including inflammatory cell infiltrates (Fig. 2). Conclusion These encouraging results support further exploration of this deep-learning algorithm to distinguish disease specific characteristics in this set of images from CD and UC patients. Further work is ongoing to validate the heatmap approach on endoscopic scores, we also plan to validate our model on IBD clinical trial datasets.

Crohn's Disease (CD) and Ulcerative Colitis (UC) are the two main Inflammatory Bowel Disease (IBD) types. We developed deep learning models to identify histological disease features for both CD and UC using only endoscopic labels. We explored fine-tuning and end-to-end training of two state-of-the-art self-supervised models for predicting three different endoscopic categories (i) CD vs UC (AUC=0.87), (ii) normal vs lesional (AUC=0.81), (iii) low vs high disease severity score (AUC=0.80). We produced visual attention maps to interpret what the models learned and validated them with the support of a pathologist, where we observed a strong association between the models' predictions and histopathological inflammatory features of the disease. Additionally, we identified several cases where the model incorrectly predicted normal samples as lesional but were correct on the microscopic level when reviewed by the pathologist. This tendency of histological presentation to be more severe than endoscopic presentation was previously published in the literature. In parallel, we utilised a model trained on the Colon Nuclei Identification and Counting (CoNIC) dataset to predict and explore 6 cell populations. We observed correlation between areas enriched with the predicted immune cells in biopsies and the pathologist's feedback on the attention maps. Finally, we identified several cell level features indicative of disease severity in CD and UC. These models can enhance our understanding about the pathology behind IBD and can shape our strategies for patient stratification in clinical trials.

Ductal Carcinoma In Situ (DCIS) is a non-obligatory precursor of Invasive Breast Cancer. It is the most common mammographically detected breast cancer. Predicting DCIS progression to invasive ductal carcinoma is a major clinical challenge due to the lack of a uniform classification system in the diagnosis and prognostication of this disease. To characterise the tissue microecology of DCIS, we proposed and tested the model "DCIS-Identification model" based on Generative Adversarial Networks (GAN) for detection and segmentation of DCIS ducts from multiplex immunohistochemistry (IHC) staining samples. We also trained a Spatially Constrained Convolutional Neural Network (SC-CNN) to detect and classify single cells based on their CA9 and FOXP3 expression. The DCIS-Identification model was evaluated on 8 whole slide images, resulting in an average Dice score of 0.95 for the segmentation performance. The single cell identification framework was tested on 10 randomly selected whole slide sections, achieving the average accuracy of 88.6% in a 5 fold cross validation scheme. With the proposed pipeline, we efficiently integrated deep learning, computational pathology and spatial statistics to report distinct differences in the microenvironments of DCIS and IDC/DCIS samples. The proposed pipeline provides a tool for a better understanding of the mechanism of tumours in DCIS and IDC/DCIS cases.

Hypoxia promotes aggressive tumor phenotypes and mediates the recruitment of suppressive T cells in invasive breast carcinomas. We investigated the role of hypoxia in relation to T-cell regulation in ductal carcinoma in situ (DCIS). We designed a deep learning system tailored for the tissue architecture complexity of DCIS, and compared pure DCIS cases with the synchronous DCIS and invasive components within invasive ductal carcinoma cases. Single-cell classification was applied in tandem with a new method for DCIS ductal segmentation in dual-stained CA9 and FOXP3, whole-tumor section digital pathology images. Pure DCIS typically has an intermediate level of colocalization of FOXP3+ and CA9+ cells, but in invasive carcinoma cases, the FOXP3+ (T-regulatory) cells may have relocated from the DCIS and into the invasive parts of the tumor, leading to high levels of colocalization in the invasive parts but low levels in the synchronous DCIS component. This may be due to invasive, hypoxic tumors evolving to recruit T-regulatory cells in order to evade immune predation. Our data support the notion that hypoxia promotes immune tolerance through recruitment of T-regulatory cells, and furthermore indicate a spatial pattern of relocalization of T-regulatory cells from DCIS to hypoxic tumor cells. Spatial colocalization of hypoxic and T-regulatory cells may be a key event and useful marker of DCIS progression.

Gliomas are brain tumours with a high mortality rate. There are various grades and sub-types of this tumour, and the treatment procedure varies accordingly. Clinicians and oncologists diagnose and categorise these tumours based on visual inspection of radiology and histology data. However, this process can be time-consuming and subjective. The computer-assisted methods can help clinicians to make better and faster decisions. In this paper, we propose a pipeline for automatic classification of gliomas into three sub-types: oligodendroglioma, astrocytoma, and glioblastoma, using both radiology and histopathology images. The proposed approach implements distinct classification models for radiographic and histologic modalities and combines them through an ensemble method. The classification algorithm initially carries out tile-level (for histology) and slice-level (for radiology) classification via a deep learning method, then tile/slice-level latent features are combined for a whole-slide and whole-volume sub-type prediction. The classification algorithm was evaluated using the data set provided in the CPM-RadPath 2020 challenge. The proposed pipeline achieved the F1-Score of 0.886, Cohen's Kappa score of 0.811 and Balance accuracy of 0.860. The ability of the proposed model for end-to-end learning of diverse features enables it to give a comparable prediction of glioma tumour sub-types.

How people walk often reveals key insights into health, quality of life and independence. Here, we propose a smartphone-based gait monitoring system which is sensitive and accurate enough to measure temporal gait parameters during unsteady walking, differentiate between normal and impaired gait, and recognise changes in the impaired gait depending on the use of medication or walking aid.

The field of immuno-oncology has expanded rapidly over the past decade, but key questions remain. How does tumour-immune interaction regulate disease progression? How can we prospectively identify patients who will benefit from immunotherapy? Identifying measurable features of the tumour immune-microenvironment which have prognostic or predictive value will be key to making meaningful gains in these areas. Recent developments in deep learning enable big-data analysis of pathological samples. Digital approaches allow data to be acquired, integrated and analysed far beyond what is possible with conventional techniques, and to do so efficiently and at scale. This has the potential to reshape what can be achieved in terms of volume, precision and reliability of output, enabling data for large cohorts to be summarised and compared. This review examines applications of artificial intelligence (AI) to important questions in immuno-oncology (IO). We discuss general considerations that need to be taken into account before AI can be applied in any clinical setting. We describe AI methods that have been applied to the field of IO to date and present several examples of their use.

Tumour cells require resources to survive and proliferate. In order to be provided with a supportive micro-environment rich with resources to sustain optimal growth, tumour cells tend to reside in close proximity to a network of blood vessels. Quantification of blood microvessel density can be a useful measure to investigate the importance of resource limitation in tumours for prognostication and assigning treatment and mode of drug delivery. Currently, immunohistochemistry (IHC) with specific antibodies and the subsequent detection of its binding in the tumour tissue are used to identify microvessels. The automated quantification of blood microvessels in Hematoxylin and Eosin (H&E) stained images is not widely studied because microvessels are very complex and heterogeneous. In addition, their manual identification is tedious, time-consuming and subjective. We investigate whether the vasculature in H&E can be robustly identified in whole slide sections that would ultimately avoid the need for IHC and manual annotations. We propose an artificial intelligence model based on Generative Adversarial Networks (GAN) that, from an input H&E image, can generate a synthetic Erythroblast Transformation specific related gene (ERG) stained image, highlighting vessel structures. We also trained a spatially constrained Convolutional Neural Network (CNN) to identify single cells on ERG stained whole slide images, and found good concordance between detected cells in synthetic and real ERG. This pipeline was evaluated on 2002 image patches of size 2000x2000 pixels, sampled from 9 whole slide images. We achieved the mean R-2 of 0.70 +/- 0.14 in our testing set. This pipeline can pave the way to study proximity of tumours cells to blood vessels. This approach has the potential to reduce the use of IHC and tissues and enable large quantitative studies.

Prostate cancer is the second most commonly diagnosed cancer among men and currently multi-parametric MRI is a promising imaging technique used for clinical workup of prostate cancer. Accurate detection and localisation of the prostate tissue boundary on various MRI scans can be helpful for obtaining a region of interest for Computer Aided Diagnosis systems. In this paper, we present a fully automated detection and segmentation pipeline using a conditional Generative Adversarial Network (cGAN). We investigated the robustness of the cGAN model against adding Gaussian noise or removing noise from the training data. Based on the detection and segmentation metrics, de-noising did not show a significant improvement. However, by including noisy images in the training data, the detection and segmentation performance was improved in each 3D modality, which resulted in comparable to state-of-the-art results.

Early diagnosis of breast cancer can increase survival rate. The assessment process for breast screening follows a triple assessment model: appropriate imaging, clinical assessment and biopsy. Retrieving prior cases with similar cancer symptoms could be used to circumvent incompatibilities in breast cancer grading. Abnormal mass lesions in breast are often co-located with normal tissue, which makes it difficult to describe the whole image with a single binary code. Therefore, we propose an AI-based method to describe mass lesions in semantic abstracts/codes. These codes are used in a searching based method to retrieve similar cases in the archive. This simple and effective network is used for unifying classification and retrieval in a single learning process, while enforcing similar lesion types to have similar semantic codes in a compact form. An advantage of this approach is its scalability to large-scale image retrievals.

Background High-throughput phenotyping based on non-destructive imaging has great potential in plant biology and breeding programs. However, efficient feature extraction and quantification from image data remains a bottleneck that needs to be addressed. Advances in sensor technology have led to the increasing use of imaging to monitor and measure a range of plants including the model Arabidopsis thaliana. These extensive datasets contain diverse trait information, but feature extraction is often still implemented using approaches requiring substantial manual input. Results The computational detection and segmentation of individual fruits from images is a challenging task, for which we have developed DeepPod, a patch-based 2-phase deep learning framework. The associated manual annotation task is simple and cost-effective without the need for detailed segmentation or bounding boxes. Convolutional neural networks (CNNs) are used for classifying different parts of the plant inflorescence, including the tip, base, and body of the siliques and the stem inflorescence. In a post-processing step, different parts of the same silique are joined together for silique detection and localization, whilst taking into account possible overlapping among the siliques. The proposed framework is further validated on a separate test dataset of 2,408 images. Comparisons of the CNN-based prediction with manual counting (R2 = 0.90) showed the desired capability of methods for estimating silique number. Conclusions The DeepPod framework provides a rapid and accurate estimate of fruit number in a model system widely used by biologists to investigate many fundemental processes underlying growth and reproduction

The manual delineation of region of interest (RoI) in 3D magnetic resonance imaging (MRI) of the prostate is time-consuming and subjective. Correct identification of prostate tissue is helpful to define a precise RoI to be used in CAD systems in clinical practice during diagnostic imaging, radiotherapy and monitoring the progress of disease. Conditional GAN (cGAN), cycleGAN and U-Net models and their performances were studied for the detection and segmentation of prostate tissue in 3D multi-parametric MRI scans. These models were trained and evaluated on MRI data from 40 patients with biopsy-proven prostate cancer. Due to the limited amount of available training data, three augmentation schemes were proposed to artificially increase the training samples. These models were tested on a clinical dataset annotated for this study and on a public dataset (PROMISE12). The cGAN model outperformed the U-Net and cycleGAN predictions owing to the inclusion of paired image supervision. Based on our quantitative results, cGAN gained a Dice score of 0.78 and 0.75 on the private and the PROMISE12 public datasets, respectively.

Current deep learning based detection models tackle detection and segmentation tasks by casting them to pixel or patch-wise classification. To automate the initial mass lesion detection and segmentation on the whole mammographic images and avoid the computational redundancy of patch-based and sliding window approaches, the conditional generative adversarial network (cGAN) was used in this study. Subsequently, feeding the detected regions to the trained densely connected network (DenseNet), the binary classification of benign versus malignant was predicted. We used a combination of publicly available mammographic data repositories to train the pipeline, while evaluating the model's robustness toward our clinically collected repository, which was unseen to the pipeline.