The potential of Convolutional Neural Networks (CNNs) in detecting various forms of cancer has been widely recognized. However, the generalizability of metabolic signature-based results across the three categories of TNM staging using deep texture analysis remains largely unexplored. This study seeks to examine the cross-TNM category and cross-cancer relevance of deep textures identified by a 3D CNN model. Our methodology involves processing patches of PET data from various glycolytic volumes, including primary tumor, malignant nodes and metastases, from PETFDG data, with high Standardized Uptake Values (SUV) for patients presenting with clinically confirmed esophageal cancer. Activation values were calculated for each filter in the CNN, and a Principal Component Analysis (PCA) is conducted to assess the model’s ability to generalize its learned metabolic signatures. Our results indicate that a CNN trained for esophageal cancer primary tumor detection can successfully demonstrate distinct patterns of cellular metabolism for primary tumor, malignant nodes and metastases that a CNN trained for esophageal cancer primary tumor detection can discern all parts of esophageal TNM staging categories.

Deep learning has demonstrated effectiveness in PET-CT lesion detection [1] and interpreting complex glycotic uptake distributions [2]. Our prior work [3] pioneered the use of Gramian matrices to characterize and synthesize PET tumour heterogeneity. This study employs our “Titanium” features derived from the activation functions within the pretrained VGG-19 convolutional neural network (CNN). From this diverse feature set, we present our survival prediction model and validate the superior efficacy of “Titanium” features in enhancing predictive modeling of 2-year survival for non-small-cell lung cancer (NSCLC) patients compared to traditional IBSI Radiomic Features. The model was evaluated on 381 NSCLC patients, expertly segmented and split into 80% training, $10 \%$ validation, and $10 \%$ testing sets. “Titanium” features were computed from activation functions across five CNN layers on three orthogonal tumor slices, providing a comprehensive 2.5D representation. This yielded 150 complex textural descriptors. Contrastingly, 203 traditional IBSI radiomic features were derived. Preprocessing included standard scaling, polynomial expansion, and L1-regularized logistic regression for feature selection. Data transformations and feature selection were determined solely from training the set to prevent data leakage. Prediction involved optimizing a multilayer perceptron on the validation set using cross-validation followed by independent testing on the held out set. Our results revealed that the “Titanium” model achieved a Receiver Operating Characteristic (ROC) Area Under the Curve (AUC) of 0.83, significantly outperforming the traditional radiomic approach, which only achieved an AUC of 0.54. This $53.7 \%$ improvement in predictive power highlights the effectiveness of “Titanium” features in capturing more nuanced and predictive information and presents a sophisticated analytical technique for clinical predictive modelling of oncological outcomes.

Fluorodeoxyglucose (FDG) Positron Emission Tomography (PET) serves as a cornerstone in functional imaging. PET imaging reveals metabolic abnormalities before morphological alterations occur. The standardized uptake value is a common metric to semi-quantify PET images. Visual scoring of PET image heterogeneity has demonstrated correlations with tumour characteristics. Quantification of this spatial variation in FDG distribution via its shape and texture has proven to be a fertile area of research under the umbrella term "Radiomics." Deep learning with convolutional neural networks (CNN) serves as large feature extractors with proven ability in tumour type classification in PET imaging. Deep learning in PET imaging has also been utilized for image enhancement, reconstruction, segmentation and image synthesis. In this work, we quantify the feature distribution of neural activations of each layer in a CNN following propagation through the network of a PET image. The Gramian matrix of the activation functions quantifies the neural representation of the CNN and thus serves as a measure of visual perception. Image synthesis using gradient descent on the target PET image Gramian matrix allows visualization of the texture. This metric is tested in PET imaging data of a recently developed texture phantom and a single primary tumour from a non-small-cell lung cancer (NSCLC) patient. It is demonstrated that the summation of the Gramian decreases with decreasing homogeneity. The visual appearance of the synthesized PET images closely resembles that ground truth PET images, both in the texture phantom and clinical image.

Positron Emission Tomography and Computed Tomography (PET-CT) is a vital imaging technique for accurate cancer diagnosis, staging, and treatment planning, offering complementary morphological and anatomical information. A 5-layer 3D convolutional deep learning texture model was employed to identify glycolytic regions in PET-CT data, achieving an average sensitivity and specificity of 96.1% and 99.4%, respectively, for binary classification targeting primary tumor patches. Using a dataset of PET-CT data from 486 esophageal patients, we analyzed network activations across each layer for characteristic activation patterns of four glycolytic uptake classes: primary tumor, bladder, liver, and myocardium. PCA analysis of the activations was performed to isolate uncorrelated features learned during training and reveal unique feature clusters in PCA space, demonstrating that glycolytic regions with high SUV values exhibit distinct textures learnable by deep learning architectures. This information was used to prune low activation probability nodes in the network, resulting in a more efficient deployable network with slightly improved classification performance. A comprehensive quantitative evaluation of redundant filters in the network, examining filter combinations that result in positive (tumor-present) and negative (tumor-absent) predictions across multiple patients will be presented, alongside preliminary results on the use of AI for automatic staging.

Nuclear Medicine (NM) Imaging serves as a powerful technique in visualizing radio-pharmaceutically targeted physiological processes allowing non-invasive monitoring of disease, it's progression and therapy response. Positron Emission Tomography (PET) imaging has become a cornerstone in oncology and is utilized in disease staging, monitoring response to therapy, detecting recurrence and predicting prognosis. Artificial intelligence is increasingly being adopted to assist radiological interpretation of PET images. In this work, we for the first time set instance object detection in PET imaging in a reinforcement learning (RL) paradigm. Q learning, a model free RL learning algorithm, is used to learn a policy which tells an agent what actions to take to maximize a reward in an environment in order to achieve a specific task. In this work Q learning with a novel reward function defined by the Kullback-Leibler (KL) divergence is used to detect instances of an object in PET images. The RL agent is tested using a phantom study and accurately identifies the location of all five spheres; producing a mean error of 1:5 voxels. Testing is also performed on two 18F-fluorodeoxyglucose investigations imaged for oesophageal cancer. The location of the cancer lesion determined by the RL agent on a sagittal cross section is in excellent correspondence to the location defined by an expert radiologist, with a mean error of 2 voxels. The RL framework thus proves promise for automated object detection in PET imaging and providers the first example of it's use in Nuclear Medicine imaging studies.

PET-CT scans using 18 F-FDG with a co-registered CT scan are increasingly used to detect cancer. This paper compares deep learning-based lesion detection tools trained on PET, CT and combined modality data. 486 pre-contoured scans were used from a retrospective cohort study into esophageal cancer. Scans were partitioned into training, validation and test sets with an 80:10:10 ratio. 1000 image segments were generated from each scan, with tumor present segments located on the contoured lesion and tumor absent segments distributed randomly within the patient but excluding the tumor. PET and CT image segments were used to train a separate dedicated 5-layer convolutional neural networks (CNN). Testing on segments from unseen scans resulted in an accuracy of greater than 95% for the PET data, and greater than 90% for CT data.

PET-CT scans using 18F-FDG are increasingly used to detect cancer, but interpretation can be challenging due to non-specific uptake and complex anatomical structures nearby. To aide this process, we investigate the potential of automated detection of lesions in 18F-FDG scans using deep learning tools. A 5-layer convolutional neural network (CNN) with 2x2 kernels, rectified linear unit (ReLU) activations and two dense layers was trained to detect cancerous lesions in 2D axial image segments from PET scans. Pre-contoured scans from a retrospective cohort study of 486 oesophageal cancer patients were split 80:10:10 into training, validation and test sets. These were then used to generate a total of similar to 14000 25x25x25 voxel image segments, where tumor present segments were centred on the marked lesion, and tumor absent segments were randomly located outside the marked lesion. ROC curves generated from the test dataset produced an average AUC of similar to 99%. Ten-fold cross validation on unseen test data was performed which resulted in a sensitivity of 99.5 +/- 0.4% and a specificity of 99.4 +/- 0.3%. A representative model was used to successfully generate volumetric tumor probability maps for the test dataset.