Classification of 18 F‐flutemetamol scans using machine learning with neuropathology as standard of truth: Neuroimaging / Optimal neuroimaging measures for tracking disease progression

Background There is a rising interest in machine learning for classification of images and lesions in medicine. A hurdle is the necessity for enough labeled cases based on an independent standard‐of‐truth. Here we applied linear support vector machine (SVM) to the phase 3 18 F‐flutemetamol end‐of‐life study (GE067‐007), where 18 F‐flutemetamol PET scans were available in close proximity to the time of autopsy and where brains were characterized in a standardized manner. Method As our primary analysis, we trained a leave‐one‐out SVM to discriminate between normal vs abnormal cases based on Bielschowsky silver staining (BSS) (the primary outcome measure of GE067‐007). As a secondary analysis, we also trained a classifier to discriminate between Aβ (Thal) phase 0‐1 vs phase 5. The dataset contained 29 normal and 72 abnormal cases according to BSS SoT and 17 phase 0‐1 and 44 phase 5 cases. To avoid bias, the number of cases was equated between classes (29 per class for BSS and 17 per class for Aβ). Selection of cases per class was performed randomly 50 times. Result Compared to the BSS SoT, the classifier had a sensitivity of 84% and a specificity of 90% (accuracy 87%) (Figure 1). Compared to Aβ phase, the sensitivity and specificity was, respectively, 97% and 100% (accuracy 98%). The three discordant BSS SoT normal cases, all had Aβ phase 3 or higher. Overall, highest feature weights were in striatum, middle frontal gyrus, precuneus, middle temporal, and cingulate. When we selected the voxels with the 10% highest amplitudes of feature weights (Figure 2) and re‐ran the analysis against the BSS SoT, performance increased to 95% sensitivity and 98.8% specificity. When we trained the algorithm to distinguish Aβ phase 4 (n=21) or 3 (n=14) from 0‐1, accuracy decreased to 88.8% and 71.9% respectively. In a further analysis, Aβ phase 3 vs 4 classification had an accuracy of 62.4% (95% CI 60.8, 63.9) and for phase 4 vs 5 an accuracy of 60.3% (95% CI 58.4, 62.1). Conclusion This study shows that SVM is able to classify 18 F‐flutemetamol scans in a binary manner (normal vs abnormal) which closely corresponds to the neuropathology measures.