A hybrid few-shot multiple-instance learning model predicting the aggressiveness of lymphoma in PET/CT images

Background and objective: Patients with aggressive non-Hodgkin lymphoma (NHL) undergo distinct therapy strategies compared with indolent NHL patients. However, it is challenging to estimate NHL aggressiveness based on visual inspection of positron emission tomography (PET) or computed tomography (CT) images. Since diffuse large B-cell lymphoma (DLBCL) and Follicular lymphoma (FL) are the most typical and dominant aggressive and indolent NHL, respectively, this study aims to develop an artificial-intelligence-enabled model to distinguish DLBCL from FL in PET/CT images as the first step to tackle this challenge.Methods: We propose a hybrid few-shot multiple-instance learning model to predict the aggressiveness of the NHL. First, rotation-based self-supervision learning (SSL) has been employed to train the encoder on a largescale, publicly available CT image dataset. Second, hybrid instance-level features are obtained for each NHL lesion by combining deep features with the radiomics features from both PET and CT modalities. Third, instancelevel features are transformed into bag-level (or patient-level) representations. Finally, bag-level representations are fed into a distance-based classifier through few-shot learning to predict NHL aggressiveness. Results: Our model achieves an accuracy of 0.751 +/- 0.008, a sensitivity of 0.787 +/- 0.012, a specificity of 0.715 +/- 0.013, an F1-score of 0.753 +/- 0.009, and an area under the curve (AUC) of 0.795 +/- 0.009 at the bag level. It outperforms the typical counterparts that use the radiomic features, random forest for feature selection, and support vector machines (SVMs) as classifiers. The three counterparts yield accuracies of 0.714 +/- 0.023, 0.705 +/- 0.008, and 0.698 +/- 0.008, respectively. Moreover, settings of the SSL training dataset (Deep lesion) and task (rotation), hybrid CT and radiomic PET features, the pool-layer strategy of maximum, and distance-based classifier generate the best model.Conclusions: A hybrid few-shot multiple-instance learning model can predict lymphoma aggressiveness in PET/CT images and could be a potential tool for determining therapy strategies. Hybrid features and the combination of SSL, few-shot learning, and weakly supervised learning are the two powerful pillars of the model, and these can be expanded to other medical applications with limited samples and incomplete annotations.