Development and Validation of a Multimodal Model in Predicting Severe Acute Pancreatitis Based on Radiomics and Deep Learning

Objective: Aim to establish a multimodal model for predicting severe acute pancreatitis (SAP) using machine learning (ML) and deep learning (DL). Methods: In this multicentre retrospective study, patients diagnosed with acute pancreatitis at admission were enrolled from January 2017 to December 2021. Clinical information within 24 h and CT scans within 72 h of admission were collected. First, we trained Model alpha based on clinical features selected by least absolute shrinkage and selection operator analysis. Second, radiomics features were extracted from 3D-CT scans and Model beta was developed on the features after dimensionality reduction using principal component analysis. Third, Model gamma was trained on 2D-CT images. Lastly, a multimodal model, namely PrismSAP, was constructed based on aforementioned features in the training set. The predictive accuracy of PrismSAP was verified in the validation and internal test sets and further validated in the external test set. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, recall, precision and F1-score. Results: A total of 1,221 eligible patients were randomly split into a training set (n = 864), a validation set (n = 209) and an internal test set (n = 148). Data of 266 patients were for external testing. In the external test set, PrismSAP performed best with the highest AUC of 0.916 (0.873-0.960) among all models [Model alpha: 0.709 (0.618-0.800); Model beta: 0.749 (0.675-0.824); Model gamma: 0.687 (0.592-0.782); MCTSI: 0.778 (0.698-0.857); RANSON: 0.642 (0.559-0.725); BISAP: 0.751 (0.668-0.833); SABP: 0.710 (0.621-0.798)]. Conclusion: The proposed multimodal model outperformed any single-modality models and traditional scoring systems.