Development of a Multi-view Multi-level Artificial Intelligence System to Stratify Risk Assessment of Mammography

Abstract Background: Recent artificial intelligence has exhibited great potential in breast imaging, but its value in precise risk stratification of mammography still needs further investigation. This study is to develop an artificial intelligence system (AIS) for accurate malignancy diagnosis and supportive decision-making on mammographic risk stratification. Methods: In this retrospective study, 49732 mammograms of 24866 breasts from 12815 women from two Asian clinics between August 2012 and December 2018 were included. We developed an AIS using multi-view mammograms and multi-level convolutional neural network features to diagnosis malignancy and further assess the relative strengths of AIS versus current BI-RADS categorization. We further evaluate AIS by conducting a counterbalance-designed AI-assisted study, where ten radiologists read 1302 cases with/without AIS assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, F1 score were measured. Results: The AIS yielded AUC of 0.910 to 0.995 for malignancy diagnosis in the validation and testing sets. Within BI-RADS 3–4 subgroups with pathological results, AIS can downgrade 83.1% of false-positives into benign groups, and upgrade 54.1% of false-negatives into malignant groups. Compared with BI-RADS, AIS performed better sensitivity and specificity in dense and no-calcification subgroups. AIS also can successfully assist radiologists identify 7 out of 43 malignancies initially diagnosed with BI-RADS 0 with specificity of 96.7%. In the counterbalance-designed AI-assisted study, the average AUC across 10 readers was significantly improved with AIS assistance (P = 0.001). Conclusion: AIS can identify malignancy on mammography and further serve as a supportive tool for stratifying BI-RADS categorization.