Over-detection of Colonic Abnormalities on CTAP and a Decrease with Experience: A Simulation of Machine Learning

68 Background: Artificial intelligence (AI) can potentially improve patient care by assisting physicians as demonstrated with the recent approvals for technologies that detect intracranial hemorrhage on CT exams of the head. AI also has potential for assisting early detection of colorectal cancer (CRC) on routine CT abdomen and pelvis (CTAP). We assessed the difference in detection of abnormal colonic findings between an expert reader (JC) and amateur readers (ARs). Methods: ARs consisting of two third year medical students (RB - AR1, ZG - AR2) studied 20 CTAP for tracing the colon and identifying pathology. Their search pattern and assessment of the colon was then evaluated by an expert radiologist (JC). They then spent two hours reviewing abnormalities in 10 scans with JC, who highlighted suspicious neoplastic findings such as colonic wall thickening, fat stranding, edema, masses, and abnormal lymph nodes. The ARs then individually read 203 CTAP scans to assess for these suspicious findings. The studies were from a single institution and were reported in a prior study in 2019 GI-ASCO. The findings of the ARs were then compared to those of the expert reader and the initial reader for each study. Data was analyzed using t-test with 2 tails. Results: The incidence of suspicious neoplastic findings was 87% and 81% for AR1 and AR2, respectively, compared to 18% in the initial reads and 33% for expert reader (p=0.01). Greatest discordance were 94% and 87% between AR1 and AR2 to the initial reads. Additionally, the incidence of suspicious findings between the first and last 20 cases (p=0.03 and 0.17) examined by ARs declined from 79 to 40% for AR1 and 69 to 55% for AR2. Conclusions: ARs are capable of detecting CRC features on CTAP from ED, but with higher false-positive (FP) rate than trained experts. The FP rate decreases with increasing experience. ARs learning course simulates AI which will likely yield high FP rate with initial training, but with improving FP with deep training, especially with larger volume of normal variants.