Results of an AI-Based Image Review System to Detect Patient Misalignment Errors in a Multi-Institutional Database of CBCT-Guided Radiotherapy Treatments

Purpose Present knowledge of patient setup and alignment errors in image-guided radiotherapy (IGRT) relies on voluntary reporting, which is thought to underestimate error frequencies. A manual retrospective patient-setup misalignment error search is infeasible due to the bulk of cases to be reviewed. We applied a deep learning-based misalignment error detection algorithm (EDA) to perform a fully-automated retrospective error search of clinical IGRT databases and determine an absolute gross patient misalignment error rate. Methods The EDA was developed to analyze the registration between planning scans and pre-treatment CBCT scans, outputting a misalignment score ranging from 0 (most unlikely) to 1 (most likely). The algorithm was trained using simulated translational errors on a dataset obtained from 680 patients treated at two radiotherapy clinics between 2017 and 2022. A receiver operating characteristic analysis was performed to obtain target thresholds. A DICOM Query and Retrieval software was integrated with the EDA to interact with the clinical database and fully automate data retrieval and analysis during a retrospective error search from 2016-2017 and 2021-2022 for the two institutions, respectively. Registrations were flagged for human review using both a hard-thresholding method and a prediction trending analysis over each individual patient's treatment course. Flagged registrations were manually reviewed and categorized as errors (>1cm misalignment at the target) or non-errors. Results A total of 17,612 registrations were analyzed by the EDA, resulting in 7.7% flagged events. Three previously reported errors were successfully flagged by the EDA and four previously-unreported vertebral body misalignment errors were discovered during case reviews. False positive cases often displayed substantial image artifacts, patient rotation, and soft-tissue anatomy changes. Conclusion Our results validated the clinical utility of the EDA for bulk image reviews, and highlighted the reliability and safety of IGRT, with an absolute gross patient misalignment error rate of 0.04% ± 0.02% per delivered fraction.