A deep learning and natural language processing-based system for automatic identification and surveillance of high-risk patients undergoing upper endoscopy: A multicenter study

Background Timely identification and regular surveillance of patients at high risk are crucial for early diagnosis of upper gastrointestinal cancer. However, traditional manual surveillance method is time-consuming, and current surveillance rate is below 50%. Here, we aimed to develop a surveillance system named ENDOANGEL-AS (automatic surveillance) for automatic identification and surveillance of high-risk patients.Methods 7874 patients from Renmin Hospital of Wuhan University between May 1 and July 31, 2021 were used as the training set, 6762 patients between August 1 and October 31, 2021 as the internal test set, and 7570 patients from two other hospitals between August 1 and October 31, 2021 as the external test sets. We first extracted descriptions of abnormalities from endoscopic and pathological reports based on natural language processing techniques to identify individuals. Then patients were classified at nine risk levels according to endoscopic and pathological findings, and a deep learning model was trained to identify demarcation line (DL) in gastric low-grade intraepithelial neoplasia (LGIN) using 1561 white-light still images for risk stratification of gastric LGIN. Finally, patients undergoing upper endoscopy were classified and assigned one of ten surveillance intervals according to guidelines. The performance of ENDOANGEL-AS was evaluated and compared with physicians.Findings Patient identification module achieved an accuracy of 100% and 99.91% in internal and external test sets, respectively. Risk level classification module achieved an accuracy of 100% and 99.85% in the internal and external test sets, respectively. DL identification module achieved an accuracy of 87.88%. ENDOANGEL-AS on surveillance interval assignment achieved an accuracy of 99.23% and 99.67% in internal and external test sets, respectively. ENDOANGEL-AS had significantly higher accuracy compared with physicians (99.00% vs 38.87%, p < 0.001). The accuracy (63.67%, p < 0.001) of endoscopists with the assistance of ENDOANGEL-AS was significantly improved. Interpretation We established a surveillance system that can automatically identify patients and assign surveillance intervals with high accuracy and good transferability.Funding This work was partly supported by a grant from the Hubei Province Major Science and Technology Inno-vation Project (2018-916-000-008) and the Fundamental Research Funds for the Central Universities (2042021kf0084).Copyright (c) 2022 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).