DEEP LEARNING FOR REAL-TIME AUTOMATED POLYP LOCALISATION IN COLONOSCOPY VIDEOS

Introduction Colonoscopic polypectomy can prevent colorectal cancer. Polyp detection rates vary considerably due to human error and missed adenomas may contribute to interval colorectal cancers. Automated polyp detection using deep learning may avoid these problems. Previous work focused on detecting the presence of polyps in individual frames captured from videos. Our aims in this pilot study were to extend this to video sequences and to explore future-proofing by using algorithms trained on old image processors to locate polyps found using newer endoscopic technologies. Methods We trained and validated a Convolutional Neuronal Network (CNN) on 18 517 frames created by merging research colonoscopy datasets (CVCClinic, ASUMayo, ETIS, CVCVideoDB and CVCColon) from the Medical Image Computing and Computer Assisted Intervention Society challenges. 75% of frames contained polyps in both standard and high definition (HD) from older processors including Olympus Exera II (160/165 series) and Pentax EPKi 7000 (90i series). Our test set consisted of 11 HD videos featuring polyps in white light collected using the latest Olympus 290 endoscopes at a UK tertiary centre. Estimated median polyp size was 4 mm (range 2–15) and morphology included (Paris Classification IIa=4, Is=6 and IIa +IIs LST-G=1). Images were manually annotated by drawing bounding boxes around polyps and quality controlled by removing uninformative frames (e.g. blurred). A total of 2611 polyp-containing frames were analysed in the test set. A true positive was scored if the computer-generated segmentation mask prediction overlapped with the bounding box. A false positive indicated a non-overlapping location (more than one can occur per frame). Results Our network operated at real-time video rate. F1-score accuracy was 92.5%. Sensitivity for polyp localisation was 98.5% and per-frame specificity 75.4%. Positive predictive value was 90.1%. Incorrect segmentation mask locations were predominantly limited to 3 videos and were generated by artefacts not represented during training. Conclusion We demonstrate through analysis of video frames that a CNN can locate polyps with high accuracy in real-time. The algorithm was trained using multiple endoscopy processors and worked with HD images from a new processor. This suggests that the CNN could remain useful as new endoscopic technologies are introduced. Further work will train our model on larger datasets including complete colonoscopy procedures. This should improve accuracy further. Such a system could be used as a red-flag technique to reduce missed adenomas during colonoscopy.