A Real-Time Polyp Detection System with Clinical Application in Colonoscopy Using Deep Convolutional Neural Networks

Abstract Background: Colorectal cancer (CRC) is still a leading cause of cancer-related deaths worldwide. The best method to prevent CRC is a colonoscopy. During this procedure, the colonoscopist searches for polyps. However, there is a potential risk of polyps being missed by the examiner. Here the automated detection of polyps helps assist the examiner during coloscopy. In the literature, there are already publications examining the problem of polyp detection. Nevertheless, most of these systems are only used in the research context and do not attain clinical application. Therefore, we introduce a system scoring best on current benchmarks and implementing it fully for clinical-ready applications. Methods: To create the polyp detection system (ENDOMIND-Advanced), we combined our own collected data from different hospitals and practices in Germany with open-source data sets to create a data set with over 500.000 annotated images. Furthermore, we show different techniques for training a CNN on polyp detection that involves preprocessing, data augmentation and hyperparameter optimization. Additionally, we developed a post-processing technique based on video detection to work in real-time with a stream of images. This allows us to leverage the incoming stream context of the endoscope while maintaining real-time performance. Furthermore, the polyp detection system is integrated into a prototype ready for application in clinical interventions. Results: First, we show that our polyp detection system is state of the art by evaluating it on the CVC-VideoClinicDB benchmark with a F1-score of 90.24%. We compare the polyp detection system to the best system in the literature and achive better results in speed and accuracy. Additionally, we show its performance on our own data and introduce a new metric called the time to the first detection. This metric is given in seconds and shows how long AI systems need to detect a polyp for the first time. Finally, we further elaborate on the explainability of our system by showing heatmaps of the neural network explaining neural activations. Conclusion: Overall we introduce a fully assembled real-time system for polyp detection with application in clinical practice and show that the system outperforms current systems on benchmark data sets with real-time performance.