Automatic bleeding detection in laparoscopic surgery based on a faster region-based convolutional neural network

Background: Laparoscopic surgery has been in great demand over the past decades; it has also brought several obstacles, such as increasing difficulty in maintaining hemostasis, changes in surgical approach, and reduced field of vision. Locating the bleeding point can help surgeons to control bleeding quickly, however, to date, there have been no tools designed for automatic bleeding tracking in laparoscopic operations. Herein, we have proposed a spatiotemporal hybrid model based on a faster region-based convolutional neural network (RCNN) for bleeding point detection in laparoscopic surgery videos. Methods: Laparoscopic videos performed at our hospital were retrieved and images containing bleeding events were extracted. Spatiotemporal features were extracted by using red-green-blue (RGB) frames and optical flow maps and a spatiotemporal hybrid model was developed based on the faster RCNN. The proposed model contributed to (I) providing real-time bleeding point detection which directly assist surgeons, (II) showing the blood's optical flow which improved bleeding point detection, and (III) detecting both arterial and venous bleeding. Results: In this study, 12 different bleeding videos were included for deep learning model training. Compared with models containing a single RGB or a single optical flow map, our model combining RGB and optical flow achieved great detection results (precision rate of 0.8373, recall rate of 0.8034, and average precision of 0.6818). Conclusions: Our approach performs well in bleeding point location and recognition, indicating its potential value in helping to maintain and re-establish hemostasis during operations.