Utilizing Mask RCNN for Monitoring Postoperative Free Flap: Circulatory Compromise Detection Based on Visible-Light and Infrared Images

The new postoperative free flap monitoring system combines visible-light and infrared techniques to overcome the limitations of our previous study, such as resistance to illumination change, patients with large movements, and the infrared images being too blurry to identify the boundary. In the visible-light system, the Mask region-based convolutional neural networks (RCNN) was adopted to segment the region of the free flap, and these time-course visible images were aligned using our proposed image registration method. Then, the registered visible images were projected onto the infrared image by the coordinate transformation. The analysis method adopted the residual factor analysis to extract the more unvarnished specific factors. The experiments were divided into two parts. In image processing, the accuracy of the coordinate transformation has a mean error of 1.78 pixels and a standard deviation of 0.98 pixels. The segmentation results showed excellent performance in the most severe case with apparent motion and rotation, covered by gauze and the ventilator, as well as illumination variation. The dice coefficient is 0.9551 +/- 0.0158, and the Hausdorff distance is 2.3943 +/- 0.3921 pixels. The image registration results also reveal that the Canny edge of the deformed image was superimposed onto the reference image well. In circulatory compromise detection, the vascular congestion was detected much earlier than manual observation, and the classified type of occlusion was the same as the clinical reports. Therefore, the dual-camera monitoring system provides a reliable tool for the surgeon to hold onto the chance of repairing the free flap with vascular obstruction.