Numerical landmark detection algorithm for fringe projection profilometry during autonomous robotic suturing

Fringe projection profilometry (FPP) is being developed as a 3D vision system to assist robotic surgery and autonomous suturing. Conventionally, fluorescence markers are placed on a target tissue to indicate suturing landmarks, which not only increase the system complexity, but also impose safety concerns. To address these problems, we propose a numerical landmark detection algorithm based on deep learning. A landmark heatmap is regressed using an adopted U-Net from the four channel data generated by the FPP. A Markov random field leveraging the structure prior is developed to search the correct set of landmarks from the heatmap. The accuracy of the proposed method is verified through ex-vivo porcine intestine landmark detection experiments.