Multiview 2d/3d Rigid Registration Via A Point-Of-Interest Network For Tracking And Triangulation

We propose to tackle the multiview 2D/3D rigid registration problem via a Point-Of-Interest Network for Tracking and Triangulation(POINT2). POINT2 learns to establish 2D point-to-point correspondences between the pre- and intra-intervention images by tracking a set of point-of-interests (POIs). The 3D pose of the pre-intervention volume is then estimated through a triangulation layer In POINT2, the unified framework of the POI tracker and the triangulation layer enables learning informative 2D features and estimating 3D pose jointly. In contrast to existing approaches,POINT2 only requires a single forward-pass to achieve a reliable 2D/3D registration. As the POI tracker is shift-invariant,POINT2 is more robust to the initial pose of the 3D pre-intervention image. Extensive experiments on a large-scale clinical cone-beam computed tomography dataset show that the proposed POINT2 method outperforms the existing learning-based method in terms of accuracy, robustness and running time. Furthermore,when used as an initial pose estimator our method also improves the robustness and speed of the state-of-the-art optimization based approaches by ten folds.