Augmented Reality in Orthognathic Surgery: A Multi-Modality Tracking Approach to Assess the Temporomandibular Joint Motion

Augmented Reality (AR) is a rapidly emerging technology finding growing applications in various surgery domains. In this study, we develop and test the feasibility of a novel AR application for Microsoft HoloLens2 Head Mounted Display (HMD) to support surgeons in the clinical evaluation of temporomandibular joint (TMJ) alterations that may require surgery. The application implements a multi-modality tracking based on the combination of a marker-less and a marker-based approach to simultaneously track the fixed part of the joint and the moving mandible. The AR application was tested on a volunteer performing the TMJ task, i.e. the opening and closing of the mouth. During the task, video recordings were taken from the HoloLens cameras to derive the trajectories as well as the horizontal and vertical excursions of the jaw movements. The AR-derived TMJ movements were then compared with standard kinesiographic acquisitions. The results demonstrated the feasibility of the proposed AR application in superimposing the 3D visualization of the joint to the patient's head, thus facilitating the diagnostic evaluation for the surgeon. The AR-derived trajectories were consistent with the kinesiography curves. Future improvements are needed to reduce the encumbrance of the tracker and to provide additional visual cues for the surgeon. The presented methodology can be easily transferred to other surgical applications which require simultaneous tracking of two anatomical parts, such as the case of bone repositioning to a pre-planned target location inmaxillofacial and orthopedic surgery.