MIXR: A Standard Architecture for Medical Image Analysis in Augmented and Mixed Reality

Medical image analysis is evolving into a new dimension: where it will combine the power of AI and machine learning with real-time, real-space displays, namely Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) - known collectively as Extended Reality (XR). These devices, typically available as head-mounted displays, are enabling the move towards the complete transformation of how medical data is viewed, processed and analysed in clinical practice. There have been recent attempts on how XR gadgets can help in surgical planning and training of medics. However, the radiological front from a detection, diagnostics and prognosis remains unexplored. In this paper we propose a standard framework or architecture called Medical Imaging in Extended Reality (MIXR) for building medical image analysis applications in XR. MIXR consists of several components used in literature; however, tied together for reconstructing volume data in 3D space. Our focus here is on the reconstruction mechanism for CT and MRI data in XR; nevertheless, the framework we propose has applications beyond these modalities.