Augmented Reality for Supporting Real Time Telementoring - an Exploratory Study Applied to Ultrasonography.

Background: Telementoring is a real time telemedicine modality that allows a specialist to remotely assist a less expert operator. Videoconferencing is the main means for implementing telementoring applications and has been used in a wide range of clinical disciplines.; Long-distance communication can be affected by misunderstanding and misinterpretation, in particular when dealing with remote guidance. Videoconferencing can be enhanced using Augmented Reality (AR), which enriches video feeds with virtual objects like 3D models, pointers and annotations.; Objective: To evaluate through an exploratory study the impact of AR in remotely assisted ultrasound exams.; Material and Methods: In previous studies we developed a real time telementoring system (CRS4-TELEMED) for allowing a specialist to remotely guide the execution of an echocardiographic exam by an operator. The system manages three streams: the video from an IP cam, showing the exam scene; the sonographic video from the ultrasound machine; the bidirectional audio for allowing communication between the specialist and the operator. The system has been extended with a set of virtual telepointers, superimposed on the sonographic video and the keyboard of the ultrasound machine, in order to allow the remote specialist, which uses an Android tablet, to indicate regions of interests. The local operator wears Android smartglasses (Epson Moverio bt-200) that enable optical see through visualization of pointers. ARToolkit was used for implementing marker based AR. Results: Preliminary tests were conducted at the Pediatric Cardiology department of AOU Brotzu in Cagliari. Doctors and nurses with different skills and experience in ultrasound exams were involved. Participants judged the system positively, considering AR telepointers useful for improving remote assistance. The following issues emerged: optical alignment is suboptimal, some training is required to understand how smartglasses and AR work, smartglasses results uncomfortable, especially if the user wears glasses. Conclusions: AR based telepointers can be used for improving telementoring, allowing a remote specialist to indicate region of interests. Issues like comfort and alignment precision can be overcome with future evolution of smartglasses. Future works include: new pointers for indicating region on the patients and how to use the ultrasound probe; a user friendly in app calibration procedure for increasing AR precision on smartglasses; a field testing for evaluating the system.