The use of haptic simulation for emergency procedures in nursing training presents a viable, versatile and affordable alternative to traditional mannequin environments. In this paper, an evaluation is performed in a virtual environment with a head-mounted display and haptic devices, and also with a mannequin. We focus on a chest decompression, a life-saving invasive procedure used for trauma-associated cardiopulmonary resuscitation (and other causes) that every emergency physician and/or nurse needs to master. Participants’ heart rate and blood pressure were monitored to measure their stress level. In addition, the NASA Task Load Index questionnaire was used. The results show the approved usability of the VR environment and that it provides a higher level of immersion compared to the mannequin, with no statistically significant difference in terms of cognitive load, although the use of VR is perceived as a more difficult task. We can conclude that the use of haptic-enabled virtual reality simulators has the potential to provide an experience as stressful as the real one while training in a safe and controlled environment.

Open AccessMoreSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InRedditEmail Cite this article Dykes Jason, Abdul-Rahman Alfie, Archambault Daniel, Bach Benjamin, Borgo Rita, Chen Min, Enright Jessica, Fang Hui, Firat Elif E., Freeman Euan, Gönen Tuna, Harris Claire, Jianu Radu, John Nigel W., Khan Saiful, Lahiff Andrew, Laramee Robert S., Matthews Louise, Mohr Sibylle, Nguyen Phong H., Rahat Alma A. M., Reeve Richard, Ritsos Panagiotis D., Roberts Jonathan C., Slingsby Aidan, Swallow Ben, Torsney-Weir Thomas, Turkay Cagatay, Turner Robert, Vidal Franck P., Wang Qiru, Wood Jo and Xu Kai 2022Correction to: 'Visualization for epidemiological modelling: challenges, solutions, reflections and recommendations' (2022) by Dykes et al.Phil. Trans. R. Soc. A.38020220296http://doi.org/10.1098/rsta.2022.0296SectionOpen AccessCorrectionCorrection to: 'Visualization for epidemiological modelling: challenges, solutions, reflections and recommendations' (2022) by Dykes et al. Jason Dykes Jason Dykes Google Scholar Find this author on PubMed Search for more papers by this author , Alfie Abdul-Rahman Alfie Abdul-Rahman Google Scholar Find this author on PubMed Search for more papers by this author , Daniel Archambault Daniel Archambault Google Scholar Find this author on PubMed Search for more papers by this author , Benjamin Bach Benjamin Bach Google Scholar Find this author on PubMed Search for more papers by this author , Rita Borgo Rita Borgo Google Scholar Find this author on PubMed Search for more papers by this author , Min Chen Min Chen Google Scholar Find this author on PubMed Search for more papers by this author , Jessica Enright Jessica Enright Google Scholar Find this author on PubMed Search for more papers by this author , Hui Fang Hui Fang Google Scholar Find this author on PubMed Search for more papers by this author , Elif E. Firat Elif E. Firat Google Scholar Find this author on PubMed Search for more papers by this author , Euan Freeman Euan Freeman Google Scholar Find this author on PubMed Search for more papers by this author , Tuna Gönen Tuna Gönen Google Scholar Find this author on PubMed Search for more papers by this author , Claire Harris Claire Harris Google Scholar Find this author on PubMed Search for more papers by this author , Radu Jianu Radu Jianu Google Scholar Find this author on PubMed Search for more papers by this author , Nigel W. John Nigel W. John Google Scholar Find this author on PubMed Search for more papers by this author , Saiful Khan Saiful Khan Google Scholar Find this author on PubMed Search for more papers by this author , Andrew Lahiff Andrew Lahiff Google Scholar Find this author on PubMed Search for more papers by this author , Robert S. Laramee Robert S. Laramee Google Scholar Find this author on PubMed Search for more papers by this author , Louise Matthews Louise Matthews Google Scholar Find this author on PubMed Search for more papers by this author , Sibylle Mohr Sibylle Mohr Google Scholar Find this author on PubMed Search for more papers by this author , Phong H. Nguyen Phong H. Nguyen Google Scholar Find this author on PubMed Search for more papers by this author , Alma A. M. Rahat Alma A. M. Rahat Google Scholar Find this author on PubMed Search for more papers by this author , Richard Reeve Richard Reeve Google Scholar Find this author on PubMed Search for more papers by this author , Panagiotis D. Ritsos Panagiotis D. Ritsos Google Scholar Find this author on PubMed Search for more papers by this author , Jonathan C. Roberts Jonathan C. Roberts Google Scholar Find this author on PubMed Search for more papers by this author , Aidan Slingsby Aidan Slingsby Google Scholar Find this author on PubMed Search for more papers by this author , Ben Swallow Ben Swallow Google Scholar Find this author on PubMed Search for more papers by this author , Thomas Torsney-Weir Thomas Torsney-Weir Google Scholar Find this author on PubMed Search for more papers by this author , Cagatay Turkay Cagatay Turkay Google Scholar Find this author on PubMed Search for more papers by this author , Robert Turner Robert Turner Google Scholar Find this author on PubMed Search for more papers by this author , Franck P. Vidal Franck P. Vidal Google Scholar Find this author on PubMed Search for more papers by this author , Qiru Wang Qiru Wang Google Scholar Find this author on PubMed Search for more papers by this author , Jo Wood Jo Wood Google Scholar Find this author on PubMed Search for more papers by this author and Kai Xu Kai Xu Google Scholar Find this author on PubMed Search for more papers by this author Jason Dykes Jason Dykes Google Scholar Find this author on PubMed , Alfie Abdul-Rahman Alfie Abdul-Rahman Google Scholar Find this author on PubMed , Daniel Archambault Daniel Archambault Google Scholar Find this author on PubMed , Benjamin Bach Benjamin Bach Google Scholar Find this author on PubMed , Rita Borgo Rita Borgo Google Scholar Find this author on PubMed , Min Chen Min Chen Google Scholar Find this author on PubMed , Jessica Enright Jessica Enright Google Scholar Find this author on PubMed , Hui Fang Hui Fang Google Scholar Find this author on PubMed , Elif E. Firat Elif E. Firat Google Scholar Find this author on PubMed , Euan Freeman Euan Freeman Google Scholar Find this author on PubMed , Tuna Gönen Tuna Gönen Google Scholar Find this author on PubMed , Claire Harris Claire Harris Google Scholar Find this author on PubMed , Radu Jianu Radu Jianu Google Scholar Find this author on PubMed , Nigel W. John Nigel W. John Google Scholar Find this author on PubMed , Saiful Khan Saiful Khan Google Scholar Find this author on PubMed , Andrew Lahiff Andrew Lahiff Google Scholar Find this author on PubMed , Robert S. Laramee Robert S. Laramee Google Scholar Find this author on PubMed , Louise Matthews Louise Matthews Google Scholar Find this author on PubMed , Sibylle Mohr Sibylle Mohr Google Scholar Find this author on PubMed , Phong H. Nguyen Phong H. Nguyen Google Scholar Find this author on PubMed , Alma A. M. Rahat Alma A. M. Rahat Google Scholar Find this author on PubMed , Richard Reeve Richard Reeve Google Scholar Find this author on PubMed , Panagiotis D. Ritsos Panagiotis D. Ritsos Google Scholar Find this author on PubMed , Jonathan C. Roberts Jonathan C. Roberts Google Scholar Find this author on PubMed , Aidan Slingsby Aidan Slingsby Google Scholar Find this author on PubMed , Ben Swallow Ben Swallow Google Scholar Find this author on PubMed , Thomas Torsney-Weir Thomas Torsney-Weir Google Scholar Find this author on PubMed , Cagatay Turkay Cagatay Turkay Google Scholar Find this author on PubMed , Robert Turner Robert Turner Google Scholar Find this author on PubMed , Franck P. Vidal Franck P. Vidal Google Scholar Find this author on PubMed , Qiru Wang Qiru Wang Google Scholar Find this author on PubMed , Jo Wood Jo Wood Google Scholar Find this author on PubMed and Kai Xu Kai Xu Google Scholar Find this author on PubMed Published:12 September 2022https://doi.org/10.1098/rsta.2022.0296This article corrects the followingResearch ArticleVisualization for epidemiological modelling: challenges, solutions, reflections and recommendationshttps://doi.org/10.1098/rsta.2021.0299 Jason Dykes, Alfie Abdul-Rahman, Daniel Archambault, Benjamin Bach, Rita Borgo, Min Chen, Jessica Enright, Hui Fang, Elif E. Firat, Euan Freeman, Tuna Gönen, Claire Harris, Radu Jianu, Nigel W. John, Saiful Khan, Andrew Lahiff, Robert S. Laramee, Louise Matthews, Sibylle Mohr, Phong H. Nguyen, Alma A. M. Rahat, Richard Reeve, Panagiotis D. Ritsos, Jonathan C. Roberts, Aidan Slingsby, Ben Swallow, Thomas Torsney-Weir, Cagatay Turkay, Robert Turner, Franck P. Vidal, Qiru Wang, Jo Wood and Kai Xu volume 380issue 2233Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences15 August 2022 Phil. Trans. R. Soc. A 380, 20210299. (Published online 15 August 2022). (https://doi.org/10.1098/rsta.2021.0299) In the original version of this article, references 113–120, 123–140 and 143 were incorrectly numbered. This has been corrected on the publisher's website. Previous Article VIEW FULL TEXT DOWNLOAD PDF FiguresRelatedReferencesDetailsRelated articlesVisualization for epidemiological modelling: challenges, solutions, reflections and recommendations15 August 2022Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences This Issue31 October 2022Volume 380Issue 2235Theme issue 'Theory, modelling and observations of marginal ice zone dynamics: multidisciplinary perspectives and outlooks' compiled and edited by Luke G. Bennetts, Cecilia M. Bitz, Daniel L. Feltham, Alison L. Kohout and Michael H. Meylan Article InformationDOI:https://doi.org/10.1098/rsta.2022.0296PubMed:36088934Published by:Royal SocietyPrint ISSN:1364-503XOnline ISSN:1471-2962History: Manuscript received23/08/2022Manuscript accepted23/08/2022Published online12/09/2022Published in print31/10/2022 License:© 2022 The Authors.Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited. Citations and impact Subjectscomputer modelling and simulationhuman-computer interaction

This paper describes the integration of haptics support into a virtual reality training simulation aimed at skills retention for paramedics. We focus on a chest decompression, a life-saving invasive procedure used for trauma-associated cardiopulmonary resuscitation (and other causes) that every emergency physician needs to master. It is not regularly performed by a paramedic, however, and therefore skills maintenance is a challenge. In our simulation, a virtual Russell PneumoFix-8 device is used to carry out the procedure and it is controlled with the 3D Systems Touch grounded force feedback device. We describe how this device has been integrated into an immersive virtual environment so that it or any other tool can be used at any location in the scene. Quantitative data has been obtained from an evaluation exercise carried out with 21 paramedics. The majority of these participants reported a good feeling of presence, according to the Spatial Presence Experience Scale. They indicated strongly that the use of haptic-enabled simulators that include the kind of interaction techniques implemented in our simulator would be beneficial for training and skills retention. The realism of using the simulator at a 1 to 1 scale was also highly scored. A System Usability Scale was also calculated and the results show that the simulator is close to an acceptable standard for usability but more work is needed. We will address this in future work.

We report on an ongoing collaboration between epidemiological modellers and visualization researchers by documenting and reflecting upon knowledge constructs-a series of ideas, approaches and methods taken from existing visualization research and practice-deployed and developed to support modelling of the COVID-19 pandemic. Structured independent commentary on these efforts is synthesized through iterative reflection to develop: evidence of the effectiveness and value of visualization in this context; open problems upon which the research communities may focus; guidance for future activity of this type and recommendations to safeguard the achievements and promote, advance, secure and prepare for future collaborations of this kind. In describing and comparing a series of related projects that were undertaken in unprecedented conditions, our hope is that this unique report, and its rich interactive supplementary materials, will guide the scientific community in embracing visualization in its observation, analysis and modelling of data as well as in disseminating findings. Equally we hope to encourage the visualization community to engage with impactful science in addressing its emerging data challenges. If we are successful, this showcase of activity may stimulate mutually beneficial engagement between communities with complementary expertise to address problems of significance in epidemiology and beyond. See https://ramp-vis.github.io/RAMPVIS-PhilTransA-Supplement/. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

We present VRIA, a Web-based framework for creating Immersive Analytics (IA) experiences in Virtual Reality. VRIA is built upon WebVR, A-Frame, React and D3.js, and offers a visualization creation workflow which enables users, of different levels of expertise, to rapidly develop Immersive Analytics experiences for the Web. The use of these open-standards Web-based technologies allows us to implement VR experiences in a browser and offers strong synergies with popular visualization libraries, through the HTML Document Object Model (DOM). This makes VRIA ubiquitous and platform-independent. Moreover, by using WebVR's progressive enhancement, the experiences VRIA creates are accessible on a plethora of devices. We elaborate on our motivation for focusing on open-standards Web technologies, present the VRIA creation workflow and detail the underlying mechanics of our framework. We also report on techniques and optimizations necessary for implementing Immersive Analytics experiences on the Web, discuss scalability implications of our framework, and present a series of use case applications to demonstrate the various features of VRIA. Finally, we discuss current limitations of our framework, the lessons learned from its development, and outline further extensions.

: Rupture risk assessment is a key to devise patient-specific treatment plans of cerebral aneurysms. To understand and predict the development of aneurysms and other vascular diseases over time, both hemodynamic flow patterns and their effect on the vessel surface need to be analyzed. Flow structures close to the vessel wall often correlate directly with local changes in surface parameters, such as pressure or wall shear stress. However, especially for the identification of specific blood flow characteristics that cause local startling parameters on the vessel surface, like elevated pressure values, an interactive analysis tool is missing. In order to find meaningful structures in the entirety of the flow, the data has to be filtered based on the respective explorative aim. Thus, we present a combination of visualization, filtering and interaction techniques for explorative analysis of blood flow with a focus on the relation of local surface parameters and underlying flow structures. In combination with a filtering-based approach, we propose the usage of evolutionary algorithms to reduce the overhead of computing pathlines that do not contribute to the analysis, while simultaneously reducing the undersampling artifacts. We present clinical cases to demonstrate the benefits of both our filter-based and evolutionary approach and showcase its potential for patient-specific treatment plans.

Mild Cognitive Impairment (MCI) is a definition of the diagnosis of early memory loss and disorientation. This study aims to identify people's symptoms through technology. However, machine learning (ML) can classify Cognitive Normal (CN) and Mild Cognitive Impairment (MCI) and Early Mild Cognitive Impairment (EMCI) using standard assessments from the Alzheimer's Disease Neuroimaging Initiative (ADNI); Montreal Cognitive (MoCA), Mini-Mental State Examination (MMSE), Functional Activities Questionnaire (FAQ). Consequently, a Multilayer Perceptron (MLP) model was assembled into tables; MCI vs CN, MCI vs EMCI, and CN vs MCI. Additionally, an MLP model was developed for CN vs MCI vs EMCI. As a result, of advanced model performance, a cascade 3-path categorisation approach was created. Similarly, the exploitation of meta-analysis indicated a combination of MLP models (MCI vs CN, MCI vs EMCI, and CN vs MCI) with an overall accuracy within an acceptable limit. In addition, better results were found when assessments were combined rather than individually. Furthermore, applying class weights and probability thresholds could improve the MLP framework by performance achieving a balanced specificity and sensitivity ratio. Altering class weights and probability thresholds when training the MLP neuro network model, the sensitivity and Accuracy could be progressed further. In conclusion, ML, VR and electrodermal activity are constrained. Introducing the possibility of activity-based applications to enhance innovative solutions for cognitive impairment diagnosis and treatment.

The UK average survival rate from out of hospital cardiac arrest (OHCA) is 8.6%, which is significantly lower than in comparable countries where survival rates can exceed 20%. A cardiac arrest victim is two to four times more likely to survive OHCA with bystander cardiopulmonary resuscitation (CPR). Mandatory teaching of CPR in schools is an effective way, endorsed by the World Health Organization, to train the entire population and improve the bystander CPR rate. Despite this, as with other UK home nations, there is significant variation in provision of CPR training within schools in Wales. Virtual reality (VR) technology offers an accessible, immersive way to teach CPR skills to schoolchildren. Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust developed Virtual Cardio Pulmonary Resuscitation (VCPR), which can be used to teach children CPR skills. There were three stages: identifying requirements and specifications; development of a prototype; and management—development of software, further funding and exploring opportunities for commercialisation.

This is the accepted version of the following article: John, N.W., Day, T.W., & Wardle, T. (2020). An Endoscope Interface for Immersive Virtual Reality. Eurographics Workshop on Visualization for Biology and Medicine, Eurographics Association, which has been published in final form at http://onlinelibrary.wiley.com. This article may be used for non-commercial purposes in accordance with the Wiley Self-Archiving Policy

Virtual reality entertainment and serious games popularity has continued to rise but the processes for level design for VR games has not been adequately researched. Our paper contributes LevelEd VR; a generic runtime virtual reality level editor that supports the level design workflow used by developers and can potentially support user generated content. We evaluated our LevelEd VR application and compared it to an existing workflow of Unity on a desktop. Our current research indicates that users are accepting of such a system, and it has the potential to be preferred over existing workflows for VR level design. We found that the primary benefit of our system is an improved sense of scale and perspective when creating the geometry and implementing gameplay. The paper also contributes some best practices and lessons learned from creating a complex virtual reality tool, such as LevelEd VR.

Background: Virtual reality (VR) technology is emerging as a powerful tool in medical training and has potential benefits for paramedic education. Aim: The aim of this paper is to report the development of ParaVR, which uses VR to maintain paramedics' skills. Methods: Computer scientists at the University of Chester and the Welsh Ambulance Services NHS Trust (WAST) developed ParaVR in four stages: identifying requirements and specifications; alpha version development; beta version development; and management—development of software, further funding and commercialisation. Results: Needle cricothyrotomy and needle thoracostomy emerged as candidates for the prototype ParaVR. The Oculus Rift head-mounted display was combined with Novint Falcon haptic device and a virtual environment crafted using 3D modelling software, which was ported to the Oculus Go virtual reality headset and the Google Cardboard VR platform. Conclusion: VR is an emerging educational tool with the potential to enhance paramedic skills development and maintenance. The ParaVR programme is the first step in the authors' development, testing and scaling up of this technology.

We report on the design, implementation and evaluation of , a framework for building immersive analytics (IA) solutions in Web-based Virtual Reality (VR), built upon WebVR, A-Frame, React and D3. The recent emergence of affordable VR interfaces have reignited the interest of researchers and developers in exploring new, immersive ways to visualize data. In particular, the use of open-standards web-based technologies for implementing VR in a browser facilitates the ubiquitous and platform-independent adoption of IA systems. Moreover, such technologies work in synergy with established visualization libraries, through the HTML document object model (DOM). We discuss high-level features of and present a preliminary user experience evaluation of one of our use cases.

With the rise in popularity of serious games there is an increasing demand for virtual environments based on realworld locations. Emergency evacuation or fire safety training are prime examples of serious games that would benefit from accurate location depiction together with any application involving personal space. However, creating digital indoor models of real-world spaces is a difficult task and the results obtained by applying current techniques are often not suitable for use in real-time virtual environments. To address this problem, we have developed an application called LevelEd AR that makes indoor modelling accessible by utilizing consumer grade technology in the form of Apple’s ARKit and a smartphone. We compared our system to that of a tape measure and a system based on an infra-red depth sensor and application. We evaluated the accuracy and efficiency of each system over four different measuring tasks of increasing complexity. Our results suggest that our application is more accurate than the depth sensor system and as accurate and more time efficient as the tape measure over several tasks. Participants also showed a preference to our LevelEd AR application over the depth sensor system regarding usability. Finally, we carried out a preliminary case study that demonstrates how LevelEd AR can be successfully used as part of current industry workflows for serious games level design.

We present ongoing work to develop a virtual reality environment for the cognitive rehabilitation of patients as a part of their recovery from a stroke. A stroke causes damage to the brain and problem solving, memory and task sequencing are commonly affected. The brain can recover to some extent, however, and stroke patients have to relearn to carry out activities of daily learning. We have created an application called VIRTUE to enable such activities to be practiced using immersive virtual reality. Gamification techniques enhance the motivation of patients such as by making the level of difficulty of a task increase over time. The design and implementation of VIRTUE is presented together with the results of a small acceptability study.

This research project developed a Virtual Reality (VR) training simulator for paramedic procedures. Currently needle cricothyroidotomy and chest drain are modelled, which could form part of a larger system for training paramedics with VR in various other procedures. The simulator incorporates a number of advanced VR technologies including Oculus Rift and haptic feedback. We have gained input and feedback from NHS paramedics and several related organisation to design the system and provide feedback and evaluation of the preliminary working prototype.

We developed an application that makes indoor modelling accessible by utilizing consumer grade technology in the form of Apple's ARKit and a smartphone to assist with serious games level design. We compared our system to that of a tape measure and a system based on an infra-red depth sensor and application. We evaluated the accuracy and efficiency of each system over four different measuring tasks of increasing complexity. Our results suggest that our application is more accurate than the depth sensor system and as accurate and more time efficient as the tape measure over several tasks. Participants also showed a preference to our LevelEd AR application over the depth sensor system regarding usability.

This research project developed a Virtual Reality (VR) training simulator for the CPR procedure. This is designed for use training school children. It can also form part of a larger system for training paramedics with VR. The simulator incorporates a number of advanced VR technologies including Oculus Rift and Leap motion. We have gained input from NHS paramedics and several related organisation to design the system and provide feedback and evaluation of the preliminary working prototype.

We describe a mixed reality environment that has been designed as an aid for training driving skills for a powered wheelchair. Our motivation is to provide an improvement on a previous virtual reality wheelchair driving simulator, with a particular aim to remove any cybersickness effects. The results of a validation test are presented that involved 35 able bodied volunteers divided into three groups: mixed reality trained, virtual reality trained, and a control group. No significant differences in improvement was found between the groups but there is a notable trend that both the mixed reality and virtual reality groups improved more than the control group. Whereas the virtual reality group experienced discomfort (as measured using a simulator sickness questionnaire), the mixed reality group experienced no side effects.

Background and objective: While Minimally Invasive Surgery (MIS) offers considerable benefits to patients, it also imposes big challenges on a surgeon’s performance due to well-known issues and restrictions associated with the field of view (FOV), hand-eye misalignment and disorientation, as well as the lack of stereoscopic depth perception in monocular endoscopy. Augmented Reality (AR) technology can help to overcome these limitations by augmenting the real scene with annotations, labels, tumour measurements or even a 3D reconstruction of anatomy structures at the target surgical locations. However, previous research attempts of using AR technology in monocular MIS surgical scenes have been mainly focused on the information overlay without addressing correct spatial calibrations, which could lead to incorrect localization of annotations and labels, and inaccurate depth cues and tumour measurements. In this paper, we present a novel intra-operative dense surface reconstruction framework that is capable of providing geometry information from only monocular MIS videos for geometry-aware AR applications such as site measurements and depth cues. We address a number of compelling issues in augmenting a scene for a monocular MIS environment, such as drifting and inaccurate planar mapping. Methods: A state-of-the-art Simultaneous Localization And Mapping (SLAM) algorithm used in robotics has been extended to deal with monocular MIS surgical scenes for reliable endoscopic camera tracking and salient point mapping. A robust global 3D surface reconstruction framework has been developed for building a dense surface using only unorganized sparse point clouds extracted from the SLAM. The 3D surface reconstruction framework employs the Moving Least Squares (MLS) smoothing algorithm and the Poisson surface reconstruction framework for real time processing of the point clouds data set. Finally, the 3D geometric information of the surgical scene allows better understanding and accurate placement AR augmentations based on a robust 3D calibration. Results: We demonstrate the clinical relevance of our proposed system through two examples: (a) measurement of the surface; (b) depth cues in monocular endoscopy. The performance and accuracy evaluations of the proposed framework consist of two steps. First, we have created a computer-generated endoscopy simulation video to quantify the accuracy of the camera tracking by comparing the results of the video camera tracking with the recorded ground-truth camera trajectories. The accuracy of the surface reconstruction is assessed by evaluating the Root Mean Square Distance (RMSD) of surface vertices of the reconstructed mesh with that of the ground truth 3D models. An error of 1.24 mm for the camera trajectories has been obtained and the RMSD for surface reconstruction is 2.54 mm, which compare favourably with previous approaches. Second, in vivo laparoscopic videos are used to examine the quality of accurate AR based annotation and measurement, and the creation of depth cues. These results show the potential promise of our geometry-aware AR technology to be used in MIS surgical scenes. Conclusions: The results show that the new framework is robust and accurate in dealing with challenging situations such as the rapid endoscopy camera movements in monocular MIS scenes. Both camera tracking and surface reconstruction based on a sparse point cloud are effective and operated in real-time. This demonstrates the potential of our algorithm for accurate AR localization and depth augmentation with geometric cues and correct surface measurements in MIS with monocular endoscopes.