In the early stages of learning a technical skill, trainees require guidance from a mentor through augmented feedback to develop higher expertise. However, the impact of such feedback and the different modalities used to communicate it remain underexplored in immersive virtual environments (IVE). This paper presents a study in which 27 participants were divided into three groups to learn a tool manipulation trajectory in an IVE. Two experimental groups received guidance from an expert using visual and/or haptic augmented feedback, while the control group received no feedback. The results indicate that both experimental groups showed significantly greater improvement in tool trajectory performance than the control group from pre- to post-test, with no significant differences between them. Analysis of their learning curves revealed similar performance improvements in tool trajectory across trials, outperforming the control group. Additionally, the visual-haptic feedback condition was linked to lower task load in three out of six dimensions of the NASA-TLX and a higher perceived interdependence with the expert's actions. These findings suggest that augmented feedback from an expert enhances the learning of tool manipulation skills. Although adding haptic feedback did not lead to better learning outcomes compared to visual feedback alone, it did enhance the overall user experience. These results offer valuable insights for designing IVEs that support mentor-trainee interactions through augmented feedback.

Virtual reality shows great promise as a technology for training healthcare professionals within a secure simulated environment. This work presents the design, development, and assessment of UltRASim: an immersive simulator for ultrasound-guided regional anesthesia. First, task and skills analyses were performed with domain experts to build the task model of the procedure and determine the simulator’s learning objectives and design constraints. Then, a face and content validity study was conducted with eighteen anesthesiologists to assess the simulator’s prototype. The responses to seven of eleven face validity questions were predominantly positive, indicating a favorable reception. The primary concerns pertained to the fidelity of haptic feedback during needle insertion. This suggests incorporating a higher fidelity haptic device in future design iterations. Conversely, responses to all six questions related to the content validity were predominantly positive. Participants found that the simulator held significant potential as a training tool, particularly for developing hand–eye coordination skills. These findings validate several design choices and highlight areas for improvement in subsequent iterations of UltRASim before its formal validation as a training tool.

We study the impact of haptic feedback on basic technical skills transfer from VR to the real world. Twenty-four volunteers were divided into two training groups (haptic and no-haptic groups) and a control group. The training groups learned to perform a “Ring Transfer” task in a VR simulator, and all participants performed pre-, post, and retention tests on a similar physical setup. Results show that skill transfer is observed for both training groups and not for the control group. The haptic group participants also improved their performance compared to the no-haptic group, but the difference was not significant.

With the mentoring model, a mentee can learn technical skills under the supervision of more experienced peers who demonstrate their knowledge through several communication modalities. Supporting the mentoring model within shared immersive training simulators holds promise in enhancing mentor-mentee interactions and learning outcomes in a safe environment. However, efficient communication within these spaces remains an open issue. This work presents a user study that explores the combination of communication modalities (verbal-visual, verbal-haptic, visual-haptic, and verbal-visual-haptic) to convey instructions to learners on the amplitude of movements to perform during a tool-handling task in an immersive environment. The study aims to examine the impact of the four modality combinations on performance (speed and accuracy of movement replication), mental workload, and participants’ user experience. The results show that participants achieved higher accuracy with the visual-haptic and verbal-visual-haptic conditions. Moreover, they performed the movements faster, and their movement trajectories were closer to the reference trajectories in the visual-haptic condition. Finally, the most preferred verbal-visual-haptic combination enhanced the users’ sense of presence, co-presence, social presence, and learning experience. No impact on the mental workload was observed. These results suggest that combining haptic and visual modalities is the best suited for enhancing learners’ performance. Adding the verbal modality can also improve the user experience in the immersive learning environment. These findings contribute to improving the design of immersive collaborative systems and pave the way for exploring novel avenues of research into the efficacy of multimodal communication for enhancing the mentoring-based acquisition of technical skills in VR. These tools hold promise for diverse applications, including medical simulation.

La formation par compagnonnage permet aux novices d'acquérir des compétences sous la supervision d'experts qui utilisent diverses modalités de communication. Cependant, reproduire ce modèle dans des simulateurs immersifs reste un défi, notamment pour assurer une communication efficace entre experts et novices. Notre étude explore l'impact de la communication multimodale expert-novice pour transmettre des instructions sur l'amplitude des mouvements dans une tâche de manipulation d'outils en environnement immersif. Les résultats révèlent que la combinaison des modalités visuelle-haptique améliore la précision, la vitesse et la qualité des mouvements. De plus, la combinaison verbale-visuelle-haptique renforce le sentiment de présence et de coprésence, et l'expérience d'apprentissage. Ces résultats suggèrent que la combinaison visuelle-haptique est optimale pour améliorer les performances des novices, et que l'intégration de la modalité verbale améliore l'expérience utilisateur. Ces conclusions ouvrent de nouvelles perspectives pour améliorer l'acquisition de gestes techniques par compagnonnage en réalité virtuelle grâce à la communication multimodale.

This work investigates the potential benefits of using a shared immersive environment for training purposes. Such an environment provides a safe space for teachers to impart their knowledge and expertise to trainees, especially when teaching technical skills that require proper tool manipulation. Our research focuses on exploring different communication modalities that can be used to teach movement amplitude during tool manipulation tasks. Specifically, we examine the effectiveness of haptic, visual, and verbal modalities in enhancing the learning process. Our user study results reveal that trainees were able to replicate movements more accurately when given instructions using the visual modality, and they were able to replicate movements faster when given instructions using the haptic modality. While verbal instructions increased the sense of copresence with the teacher, it was the least preferred modality. These findings suggest that multimodality could be the most appropriate approach to enhance the teaching of movement amplitude skills. Our study provides insights for improving the design of immersive shared systems. It opens up new avenues for further research on the effectiveness of shared immersive virtual environments in supporting the teaching of technical skills.

We present the design and evaluation of an immersive ultrasound-guided locoregional anesthesia simulator. A face and content validation study with eighteen anesthesiologists was conducted. The results show that the developed system is a promising tool suited for developing hand-eye coordination skills. On the other hand, the study raised some issues related to the fidelity of the haptic feedback. These findings support our design choices and suggest improvements before the validation of the simulator.

We present an exploratory study to compare the haptic, visual, and verbal modalities for communicating distance information in a shared virtual environment. The results show that the visual modality decreased the distance estimation error while the haptic modality decreased the completion time. The verbal modality increased the sense of copresence but was the least preferred modality. These results suggest that a combination of modalities could improve communication of distance information to a partner. These findings can contribute to improving the design of collaborative VR systems and open new research perspectives on studying the effectiveness of multimodal interaction.

Shared immersive environments are used to teach technical skills and communicate relevant information. However, designing the appropriate interfaces and interactions to support this communication process remains an open issue. We explore using three modalities to communicate movement amplitude during tool manipulation tasks in a shared immersive environment. The haptic, visual, and verbal modalities were used separately to instruct a learner about the amplitude of the movements to perform in the 3D space. The user study comparing these modalities shows that instructions given through the visual modality permitted to decrease the distance estimation error. In contrast, the haptic modality helped the learners perform the task significantly faster. The verbal modality significantly increased the perceived sense of copresence but was the least preferred modality. This research contributes to understanding the importance of each modality when communicating spatial skills in a shared immersive environment. The results suggest that combining modalities could be the most appropriate way to transfer movement amplitude information to a learner by improving performance and user experience. These findings can enhance the design of immersive collaborative systems and open new perspectives for further research on the effectiveness of multimodal interaction to support learning technical skills in VR. Designed tools can be used in different fields, such as medical teaching applications.