Using virtual reality to assess competence in abdominal point-of-care ultrasound

Objective This study aimed to develop a virtual reality test for abdominal point-of-care competence, to gather validity evidence for the test, and to establish a pass/fail score. Methods The developed test consisted of four abdominal point-of-care ultrasound cases. Medical students and doctors with varying abdominal point-of-care ultrasound experience were invited to the test and divided into three study groups: Novices, intermediates, and experienced abdominal point-of-care ultrasound operators. Data from the following items were used for item analysis and to examine internal consistency: The ability to correctly enter patient identification, orientate the ultrasound probe, and select a patient diagnosis (hydronephrosis, cholecystitis, gallbladder stones, abdominal aortic aneurysm, and/or urine retention). The contrasting groups’ standard setting method was used to establish a pass/fail score. Results Thirty-one participants were included in the test. The item analysis included 49 items and a credible pass/fail score of 31 points was established (minimum of 0 points, maximum of 49 points). A one-way ANOVA was used to compare the mean test scores between the groups and showed significant difference between all three groups (p<0.001). Internal consistency was high (Cronbach’s alpha of 0.91), and an independent t-test showed statistically significant difference between the total scores of novices (mean= 23.7 points; SD= 8.7) and experienced operators (mean= 39.1 points; SD= 7.9), p<0.001. Conclusion We developed a test for abdominal point-of-care ultrasound competence assessment in virtual reality. Solid validity evidence was gathered, a credible pass/fail score was established, and the test could distinguish between novices and experienced abdominal point-of-care ultrasound operators.