Adapting Usability Metrics for a Socially Assistive, Kinesthetic, Mixed Reality Robot Tutoring Environment

The field of Socially Assistive Robot (SAR) tutoring has extensively explored both subjective and objective usability metrics for seated tablet-based human-robot interactions. As SAR tutoring introduces kinesthetic mixed reality environments where students can move around and physically manipulate virtual objects, usability metrics for such interactions need to be re-evaluated. This paper applies standard usability metrics from seated 2D interactions to a kinesthetic mixed reality environment and validates those metrics with post-interaction survey data. Using data from a pilot study (n = 9) conducted with a mixed reality SAR tutor, three commonly-used metrics of usability for seated 2D tutoring interfaces were collected: performance, manipulation time, and gaze. The strength of each usability metric was compared to subjective survey-based scores measured with the System Usability Scale (SUS). The results show that usability scores were correlated with the gaze metric but not with the manipulation time or performance metrics. The findings provide interesting implications for the design and evaluation of kinesthetic mixed reality robot tutoring environments.