Touch-Move-Release: Studies of Surface and Motion Gestures for Mobile Augmented Reality

Despite a broad range of mobile AR applications available to date, the majority still primarily use surface gestures, i.e., gesturing on the touch screen surface of the device and do not utilise the affordance of three-dimensional user interaction that AR interfaces support. In this research, we compared two methods of gesture interaction for mobile AR applications: Surface Gestures and Motion Gestures, which take advantage of the spatial information of the mobile device. We have conducted two user studies: an elicitation study (n = 21) and a validation study (n = 10). The first study elicited two sets of 504 gestures, surface and motion gestures, for 12 everyday mobile AR tasks. The two sets of gestures were classified and compared in terms of goodness, ease of use, and engagement. As expected, the participants’ elicited surface gestures are familiar and easy to use, while motion gestures were found more engaging. Using design patterns derived from the elicited motion gestures, we proposed a novel interaction technique called “TMR” (Touch-Move-Release). Through validation study we found that the TMR motion gesture enhanced engagement and provided a better game experience. In contrast, the surface gesture provided higher precision resulting in higher accuracy and was easier to use. Finally, we discuss the implications of our findings and give our design recommendations for using the elicited gestures.