Augmented reality as means for creating shared understanding

The motivation for the work presented in this paper comes primarily from user experience of video-conferencing (v-c) settings in real-life collaboration. The design issues in this context focus on making such settings interactive enough to support natural communication and collaboration. The initial assumption is that users in an interactive v-c setting should be able to navigate the remote space in order to establish clear reference by pointing to people and objects in it. Clear reference to parts of the context in which conversations take place (that is, deictic reference) is an important factor in effective commu-nication. With this aim, we enhanced the videoconferencing system with the ability to visualize abstract representations of pointers and investigated pointing gesture as a tool for collaborative referring. We thus designed a prototype that combines the communicative function of pointing gesture with a hybrid representation of real video and virtual objects (pointers) that identify particular parts of it. A game controller was employed for pointing and Augmented Reality (AR) for visualizing the referent in live video stream. Usability tests were run on two versions of the prototype using five common types of joint task. Evidence based on data from video recording, questionnaire, and interview, shows effectiveness of the system in mediating the communicative function of pointing. Test users adapted to the context of interpretation quickly. Feedback was provided for enhancing visualization and pointing technique. The method successfully captured relevant visuo-gestural and linguistic aspects of communication to inform design.