Exploring In-The-Wild Game-Based Gesture Data Collection

This paper presents an automatic 3D gesture collection concept and architecture based on a rhythm game for public displays. The system was implemented using an off-the-shelf gesture controller, was deployed on a public vertical screen, and was used to study the effects of alternative gesture guidance conditions. In the evaluation presented, we examined how alternative gesture guidance conditions affect users' engagement. The study showed that demonstration animation (CDA) and tracking state feedback (TSI) each encourages sustained game engagement. The underlying concept and architecture presented here offer actionable UI design insight to help creating large gesture corpus from diverse populations.