Hand-posture-augmented multitouch interactions for exploratory visualization

Conventional multitouch-based visualization systems mostly use just the touch information elicited from the touch surface to support the visualization applications, while rich contextual and dynamic information contained in user's hand postures above the surface remains untapped. This paper addresses this issue by proposing a family of finger-and-hand gestures that can be robustly recognized and employed over a multitouch tabletop surface. We describe how these gestures can be recognized in real-time with the use of depth sensing, and suggest several examples of how touch-based information augmented with information like hand differentiation, hand posture discrimination and finger tilt dynamics can improve conventional exploratory visualization. We present two case studies to show how the augmented gestures can be employed in different exploratory visualization scenarios more effectively.