Hand Gestures for Intelligent Tutoring Systems: Dataset, Techniques & Evaluation

Analysis of hand gestures in one-to-one tutoring gives a number of characteristics of social interaction and behavior between the tutor and the student. This analysis can not only aid in understanding the effectiveness of the learning methodology and developing new techniques for learning, but also help in developing intelligent and online tutoring systems. Although there exists a comprehensive literature on recognizing hand gestures, there is limited work on recognizing such gestures in the context of one-to-one tutoring systems. In this paper, we first introduce a new dataset that comprises a set of 2166 richly labeled video sequences of multiple subjects, showing 4 different classes of most prominent gestures in one-to-one tutoring. In addition to the dataset, two methods comprising appearance based cues and motion based cues are proposed and evaluated on this dataset. A detection accuracy of over 53% is achieved when the proposed techniques are validated across 6 different subjects, which can be used as a benchmark for future works that can employ the proposed datasets for hand gestures for one-to-one tutoring systems.