An Approach for Detecting Gaming the System Behavior in Programming Problem-Solving

In ITS, for students’ success in carrying out their activities, it is essential to provide scaffolding, such as hints and feedback. Although using the ITS has increased student engagement and effort in the classroom, some students have responded to the ITS’s support facilities with an inappropriate behavior called gaming the system. Thus, in this article, we explore the phenomenon of gaming the system behavior, studying underlying factors related to automatically detecting when students game the system. We aim to develop a model for detecting gaming the system behavior in a computer programming student during problem-solving activities. Particularly, we have had special attention on testing the influence of variables such as (i) the student’s level of belief and the system’s level of belief in the difficulty level of the problem in detecting gaming the system; and (ii) partial submission of the problem associated with complete submission, in program form, in the detection of system manipulation. To conduct the detector’s development, we developed an appropriate environment for data collection and preparation of these data to be used by supervised machine learning algorithms, allowing the detection of the behavior. The development of the detector involved training single and ensemble machine learning algorithms to classify the system’s gaming behavior and obtain models with the best accuracy, including exploring, evaluating, and comparing different algorithms. The results show, considering the average of all algorithm results concerning all measures, the XGBoost ensemble classifier achieved the best performance.