Team Training for Enemy Identification Using an Intelligent Tutoring System

Team training has been identified as critical to the operations of the Department of Defense (DoD) due to the complex and frequent interactions required in military teams. Effective training is necessary to develop complete understandings of the task and to build cooperative teams. Intelligent Team Tutoring Systems (ITTSs) have the potential to reduce training costs, improve learning, and increase feedback consistency in comparison to traditional human tutors. Currently, ITTSs are underdeveloped due to the state of the technology and the complex nature of intelligent agents, which require a variety of considerations and many hours to create. In this paper, the authors explore the impact of automated tutor feedback and team composition on performance for participants tasked with identifying and tracking enemy combatants. Thirty-seven three-person teams, each composed of two spotters and one sniper, were tutored on their surveillance task performance over four trials. The scenario was constructed using Virtual Battle Space 2.0 (VBS2) and a version of the Generalized Intelligent Framework for Tutoring (GIFT), which assessed learners and delivered real-time feedback. In 18 teams, members received private, individualized feedback, while in 19 teams, members received individualized public feedback (i.e., their teammates could observe). Additionally, all teams experienced a change in team composition as the sniper and one of the surveillance spotters traded roles for the fourth trial. Each team’s performance in the task was assessed. Evidence of training effectiveness is observed in participants’ subjective performance and task workload. While feedback privacy was not found to influence the subjective performance, an effect was found for objective performance. These results about the effectiveness of feedback in team settings will influence the future study and development of ITTSs for the military by adding to the literature on how automated feedback should be designed within team training settings. Disciplines Communication | Human Factors Psychology | Operations Research, Systems Engineering and Industrial Engineering Comments This is a proceeding published as Kaitlyn M. Ouverson, Alec Ostrander, Anastacia MacAllister, Adam Kohl, Jamiahus Walton, Stephen B. Gilbert, Michael C. Dorneich, Eliot Winer, and Anne M. Sinatra. "Team Training for Enemy Identification Using an Intelligent Tutoring System." In: Proceedings of the 2018 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). Paper no. 18291. Authors Kaitlyn Ouverson, Alec Ostrander, Anastacia MacAllister, Adam Kohl, Jamiahus Walton, Stephen B. Gilbert, Michael C. Dorneich, Eliot H. Winer, and Anne M. Sinatra This conference proceeding is available at Iowa State University Digital Repository: https://lib.dr.iastate.edu/ imse_conf/225 Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2018 2018 Paper No. 18291 Page 1 of 14 Team Training for Enemy Identification Using an Intelligent Tutoring System Kaitlyn M. Ouverson, Alec Ostrander, Anastacia MacAllister, Adam Kohl, Jamiahus Walton, Stephen B. Gilbert, Michael C. Dorneich, Eliot Winer Anne M. Sinatra STTC Iowa State University Ames, IA Orlando, FL {kmo, alecglen, anastac, adamkohl, anastac, alecglen, adamkohl, jwalton} @iastate.edu anne.m.sinatra.civ @mail.mil