A critical step towards leveraging machine learning to create models for adaptive team-based coaching is collecting a large corpus of training data that can serve as a source for inducing data-driven coaching policies. In this paper we describe ongoing work using the Generalized Intelligent Framework for Tutoring (GIFT) to build reinforcement learning (RL) based coaching policies for promoting team performance in the domain of crew gunnery training. We describe our approach towards collecting a corpus of multimodal training data including video, communication, and simulation-trace data from U.S. Army gunnery crews who are completing simulation-based training exercises to prepare for crew gunnery qualification and sustainment. The dataset will be used to induce data-driven coaching policies for promoting individual and crew gunnery performance. We are using a component of GIFT called GIFT Data Collector to collect multimodal training data from simulation-based training stations as U.S. Army gunnery crews complete their assigned training exercises using the Virtual Battlespace 3 (VBS3) simulation platform. We discuss the data analysis pipeline that the team is developing to support data formatting, cleaning, filtering, and feature extraction processes that will be used to induce coaching policies. We also discuss how we plan to utilize multimodal training data, including crew communication logs, to iteratively refine the domain assessment model to support individual and team performance analysis and state classification. We conclude with a discussion of our upcoming research activities that aim to evaluate the acceptance and effectiveness of the coaching policies in an empirical study.

In this paper we present the outcomes of a user centered qualitative usability evaluation across a set of tools and methods used to support a hybrid team intelligent tutoring strategy. Feedback was received across user tools designed to interface human trainers with adaptive instructional components used to monitor performance in real-time, promote reflection and discussion during a scenario review, and being able to explore performance and data over time to track competency development objectives. The methodology and results of the interviews are shared, with a discussion focused on insights and required features these tools require.

Intelligent learning environments can be designed to support the development of learners' cognitive skills, strategies, and metacognitive processes as they work on complex decision-making and problem-solving tasks. However, the complexity of the tasks may impede the progress of novice learners. Providing adaptive feedback to learners who face difficulties requires learner modeling approaches that can identify learners' proficiencies and the difficulties they face in executing required skills, strategies, and metacognitive processes. This paper discusses a multilevel hierarchical learner modeling scheme that analyzes and captures learners' cognitive processes and problem-solving strategies along with their performance on assigned tasks in a game-based environment called UrbanSim that requires complex decision making for dealing with counterinsurgency scenarios. As the scenario evolves in a turn-by-turn fashion, UrbanSim evaluates the learners' moves using a number of performance measures. Our learner modeling scheme interprets the reported performance values by analyzing the learners' activities captured in log files to derive learners' proficiencies in associated cognitive skills and strategies, and updates the learner model. We discuss the details of the learner modeling algorithms in this paper, and then demonstrate the effectiveness of our approach by presenting results from a study we conducted at Vanderbilt University.

A training ecosystem leverages multiple complementary instructional resources to target competency and skill development. In this paper, we introduce work that is integrating assessment functions in the Generalized Intelligent Framework for Tutoring (GIFT) with core components in the Total Learning Architecture (TLA) to support persistent performance tracking and reporting in dynamic simulation-based environments. This capability creates a data strategy to translate multi-modal raw data into contextualized statements of performance for use in a long-term readiness monitoring strategy. In this paper we discuss the integration activities, what this new extended architecture supports, and provide a high-level use case associated with infantry squad level competency sets.

The Generalized Intelligent Framework for Tutoring (GIFT; Sottilare, Goldberg, Brawner & Holden, 2012) is a framework and tool set for the creation of intelligent and adaptive tutoring systems (Brawner, 2012; Sottilare, Goldberg, Brawner & Holden, 2012). Since its inception, GIFT has become a standard for authoring, deploying, managing, and evaluating Intelligent tutoring system (ITS) technologies. In that time GIFT has pursued best practices for automated instruction, course authoring and sound instructional strategies, as well as to facilitate ongoing ITS research. With GIFT, users can create tutors with domain agnostic tools that vary from simple content delivery to adaptive and individualized learner experiences. To date, GIFT has already been used across various domains from existing external simulations, serious games, and computer-based training environments to teach physics, train military tasks and tactics, and solve cognitive problems.

The Engine for Management of Adaptive Pedagogy (EMAP) is the Generalized Intelligent Framework for Tutoring's (GIFT) first implementation of a domain-independent pedagogical manager. It establishes a framework within GIFT that adheres to sound instructional system design, while also providing tools and methods to create highly personalized and adaptive learning experiences. In this paper, we present the components of the EMAP, we high- light their utility when authoring an EMAP managed lesson, and we review the limitations associated with its first instantiation.