There are a variety of technologies needed to create a fully synthetic training experience from the “bottom up” (ie. “a fully created synthetic experience”) and from the “top down” (ie. created for the user who needs training). This paper outlines the technologies required with the note that many of them currently exist in research form; it describes the world soon to come to computer aided instruction.

This poster paper will discuss a variety of items related to Human-Machine Teaming and research in support in-creasing control of autonomous machines present in phys-ical problem domains of interest. Many military tasks can be decomposed into their primary elements – intelligence preparation, reconnaissance, movement, maneuver, fires, and and support across the combat domains of interest – air, land, sea, etc. There are an increasing number of au-tonomous and semi-autonomous ground-based systems available, such as the Multi-Utility Tactical Transport (MUTT) Unmanned Ground Vehicle, for movement of materials, or Quadrupedal Unmanned Ground Vehicle (QUGV) for the disposal of explosive ordinance, comple-mented by aerial platforms considering of a wide variety of Unmanned Aerial Systems (UAS) for the gathering and transmitting of information, held together by a common backbone and network. The preponderance of new systems and capabilities brings new issues, one of which is critical to research - how to control and manage a large number of systems. Commercial systems of significantly less capability, such as light-up drone shows, involve approximately 2 people per 100 drones; meaning that a single controller for a sin-gle drone is simply a non-starter. How can research be applied to scale the complexity of operations upwards without additional demands of personnel? This poster discusses several portions of early-stage research into a variety of applications, including: Reasoning about mixed-team processes, includ-ing real/synthetic teammates, and how infor-mation gathered about the use of synthetic teammates within simulation can be transitioned into utilization for a robotic teammate Embedding affective information into dialogue channels in order to save cognitive bandwidth Repurposing of foundational dialogue models for specific tasks and purposes Utilizing psychological research to design sys-tems in order to infer user intent Theory of mind research for autonomous systems regarding their human operators Simulated environments and agents in order to test the simulations in representative areas. Common-Sense reasoning augmented by Large Language Model technology for instructing ro-botic platforms The following things will be discussed at the poster ses-sion, representing a portfolio of ongoing work addressing the problems of large-scale multi-agent command and control. While the near-term application of these tech-nologies is into military problem domains; the poster presentation is cleared for public release.

The emergence of widely-used artificial intelligence (AI) has created a critical need for AI expertise, not just as a research area but for workers in the wide variety of careers and roles that AI disrupts. While AI is still an area of research for new processing, application, and development – it continues to partially automate, augment, or replace many of the tasks which are performed through active use of human hands. While recently publicized items such as ChatGPT and MidJourney have made press in their adjustment to writing and image generation technology, the basic workflow of copyeditors and digital artists was completely transformed, inside of the year, to a combination of partially automated or fully automated AI tasks. While some blame AI as part of the “problem”, it is naturally part of the “solution” – AI tools to help workers develop AI competencies. The paper describes an array of strategies which the DoD and its ICT UARC are using to address the fundamental problem of quickly upskilling the DoD workforce of over 2 million adult learners.

Artificially intelligent agents are enjoying increased adoption in both the video game and simulation industry - being used for both training and education as well as entertainment purposes with largescale real-time strategy games. Both type of systems have a need for the simulation of large numbers of units. While traditional scalability solutions (e.g. segmenting the terrain and using separate hardware to process various sections) can be used, this is wasteful of processing power which could have been better used to approximate the results of far-flung conflict while disaggregating close-in conflict for the benefit of the user/trainee. Approaches to do so haven’t been investigated due to the smaller scales of simulations until recent developments. This paper provides an introduction to a modeling approach that can be used to bridge optimal control techniques with wargaming AI agents in order to provide cohesive aggregated and disaggregated AI forces within simulations.

Artificially intelligent agents are seeing increased adoption in both the video game and simulation industry for training, education, and entertainment purposes. These systems often need realistic and believable opponents that must achieve objectives in the face of competing and contradictory priorities and frequently require the rapid creation of a wide spectrum of agents with disparate behaviors that reflect tactical realism. This in turn drives the need for the dynamic training of such agents from available source data. Approaches to do so have yet to be widely investigated due to the smaller scales of these simulation environments. This paper discusses techniques to quickly design and generate a variety of AI agents that follow desired tactics and procedures, including realistic situations that require trade-off decisions between competing objectives. Techniques described include an investigation into deep reinforcement agents that have separable reward structures and can prioritize and re-prioritize goals based on a hierarchy.

Training soldiers through virtual reality simulations has many benefits, two of which are reduced cost to conduct training operations and the ability to repeatedly perform activities that are not feasible in real life. The use of automated agents in place of human players or instructors has reduced cost. The sophistication of such agents can range from highly scripted scenarios designed to teach a specific concept to advanced artificially intelligent behaviors that react realistically to a wider range of in-game activities. Repeatability with realistic variation in the automated courses of action (COAs) is a primary objective, but often this comes at the cost of consistency to the doctrine (e.g., “what might I expect that support squad do in this case?”) and explainability of behaviors during after action review (e.g., “why did the enemy do that in this situation?”). This paper describes early progress to develop advanced automated forces that are informed by a given corpus of doctrine. They anticipate future states with uncertainty to generate diverse COAs that guide higher level behaviors of nonplayer characters in a simulation, and they may be queried to explain behaviors at various points of the simulation. We are working with the US Army Combat Capabilities Command Soldier Center (CCDC-SC) to develop this technology for future integration into advanced virtual training systems.

This paper reviews horizontal and vertical scaling methodologies for adaptive instructional system (AIS) software architectures. The term AIS refers to any instructional approach that accommodates individual differences to facilitate and optimize the acquisition of knowledge and/or skills. The authors propose a variety of scaling methods to enhance the interaction between AISs and low-adaptive training ecosystems with the goal of increasing adaptivity and thereby increasing learning and performance. Typically, low-adaptive training systems only accommodate differences in the learner’s in-situ performance during training and do not consider the impact of other factors (e.g., emotions, prior knowledge, goal-orientation, or motivation) that influence learning. AIS architectures such as the Generalize Intelligent Framework for Tutoring (GIFT) can accommodate individual differences and interact with low-adaptive training ecosystems to model a common operational picture of the training relative. These capabilities enable AISs to track progress toward learning objectives and to intervene and adapt the training ecosystem to needs and capabilities of each learner. Finding new methods to interface AISs with a greater number of low-adaptive training ecosystems will result in more efficient and effective instruction.

The goal of a team-centered training approach is to optimize both individual and team performance as a group of individuals band together to pursue complex objectives in complex environments. Teams are a common strategy for both large and small organizations to use in pursuit of their goals. This paper considers technical approaches to model and validate the performance of virtual teammates where their decision-making and behaviors are driven by intelligent agents which are autonomous entities which observe their environment through sensors and act within that environment to achieve assigned goals. In military training, virtual teammate technology has also been referred to as a “stand-in BLUFOR” where BLUFOR denotes friendly or allied forces. As with human teams, human-agent teams seek to cooperate to achieve their common goals and trust plays a critical role in the success of the team. The primary goal of this paper is to describe a validation process to raise the level of technology acceptance of virtual teammates by their human teammates.

This paper serves to connect the papers between the AIS conceptual modeling group and the architectural interchange group by deriving requirements for the required components and the information that they need to exchange. It serves as an update to the original work on the subject, prior to the establishment of the conceptual modeling subgroup.

Intelligent learning environments can be designed to support the development of learners' cognitive skills, strategies, and metacognitive processes as they work on complex decision-making and problem-solving tasks. However, the complexity of the tasks may impede the progress of novice learners. Providing adaptive feedback to learners who face difficulties requires learner modeling approaches that can identify learners' proficiencies and the difficulties they face in executing required skills, strategies, and metacognitive processes. This paper discusses a multilevel hierarchical learner modeling scheme that analyzes and captures learners' cognitive processes and problem-solving strategies along with their performance on assigned tasks in a game-based environment called UrbanSim that requires complex decision making for dealing with counterinsurgency scenarios. As the scenario evolves in a turn-by-turn fashion, UrbanSim evaluates the learners' moves using a number of performance measures. Our learner modeling scheme interprets the reported performance values by analyzing the learners' activities captured in log files to derive learners' proficiencies in associated cognitive skills and strategies, and updates the learner model. We discuss the details of the learner modeling algorithms in this paper, and then demonstrate the effectiveness of our approach by presenting results from a study we conducted at Vanderbilt University.

Adaptive instructional systems such as the Generalized Intelligent Framework for Training (GIFT) can tailor training to meet the learning needs of individuals and teams. A significant cost driver in the design and development of adaptive instructional systems is the manual creation of training scenarios. Delivering personalized instruction to students requires the creation of a broad range of instructional materials. Without effective automation, the tailoring that adaptive instructional systems implement is limited by the small number of instructional variants that a human author can define, as well as a one-size-fits-all approach to training. Further, additional scenarios are useful for enhancing replay through drill-and-practice of specific skills. Generating training scenarios for adaptive instructional systems includes two key components: (1) creating novel scenario content, and (2) devising models that dynamically tailor scenario content to learners.

Recent years have seen growing interest in utilizing sensors to detect learner affect. Modeling frustration has particular significance because of its central role in learning. However, sensor-based affect detection poses important challenges. Motion-tracking cameras produce vast streams of spatial and temporal data, but relatively few systems have harnessed this data successfully to produce accurate run-time detectors of learner frustration outside of the laboratory. In this paper, we introduce a data-driven framework that leverages spatial and temporal posture data to detect learner frustration using deep neural network-based data fusion techniques. To train and validate the detectors, we utilize posture data collected with Microsoft Kinect sensors from students interacting with a game-based learning environment for emergency medical training. Ground-truth labels of learner frustration were obtained using the BROMP quantitative observation protocol. Results show that deep neural network-based late fusion techniques that combine spatial and temporal data yield significant improvements to frustration detection relative to baseline models.

The Thirty‐Second International Florida Artificial Intelligence Research Society Conference was held May 19‐22, 2019, at the Lido Beach Resort in Sarasota, Florida, USA. The conference events included tutorials, invited speakers, special tracks, and presentations of papers, posters, and awards. The conference chair was Vasile Rus from the University of Memphis. The program cochairs were Keith Brawner from the Army Research Laboratory and Roman Barták from Charles University, Prague. The special tracks were coordinated by Eric Bell.

One on one tutoring from human expert tutors to human students is the most effective form of instruction found to date. There are many actions that human tutors perform which make them remarkably effective, including the attention that they pay to the cognitive and affective states of the human students that they tutor, and the use of this knowledge to modify the way that they instruct the material. According to theoretical models, learner state data is used to inform instructional data and decisions, which then influences the learning of the student. Naturally, the data about student state must be available in order to be used to adjust the instruction. Success amongst operational systems, however, has not been observed with generalised modelling techniques. Individualised and adaptive modelling techniques from other domains in the literature present an alternative to the approach which is not observing significant operational success. This work investigates individualised adaptive models, validates the approach, and shows that it can produce models of acceptable quality, but that doing so does not obviate the experimenter from creating quality generalised models prior to individualising.

There are a number of future efforts to revise military training and generally bring it into the 21st century, including the Army Learning Model, Synthetic Training Environment, Sailor 2025 initiative, and other service-level training revamps. These revamps are expected to do more than the past developments in content and LMS standards – tracking students, providing mappings of competencies, recommending for and against future training items, and other relatively advanced tasks. The Institute of Electrical and Electronics Engineers Adaptive Instructional Systems group has created the Adaptive Instructional Systems standards group, which is investigating the issues faced by the next wave of learning software. This paper discusses some of the technical and social issues of moving to the new model of education.

Learning content is increasingly diverse in order to meet learner needs for individual personalization, progression, and variety. Learners may encounter material through different content, which invites a measurable comparison method in order to tell when delivered content is sufficient or similar. Content recommendation and generation similarly motivate a fine-grained measure that enhances the search for just the right content or identifies where new learning content is needed to support all learners. Complexity offers a fine-grained way of measuring content which works across instructional domains and media types, potentially adding to existing qualitative and quantitative content descriptions. Reductionist complexity measures focus on quantifiable accounting which practitioners and computers in support of practice can use together to communicate about the complexity of learning content. In addition, holistic complexity measures incorporate contextual influences on complexity that practitioners typically reason about when they understand, choose, and personalize learning content. A combined measure of complexity uses learning objectives as a focus point to let teachers and trainers manage the scope of reductionist elements and capture holistic context factors that are likely to affect the learning content. The combined measure has been demonstrated for automated content generation. This concrete example enables an upcoming study on the expert acceptance and usability of complexity for differentiating between hundreds of generated scenarios. As the combined complexity measure is refined and tested in additional domains, it has potential to help computers reason about learning content from many sources in a unified manner that experts can understand, control, and accept.

This paper discusses the need for adaptive instructional system (AIS) standards and suggests the Generalized Intelligent Framework for Tutoring (GIFT) as a starting point for discussing component level interaction as a potential candidate for standardization. GIFT is an open, modular architecture to support authoring, delivery, instruction, and evaluation of adaptive instruction. Adaptive instruction is usually delivered by Intelligent Tutoring Systems (ITSs) and like most ITSs is composed of four basic components: a learner model, an instructional model, a domain model, and an interface model. We are suggesting that the data exchanged between these four models (that are in the form of messages in GIFT) are candidates for standardization in that they solve the problem of interoperability while simultaneously allowing for flexibility of form and function within each of the common components. This paper examines the type and form of GIFT messages and makes a case for their consideration as an initial starting place for AIS standards for interoperability.