Most computational theories of cognition lack a representation of physiology. Understanding the cognitive effects of compounds present in the environment is important for explaining and predicting changes in cognition and behavior given exposure to toxins, pharmaceuticals, or the deprivation of critical compounds like oxygen. This research integrates physiologically based pharmacokinetic (PBPK) model predictions of caffeine concentrations in blood and tissues with ACT-R's fatigue module to predict the effects of caffeine on fatigue. Mapping between the PBPK model parameters and ACT-R model parameters is informed by the neurophysiological literature and established associations between ACT-R modules and brain regions. The results from three such parameter mappings are explored to explain observed data from sleep-deprived participants performing the psychomotor vigilance test with and without caffeine. Predicted caffeine concentrations in the brain are used to modulate procedural parameters in the fatigue module to explain caffeine's effects on multiple performance metrics.

Study Objectives: A cognitive throughput task known as the Digit Symbol Substitution Test (DSST) (or Symbol Digit Modalities Test) has been used as an assay of general cognitive slowing during sleep deprivation. Here, the effects of total sleep deprivation (TSD) on specific cognitive processes involved in DSST performance, including visual search, spatial memory, paired-associate learning, and motor response, were investigated through targeted task manipulations. Methods: A total of 12 DSST variants, designed to manipulate the use of specific cognitive processes, were implemented in two laboratory-based TSD studies with N = 59 and N = 26 subjects, respectively. In each study, the Psychomotor Vigilance Test (PVT) was administered alongside the DSST variants. Results: TSD reduced cognitive throughput on all DSST variants, with response time distributions exhibiting rightward skewing. All DSST variants showed practice effects, which were however minimized by inclusion of a pause between trials. Importantly, TSD-induced impairment on the DSST variants was not uniform, with a principal component analysis revealing three factors. Diffusion model decomposition of cognitive processes revealed that inter-individual differences during TSD on a two-alternative forced choice DSST variant were different from those on the PVT. Conclusions: While reduced cognitive throughput has been interpreted to reflect general cognitive slowing, such TSD-induced impairment appears to reflect cognitive instability, like on the PVT, rather than general slowing. Further, comparisons between task variants revealed not one, but three distinct underlying processes impacted by sleep deprivation. Moreover, the practice effect on the task was found to be independent of the TSD effect and minimized by a task pacing manipulation.

The model of situation awareness (SA) as described by Mica Endsley in her articles over the past couple decades is based in empirical psychological research and validated through testing and application across many domains. While there are many similar descriptive (“box-and-arrow”) models of human SA, some of which differ significantly from Endsley’s, few computational process models of human SA exist. The vigorous debate over the proper form and function of descriptive SA models offers us a valuable insight into how best to evaluate process models for their adherence to human cognitive processes. Here we propose criteria, based on the descriptive SA model literature, for evaluating computational process models’ SA capabilities, provide an example of that evaluation, and argue for the utility of computational process models in testing theoretical claims about situation awareness.

Autonomous systems are a new frontier pushing socio-technical advancement. Such systems will be required to team with humans. Consequently, the ability to coordinate with teammates is critical. We have developed and empirically evaluated an autonomous synthetic teammate (AST) designed to operate in a task in which it receives information from a visual data display and chat messages from human teammates. Teams with the AST performed as well on most performance measures as teams without it. Further, the AST performed its piloting task well. Human participants performed their tasks as well with the AST piloting the system as they did with a human pilot. Nonetheless, we observed issues that show there remains room for improving human-AST coordination. These issues illuminate limitations in the AST’s situation representation and point to directions for further improvement and future research.

We demonstrate a set of software tools designed to facilitate computational cognitive modeling of multitasking performance. The Modifiable Multitasking Environment (ModME) offers a flexible, browser-based platform for creating multitasking experiments. Simplified Interfacing for Modeling Cognition–JavaScript (SIMCog-JS) provides communication between the browser-based experiments in ModME and the Java implementation of the ACT-R cognitive architecture. The baseline configuration of these software packages enables an ACT-R model to perform pilot-like multitasking in the modified Multi-Attribute Task Battery, which is implemented as the baseline task available in ModME. We show how this combination facilitates the development of models for assessing multitasking workload. In this demonstration, we will explain the software packages and allow attendees to interact with system elements, particularly the ModME graphical user interfaces. All software is available open source for attendees to try themselves.

We present two visualization approaches illustrating the value of formal cognitive models for predicting, capturing, and understanding eye tracking as a manifestation of underlying cognitive processes and strategies. Computational cognitive models are formal theories of cognition which can provide predictions for human eye movements in visual decision-making tasks. Visualizing the internal dynamics of a model provides insights into how the interplay of cognitive mechanisms influences the observable eye movements. Animation of those model behaviors in virtual human agents gives explicit, high fidelity visualizations of model behavior, providing the analyst with an understanding of the simulated human’s behavior. Both can be compared to human data for insight about cognitive mechanisms engaged in visual tasks and how eye movements are affected by changes in internal cognitive strategies, external interface properties, and task demands. We illustrate the visualizations on two models of visual multitasking and juxtapose model performance against a human operator performing the same task.

Eye movements and pupil size have been used to assess workload in previous research. However, the results presented in the literature vary, and the tasks have been too simple at times or the experimental conditions (e.g. lighting) too tightly controlled to determine if the use of eye data to assess workload is useful in real-world contexts. This research investigates the use of ten eye movement, eyelid, or pupil related metrics as input to support vector machines for classifying workload in a complex task. The results indicate that both pupil size and percentage of eye closure are useful for predicting workload. Further, the combination of the two metrics increases the robustness and accuracy of the workload predictions.

Human visual search plays an important role in many human–computer interaction HCI tasks. Better models of visual search are needed not just to predict overall performance outcomes, such as whether people will be able to find the information needed to complete an HCI task, but to understand the many human processes that interact in visual search, which will in turn inform the detailed design of better user interfaces. This article describes a detailed instantiation, in the form of a computational cognitive model, of a comprehensive theory of human visual processing known as “active vision” Findlay & Gilchrist, 2003. The computational model is built using the Executive Process-Interactive Control cognitive architecture. Eye-tracking data from three experiments inform the development and validation of the model. The modeling asks—and at least partially answers—the four questions of active vision: a What can be perceived in a fixation? b When do the eyes move? c Where do the eyes move? d What information is integrated between eye movements? Answers include: a Items nearer the point of gaze are more likely to be perceived, and the visual features of objects are sometimes misidentified. b The eyes move after the fixated visual stimulus has been processed i.e., has entered working memory. c The eyes tend to go to nearby objects. d Only the coarse spatial information of what has been fixated is likely maintained between fixations. The model developed to answer these questions has both scientific and practical value in that the model gives HCI researchers and practitioners a better understanding of how people visually interact with computers, and provides a theoretical foundation for predictive analysis tools that can predict aspects of that interaction.

Circadian rhythms cause alertness declines at night, producing performance decrements across cognitive domains and tasks. Building on the learning mechanisms for declarative knowledge instantiated in the ACT-R cognitive architecture, this research seeks to explain the effects of circadian rhythms on performance of an orientation task performed repeatedly across two weeks by participants working either day or night shifts. The differences in performance between the two groups are best explained by varying the decay rate in declarative knowledge as a function of the time of day the task was performed. The model accounts well for task learning reflected in decreases in response times across days, as well as differences in learning between the day and night shift conditions.

The Digit Symbol Substitution Test (DSST) has been used to study various effects, like aging and fatigue. Multiple cognitive and perceptual processes, like associative memory and visual search, are prominently utilized in the DSST. Understanding how these processes contribute to execution of tasks like the DSST is important to human factors research, as moderators like age and fatigue may differentially affect these processes. This study investigates performance on variants of the DSST that emphasize either visual search or associative memory with experimentation and computational cognitive modeling. While there are similarities in performance across task variants, the observed data suggests that, when visual search is possible, people appear to not utilize memory to the extent they would if relying on memory alone. The modeling suggests that behavior differences in the DSST variants results partly from procedural (i.e. strategic) choices and partly from the demands of the tasks.

Human-computer systems intended for time-critical multitasking need to be designed with an understanding of how humans can coordinate and interleave perceptual, memory, and motor processes. This paper presents human performance data for a highly-practiced time-critical dual task. In the first of the two interleaved tasks, participants tracked a target with a joystick. In the second, participants keyed-in responses to objects moving across a radar display. Task manipulations include the peripheral visibility of the secondary display (visible or not) and the presence or absence of auditory cues to assist with the radar task. Eye movement analyses reveal extensive coordination and overlapping of human information processes and the extent to which task manipulations helped or hindered dual task performance. For example, auditory cues helped only a little when the secondary display was peripherally visible, but they helped a lot when it was not peripherally visible.

The Effects of Work Shift and Strategy on an Orientation Task Tim Halverson (thalverson@gmail.com) Oak Ridge Institute for Science and Education Air Force Research Laboratory Mesa, AZ 85212 USA Glenn Gunzelmann (glenn.gunzelmann@mesa.afmc.af.mil) Air Force Research Laboratory Mesa, AZ 85212 USA L. Richard Moore Jr. (larry.moore@mesa.afmc.af.mil) Lockheed Martin Air Force Research Laboratory Mesa, AZ 85212 USA Hans P.A. Van Dongen (hvd@wsu.edu) Sleep and Performance Research Center, Washington State University Spokane, WA 99210 USA Abstract Cognitive alertness decreases at night due to circadian rhythms with adverse effects on performance across domains and tasks, including real-world tasks like driving and flying. Additionally, the strategy used on a task may have a substantial effect on performance. However, little is known about whether and how circadian rhythms and strategy interact to affect performance. The current study investigates participants’ performance on an orientation task performed over a period of two weeks. Participants were assigned to simulated day or night shift conditions, and were trained to use one of two strategies for the orientation task. The results indicated that shift condition had little impact on a more declarative strategy for the task, but had a significant impact on a more spatial strategy. The results illustrate how different aspects of cognitive functioning may be affected differently by circadian rhythms, and point to some important implications for training and task performance in real-world contexts. Keywords: spatial; sleep; circadian rhythm; fatigue; learning; shift work Introduction Critical, safety-sensitive activities, such as driving and air traffic control, are performed at all times of the day and night. Yet, it is not well understood how nighttime operations affect task performance in contexts such as these. Most research on night and shift work has focused on how shift differences affect sleep and frequency of accidents (e.g., Akerstedt, 1988). Little work has focused on how shift work and task differences affect different cognitive processes alone or in interaction. Variations in alertness due to circadian rhythms and sleep loss have been shown to affect various components of cognitive functioning (Jackson & Van Dongen, in press). For example, vigilant attention (Lim & Dinges, 2008), perceptual learning (Mednick, Nakayama & Stickgold, 2003), and motor learning (Walker, Brakefield, Morgan, Hobson & Stickgold, 2003) are all affected by fluctuations in alertness associated with time awake and circadian rhythms. For shift work, circadian rhythms are particularly important. Circadian rhythms are driven by a biological clock in the suprachiasmatic nuclei of the hypothalamus, which imposes cyclical changes in alertness throughout the day, leading to increased pressure for sleep at night. This leads to nocturnal degradations in cognitive performance (Van Dongen & Dinges, 2005), as demonstrated in a variety of tasks and domains (e.g. Caldwell, 2003; Dinges, 1995). The present research investigates how strategies recruiting different cognitive-perceptual processes may be differentially affected by fluctuations in alertness resulting from circadian rhythms in laboratory-simulated shift work. This is accomplished within the context of a spatial direction task, where distinct alternative cognitive strategies have been identified (Gunzelmann, Anderson & Douglass, 2004). In this task, participants are presented with two views of a set of objects (Figure 1). One of the views (the left side in Figure 1) is an overhead, ego-oriented perspective, based on a viewpoint at the bottom of the screen. Within the ego- oriented view, one of the objects (small circles) in each trial is filled in to identify it as a target. The other view (the right side in Figure 1) shows a map-like perspective with the viewpoint indicated by the arrow, which may be misaligned relative to the ego-oriented view on the left. The task requires participants to identify the location of the target in the map-like perspective. In the study described here, participants were taught to use one of two strategies for the spatial direction task: one based on counting and the other on mental rotation, as in Gunzelmann et al. (2004). The strategies are described in more detail below. The key feature is that the strategies emphasize different cognitive functions, declarative and

Cognitive alertness decreases at night due to circadian rhythms with adverse effects on performance across domains and tasks, including real-world tasks like driving and flying. Additionally, the strategy used on a task may have a substantial effect on performance. However, little is known about whether and how circadian rhythms and strategy interact to affect performance. The current study investigates participants' performance on an orientation task performed over a period of two weeks. Participants were assigned to simulated day or night shift conditions, and were trained to use one of two strategies for the orientation task. The results indicated that shift condition had little impact on a more declarative strategy for the task, but had a significant impact on a more spatial strategy. The results illustrate how different aspects of cognitive functioning may be affected differently by circadian rhythms, and point to some important implications for training and task performance in real-world contexts.

Visual search is an integral component in many human activities. The eye movements produced during such activities can provide valuable information about people's cognitive processes. This research investigates, with detailed eye movement data analysis and computational cognitive modeling, the perceptual, strategic, and oculomotor processes people use to visually search. A cognitive model is evolved in a principled manner based on eye movement data, past modeling efforts, and recent psychological literature. In the model, re-usable, parsimonious, local strategies interact with perceptual-motor constraints to predict the bulk of the eye movement data, including aspects of the data that appear to require task-specific global strategies in addition to fixation-to-fixation local strategies. The analysts evolve a base level model with a random strategy into a robust and reusable model with a flexible strategy that could work with a wide range of visual stimuli.

This research investigates the use of auditory displays in complex tasks with multiple visual displays. Participants were trained extensively in a dual task that requires time- pressured, manual responses to stimuli on two displays. Continuous responses were required on the first display. Complex decisions were intermittently required on the second display. The number of decisions required within a short time varied from one to eight. A 2x2 factorial design was used. In two conditions the participants could not see the other display in their periphery. In two conditions auditory cues were presented for visual events on the second display. Each participant completed sessions across three consecutive days. It was found that auditory displays allow for considerable strategy optimization, but only when peripheral information was not available and only after considerable practice. Additionally, the lack of peripheral information negatively affected performance more when fewer decisions were required. Design implications are discussed. Author Keywords Multitasking, auditory displays, gaze contingency

An empirical study explored the extent to which people can map locations in auditory space to locations on a visual display for four different transformations (or mappings) between auditory and visual surfaces. Participants were trained in each of four transformations: horizontal square, horizontal arc, vertical square, and vertical spherical surface. On each experimental trial, a sound was played through headphones connected to a spatialized sound system that uses a non-individualized head-related transfer function. The participant's task was to determine, using one transformation at a time, which of two objects on a visual display corresponded to the location of the sound. Though the two vertical transformations provided a more direct stimulus-response compatibility with the visual display, the two horizontal transformations made better use of the human auditory system's ability to localize sound, and resulted in better performance. Eye movements were analyzed, and it was found that the horizontal arc transformation provided the best auditory cue for moving the eyes to the correct visual target location with a single saccade.