Understanding how cognitive biases influence adversarial decision-making is essential for developing effective cyber defenses. Capture-the-Flag (CTF) competitions provide an ecologically valid testbed to study attacker behavior at scale, simulating real-world intrusion scenarios under pressure. We analyze over 500,000 submission logs from picoCTF, a large educational CTF platform, to identify behavioral signatures of cognitive biases with defensive implications. Focusing on availability bias and the sunk cost fallacy, we employ a mixed-methods approach combining qualitative coding, descriptive statistics, and generalized linear modeling. Our findings show that participants often submitted flags with correct content but incorrect formatting (availability bias), and persisted in attempting challenges despite repeated failures and declining success probabilities (sunk cost fallacy). These patterns reveal that biases naturally shape attacker behavior in adversarial contexts. Building on these insights, we outline a framework for bias-informed adaptive defenses that anticipate, rather than simply react to, adversarial actions.

Coordinated teamwork is essential in fast-paced decision-making environments that require dynamic adaptation, often without an opportunity for explicit communication. Although implicit coordination has been extensively considered conceptually in the existing literature, the majority of work has focused on co-located, synchronous teamwork (such as in sports teams) or, in distributed teams the focus has been primarily on coordination of knowledge work. However, many teams (firefighters, military, law enforcement, emergency response) need to coordinate their movements in physical space without the benefit of visual cues or extensive explicit communication. This paper investigates how three dimensions of spatial coordination, namely exploration diversity, movement specialization, and adaptive spatial proximity, influence team performance in a collaborative online search and rescue task where explicit communication is restricted, and team members must rely on movement patterns to infer others’ intentions and coordinate their actions. Our metrics capture the relational aspects of teamwork by measuring spatial proximity, distribution patterns, and alignment of movements within shared environments. We analyze data from an experiment including 34 four-person teams (136 participants) assigned to specialized roles in a search and rescue task. Our results demonstrate that spatial specialization positively predicts team performance, while adaptive spatial proximity exhibits a marginal inverted U-shaped relationship, suggesting that moderate levels of adaptation are optimal. Furthermore, the temporal dynamics of these spatial coordination metrics clearly differentiate high- from low-performing teams over time. These findings provide insights into implicit spatial coordination in role-based teamwork and highlight the importance of balanced adaptive strategies, with implications for team training and the development of AI-assisted team support systems.

Adversary emulation is commonly used to test cyber-defense performance against known threats to organizations. However, many adversary emulation methods often rely on automated planning and underplay the role of human cognition. Consequently, defenders are often underprepared for human attackers who can think creatively and adapt their strategies. In this paper, we propose the design of adversarial cognitive agents that are dynamic, adaptable, and able to learn from experience. These cognitive agents are built based on the theoretical principles of Instance-Based Learning Theory (IBLT) of experiential choice in dynamic tasks, making them more challenging than strategically optimal adversaries for human defenders. Our research offers three main contributions. First, in a simulation experiment, we demonstrate how IBL attacker agents can learn from experience and become as efficient as optimal strategic algorithms against a strategic defender. In a second simulation experiment, the IBL attackers are pitted against an IBL defender, showing that the IBL attacker can be a more challenging adversary for the IBL defender, while the IBL defender can learn to counter carefully crafted optimal attack strategies. To test these observations, we conducted a third experiment, where humans played the role of defenders against both strategic and IBL attackers in an interactive task. The results confirm the predictions of the second simulation experiment: a cognitive attackers are more challenging for human defenders than strategic attackers. These insights contribute to informing future adversary emulation efforts and training of cyber defenders.

As artificial intelligence (AI) becomes increasingly integrated into complex decision-making environments, there is a growing need to develop AI systems that complement human capabilities. AI and humans offer distinct strengths: AI excels at processing large datasets, identifying statistical patterns and optimizing predefined objectives, whereas humans are skilled at navigating uncertainty, novelty and interpersonal challenges. The synergy between humans and AI is particularly vital in dynamic decision-making domains — such as disaster response situations — in which rapid analysis of AI results must be balanced with human judgement and ethical considerations. In this Perspective, we provide a conceptual framework to integrate human decision-making with AI, focusing on cognitive AI: a computational approach that models human cognitive processes to create AI systems that learn and make decisions in ways similar to those of humans. We discuss the elements and necessary capabilities of cognitive AI and how to realize human–AI complementarity in decision-making while considering ethical risks. By advancing these areas, researchers can lay the groundwork for adaptive and cognitively grounded human–AI teamwork that is aligned with human values and goals. Complex decision-making tasks can be addressed with artificial intelligence (AI) systems that complement human capabilities. In this Perspective, Gonzalez and Heidari provide a conceptual framework to integrate human and AI decision-making, highlighting technical challenges and ethical risks.

Cognitive biases shape human decision making, yet their role in cybersecurity remains under-explored. Notably, little research examines whether cyber attackers succumb to cognitive biases-an insight that could inform defense strategies. This study addresses this gap by testing whether traditional cognitive biases replicate in a cybersecurity context. We develop cybersecurity-specific versions of three biases: Availability (Frequency and Recall), Recency, and Loss Aversion (Endowment, Gain, and Loss Framing). A within-subjects experiment compared responses to bias problems in cybersecurity and non-security contexts. The results indicate a successful translation of Frequency, Recency, Endowment, and Loss Framing biases to cybersecurity. However, Recall and Gain Framing biases did not translate effectively. Interestingly, many participants who exhibited bias in the general context did not in the cybersecurity context. Individual-level analyses suggest that participants engage in more deliberate reasoning when faced with cybersecurity-related problems, whereas they rely more on intuition in general scenarios. These findings underscore the nuances of designing cybersecurity scenarios that use cognitive biases but suggest that designing cyber defenses according to cognitive biases can be a successful strategy.

Social engineering attacks such as phishing emails remain a critical method for cybercriminals to exploit sensitive data. Although the threat of AI-generated content in such attacks is growing, current training methods predominantly rely on simplistic human-designed emails. This research introduces a novel experimental paradigm to investigate differences in the detection of human-generated versus AI-generated phishing emails, as well as two different methods by which cyberattackers could use AI as a tool to generate phishing emails. Our behavioral results reveal that emails co-created by humans and Generative-AI models pose a greater challenge to end users compared to emails created by GPT-4 or Humans working alone. We also propose a cognitive model that predicts user behavior during training, which offers the potential to be used in future user training to improve training outcomes. Our work contributes by (1) identifying critical weaknesses in current social engineering training, (2) describing biases that human participants demonstrate when viewing GPT-4 written content in emails, and (3) proposing a cognitive model-driven solution to better train users against evolving threats.

Autonomous agents are becoming increasingly prevalent and capable of collaborating with humans on interdependent tasks as teammates. There is increasing recognition that human-like agents might be natural human collaborators. However, there has been limited work on designing agents according to the principles of human cognition or in empirically testing their teamwork effectiveness. In this study, we introduce the Team Defense Game (TDG), a novel experimental platform for investigating human-autonomy teaming in cyber defense scenarios. We design an agent that relies on episodic memory to determine its actions (Cognitive agent) and compare its effectiveness with two types of autonomous agents: one that relies on heuristic reasoning (Heuristic agent) and one that behaves randomly (Random agent). These agents are compared in a human-autonomy team (HAT) performing a cyber-protection task in the TDG. We systematically evaluate how autonomous teammates’ abilities and competence impact the team’s interaction and outcomes. The results revealed that teams with Cognitive agents are the most effective partners, followed by teams with Heuristic and Random agents. Evaluation of collaborative team process metrics suggests that the cognitive agent is more adaptive to individual play styles of human teammates, but it is also inconsistent and less predictable than the Heuristic agent. Competent agents (Cognitive and Heuristic agents) require less human effort but might cause over-reliance. A post-experiment questionnaire showed that competent agents are rated more trustworthy and cooperative than Random agents. We also found that human participants’ subjective ratings correlate with their team performance, and humans tend to take the credit or responsibility for the team. Our work advances HAT research by providing empirical evidence of how the design of different autonomous agents (cognitive, heuristic, and random) affect team performance and dynamics in cybersecurity contexts. We propose that autonomous agents for HATs should possess both competence and human-like cognition while also ensuring predictable behavior or clear explanations to maintain human trust. Additionally, they should proactively seek human input to enhance teamwork effectiveness.

Attention can be used to inform choice selection in contextual bandit tasks even when context features have not been previously experienced. One example of this is in dimensional shifts, where additional feature values are introduced and the relationship between features and outcomes can either be static or variable. Attentional mechanisms have been extensively studied in contextual bandit tasks where the feedback of choices is provided immediately, but less research has been done on tasks where feedback is delayed or in counterfactual feedback cases. Some methods have successfully modeled human attention with immediate feedback based on reward prediction errors (RPEs), though recent research raises questions of the applicability of RPEs onto more general attentional mechanisms. Alternative models suggest that information theoretic metrics can be used to model human attention, with broader applications to novel stimuli. In this paper, we compare two different methods for modeling how humans attend to specific features of decision making tasks, one that is based on calculating an information theoretic metric using a memory of past experiences, and another that is based on iteratively updating attention from reward prediction errors. We compare these models using simulations in a contextual bandit task with both intradimensional and extradimensional domain shifts, as well as immediate, delayed, and counterfactual feedback. We find that calculating an information theoretic metric over a history of experiences is best able to account for human-like behavior in tasks that shift dimensions and alter feedback presentation. These results indicate that information theoretic metrics of attentional mechanisms may be better suited than RPEs to predict human attention in decision making, though further studies of human behavior are necessary to support these results.

Research on decision making has identified a large number of systematic patterns in how humans process information and make choices; these are often called cognitive biases because they reflect deviations from perfectly rational choices. Cyber attackers make extensive use of vulnerabilities in decision making (e.g., in many types of social engineering attacks), but cyber defenses rarely try to exploit similar vulnerabilities in cyber attackers. We evaluate the role of Denominator Neglect (DN) and the Hot Hand Fallacy (HHF) in decision made by cyber attackers; both of these are effects of using the Representativeness Heuristic. Our experimental design combines established tasks used to measure these effects with Cyber Isomorphs (CI)-novel tasks presented in a cyber context that mirror tasks use in the literature to the extent possible. This allows us to test whether the effects observed in the literature will translate consistently into more realistic cyber attack scenarios. We conducted a survey comprising both tasks with 85 participants screened for basic cybersecurity knowledge. We observe that participants make choices consistent with denominator neglect and the hot hand fallacy in both variants of the task, with statistically significant differences from the rational baseline in most conditions. The effect size for HHF was particularly strong with biased choices in more than 75% of cases. We also observe that the effects are often stronger in the cyber tasks than the original tasks.

Research on human-AI collaboration often prioritizes objective performance. However, understanding human subjective preferences is essential to improving human-AI complementarity and human experiences. We investigate human preferences for controllability in a shared workspace task with AI partners using Behavior Shaping (BS), a reinforcement learning algorithm that allows humans explicit control over AI behavior. In one experiment, we validate the robustness of BS in producing effective AI policies relative to self-play policies, when controls are hidden. In another experiment, we enable human control, showing that participants perceive AI partners as more effective and enjoyable when they can directly dictate AI behavior. Our findings highlight the need to design AI that prioritizes both task performance and subjective human preferences. By aligning AI behavior with human preferences, we demonstrate how human-AI complementarity can extend beyond objective outcomes to include subjective preferences.

While pairwise cooperation has been extensively studied through the Prisoner’s Dilemma (PD), our understanding of how cooperation emerges in small groups remains limited. We extend the classic dyadic PD framework to a triadic framework, examining two sets of PD games per individual and how individual strategies and relationships aggregate to group cooperation. Through two experiments (N=519), we investigate: (1) how structural incentives shape cooperation by varying the K-index (0.4/0.8), a theoretical value that predicts greater cooperation for higher K values, and (2) how different degrees of information about mutual interdependence affect group behavior. We find that, under minimal information conditions, a higher K-index promotes sustained cooperation in the triadic setting, in alignment with the theoretical definition of the K-index. However, while experiential information (observing others’ actions/outcomes) enhances cooperation, descriptive information (complete payoff matrices) paradoxically reduces cooperation. Analysis of triadic interactions reveals that selective cooperation by a third player in the group can stabilize cooperative dyadic relationships and destabilize defective dyadic relationships. These findings provide insights for designing cooperative systems, particularly in contexts where organizations must balance information sharing benefits against strategic risks.

For public health programs with limited resources, the ability to predict how behaviors change over time and in response to interventions is crucial for deciding when and to whom interventions should be allocated. Using data from a real-world maternal health program, we demonstrate how a cognitive model based on Instance-Based Learning (IBL) Theory can augment existing purely computational approaches. Our findings show that, compared to general time-series forecasters (e.g., LSTMs), IBL models, which reflect human decision-making processes, better predict the dynamics of individuals' states. Additionally, IBL provides estimates of the volatility in individuals' states and their sensitivity to interventions, which can improve the efficiency of training of other time series models.

During the past decade, researchers of behavioral cyber security have created cognitive agents that are able to learn and make decisions in dynamic environments in ways that assimilate human decision processes. However, many of these efforts have been limited to simple detection tasks and represent basic cognitive functions rather than a whole set of cognitive capabilities required in dynamic cyber defense scenarios. Our current work aims at advancing the development of cognitive agents that learn and make defense-dynamic decisions during cyber attacks by intelligent attack agents. We also aim to evaluate the capability of these cognitive models in ``Turing-like'' experiments, comparing the decisions and performance of these agents against human cyber defenders. In this paper, we present an initial demonstration of a cognitive model of the defender that relies on a cognitive theory of dynamic decision-making, Instance-Based Learning Theory (IBLT); we also demonstrate the execution of the same defense task by human defenders. We rely on OpenAI Gym and CybORG and adapt an existing CAGE scenario to generate a simulation experiment using an IBL defender. We also offer a new Interactive Defense Game (IDG), where \textit{human} defenders can perform the same CAGE scenario simulated with the IBL model. Our results suggest that the IBL model makes decisions against two intelligent attack agents that are similar to those observed in a subsequent human experiment. We conclude with a description of the cognitive foundations required to build autonomous intelligent cyber defense agents that can collaborate with humans in autonomous cyber defense teams.

Modeling human cognitive processes in dynamic decision-making tasks has been an endeavor in AI for a long time because such models can help make AI systems more intuitive, personalized, mitigate any human biases, and enhance training in simulation. Some initial work has attempted to utilize neural networks (and large language models) but often assumes one common model for all humans and aims to emulate human behavior in aggregate. However, the behavior of each human is distinct, heterogeneous, and relies on specific past experiences in certain tasks. For instance, consider two individuals responding to a phishing email: one who has previously encountered and identified similar threats may recognize it quickly, while another without such experience might fall for the scam. In this work, we build on Instance Based Learning (IBL) that posits that human decisions are based on similar situations encountered in the past. However, IBL relies on simple fixed form functions to capture the mapping from past situations to current decisions. To that end, we propose two new attention-based neural network models to have open form non-linear functions to model distinct and heterogeneous human decision-making in dynamic settings. We experiment with two distinct datasets gathered from human subject experiment data, one focusing on detection of phishing email by humans and another where humans act as attackers in a cybersecurity setting and decide on an attack option. We conducted extensive experiments with our two neural network models, IBL, and GPT3.5, and demonstrate that the neural network models outperform IBL significantly in representing human decision-making, while providing similar interpretability of human decisions as IBL. Overall, our work yields promising results for further use of neural networks in cognitive modeling of human decision making.

Despite enormous efforts to develop defenses against phishing attacks, humans still struggle to detect phishing emails given the constantly evolving attacker strategies. This paper aims to test the predictive capabilities of a cognitive model that represents the individual susceptibility to phishing emails. We developed an instance-based learning model (IBL) that captures the frequency, recency, and familiarity aspects of decision-making and explores its potential for personalized anti-phishing training. We investigate the same cognitive model using three different methods. At the same time, we compare the performance of these three models to human classification of phishing and ham emails and determine the synchronization rates with human participants in a Phishing Training Task. Our results reveal that using prior human experience and optimizing parameters improves model accuracy. These findings suggest significant advances in modeling human decision-making patterns and cognitive processes, demonstrating strong alignment with human decisions during training.

Understanding a cyber attacker's behavior can help improve cyber defenses. However, significant research is needed to learn about attackers’ decision-making processes. For example, some advancement has been made in understanding attackers’ decision biases and the potential that measuring such biases would have for cyber defenses. However, currently, there are no publicly available datasets that could be used to learn about attackers' cognitive biases. New research is needed to provide clear metrics of attacker cognitive biases in professional red teamers, using testbeds that represent realistic cybersecurity scenarios. New studies should go beyond exploratory observations and rely on formal metrics of cognitive biases that can use the actions taken by the adversaries (i.e., rely on what adversaries "do" more than what they "say") and be able to demonstrate how defense strategies can be informed by such attacker biases. In this paper, we start to build upon existing work to demonstrate that we can detect and measure professional red teamers' cognitive biases based on the actions they take in a realistic Advanced Persistent Threat (APT) scenario. We designed a cybersecurity scenario in which an attacker would execute an APT-style attack campaign. The goal for the attacker was to obtain sensitive documents from the target network. To achieve this goal, human attackers were asked to perform network reconnaissance, laterally move to hosts and gain access to the relevant systems, and finally, perform data exfiltration as a post-exploitation task. We used the CyberVAN testbed for our experimentation. CyberVAN is a flexible cyber range that offers a high-fidelity representation of heterogeneous network environments. CyberVAN supports a Human-in-the-loop (HITL) capability that allows participants to remotely log into a VM in a network scenario and interact with other VMs in that scenario. For our experimentation, we designed a network in CyberVAN to enable a multi-step attack campaign wherein participants were required to make decisions at each step in order to progress toward the goal. The network was divided into three levels to represent the different stages of the attack campaign. Participants were provided necessary tools to scan the network, to crack passwords and exploit vulnerabilities. Attackers start their activities from the attacker host, a designated host external to the target network. At level 1 their goal is to gain unauthorized access to one of five hosts by cracking the passwords of valid users on the system. Once attackers successfully log in to a host at level 1, they pivot to a host at level 2 by remotely exploiting security vulnerabilities present in that host. The host was configured with real services containing known vulnerabilities that are remotely exploitable. At level 2, the attacker’s goal is to gain access to the target host at level 3 and exfiltrate as many files as possible from the target machine. From level 2, attackers are given two options to execute the attack: (i) an open-source tool that is reliable but requires additional effort to set up and execute, and (ii) a prepared shell script that is unreliable (small probability of success) but easy to execute. Upon compromising the target host, the final action is to exfiltrate as many files as possible from the host to an external drop site. For exfiltration, attackers choose between standard file transfer applications such as SCP and FTP. Attackers were periodically informed that the network defenders might be monitoring the network and that they might be detected at any stage of the task. If detected, attackers were returned to the previous step and had to perform the task again by choosing a different host/credential/exploit. Results provided evidence of default effect bias, availability bias, and recency bias. Participants chose the first or the last IP address from the network scan result, representing an indication of default effect bias. We also observed that participants preferred simple/easy-to-execute options over complex and reliable options indicative of complexity aversion. Similarly, we observe that recently discovered vulnerabilities were exploited 67% of the time although they only made up 50% of the available vulnerabilities indicative of recency bias. This paper provides initial evidence to identify the cognitive biases and behaviors in cyberattackers.

Social engineering attacks are commonly used by cybercriminals to gain valuable and sensitive data. Although the concern of attackers using AI-generated content is serious, training against social engineering attacks is typically based on simple human-designed emails. Our research introduces an experimental paradigm to determine whether there is a difference in the detection of human-generated and AI-generated emails. The behavioral results show that emails written by humans and stylized by Generative-AI models are more challenging for end-users. Alongside this novel experiment, we propose a cognitive model that can be used to predict end-user behavior during training, with the potential to improve the quality of examples used during training and the training feedback. Overall, the contributions of this work are, first, the outline of some limitations to current social engineering training methods and, second, pinpointing a potential solution to these limitations through the use of a cognitive model to improve learning outcomes.

Cosine similarity between two documents can be computed using token embeddings formed by Large Language Models (LLMs) such as GPT-4, and used to categorize those documents across a range of uses. However, these similarities are ultimately dependent on the corpora used to train these LLMs, and may not reflect subjective similarity of individuals or how their biases and constraints impact similarity metrics. This lack of cognitively-aware personalization of similarity metrics can be particularly problematic in educational and recommendation settings where there is a limited number of individual judgements of category or preference, and biases can be particularly relevant. To address this, we rely on an integration of an Instance-Based Learning (IBL) cognitive model with LLM embeddings to develop the Instance-Based Individualized Similarity (IBIS) metric. This similarity metric is beneficial in that it takes into account individual biases and constraints in a manner that is grounded in the cognitive mechanisms of decision making. To evaluate the IBIS metric, we also introduce a dataset of human categorizations of emails as being either dangerous (phishing) or safe (ham). This dataset is used to demonstrate the benefits of leveraging a cognitive model to measure the subjective similarity of human participants in an educational setting.

Cognitive models that represent individuals provide many benefits for understanding the full range of human behavior. One way in which individual differences emerge is through differences in knowledge. In dynamic situations, where decisions are made from experience, models built upon a theory of experiential choice (instance-based learning theory; IBLT) can provide accurate predictions of individual human learning and adaptivity to changing environments. Here, we demonstrate how an instance-based learning (IBL) cognitive model, implemented in a cognitive architecture (Adaptive Control of Thought-Rational), can be used to model an individual's decisions in a cybersecurity defense task, accounting for both population average and individual variances. The same IBL model structure with identical architectural parameters generates the full range of human behavior through stochastic memory retrieval processes operating over and contributing to unique experiences. Recurrence quantification analyses allow us to look beyond average behavior between and within individuals to sequential patterns of trial-to-trial behavior. We show how model-tracing and knowledge-tracing techniques can be used to align the model to an individual in real time to drive adaptive and personalized signaling algorithms for a cybersecurity defense system. We also present a method for introspecting into the cognitive model to gain further insight into the cognitive salience of features factored into individual decisions. The combination of techniques provides a blueprint for personalized modeling of individuals. We discuss the results and implications of this adaptive and personalized method for cybersecurity defense and more generally for intelligent artifacts tailored to individual differences in domains such as human-machine teaming.

Temporal credit assignment is crucial for learning and skill development in natural and artificial intelligence. While computational methods like the TD approach in reinforcement learning have been proposed, it's unclear if they accurately represent how humans handle feedback delays. Cognitive models intend to represent the mental steps by which humans solve problems and perform a number of tasks, but limited research in cognitive science has addressed the credit assignment problem in humans and cognitive models. Our research uses a cognitive model based on a theory of decisions from experience, Instance-Based Learning Theory (IBLT), to test different credit assignment mechanisms in a goal-seeking navigation task with varying levels of decision complexity. Instance-Based Learning (IBL) models simulate the process of making sequential choices with different credit assignment mechanisms, including a new IBL-TD model that combines the IBL decision mechanism with the TD approach. We found that (1) An IBL model that gives equal credit assignment to all decisions is able to match human performance better than other models, including IBL-TD and Q-learning; (2) IBL-TD and Q-learning models underperform compared to humans initially, but eventually, they outperform humans; (3) humans are influenced by decision complexity, while models are not. Our study provides insights into the challenges of capturing human behavior and the potential opportunities to use these models in future AI systems to support human activities.