Numerous traditional assessments have been developed to determine suitability of US military recruits for cyber careers. Cyber career field managers expressed a concern there may be well-qualified candidates that lack cyber knowledge, and therefore are not identified with knowledge-based tests. Technological advances such as serious gaming may provide opportunities to assess constructs traditional methods do not effectively measure. The purpose of this effort was to identify potential gains in validity that could be achieved beyond traditional methods through the use of serious games for several cyber jobs (both for enlisted and officer positions). Throughout this phase of research, an extensive literature review of military and civilian assessments targeted cyber occupations. Then, military subject matter experts in these career fields provided input and guidance (e.g., focus on aptitudes and traits as knowledge and skill are rapidly outdated). A gap analysis between all measures of such constructs identified a short list of candidates for measurement in a serious game. A survey of 800 airmen in the 1N4X1A, 3D1X2 and 17DEX/SX career fields was conducted; 290 respondents identified six constructs to be the focus for serious game assessment. The game was developed, and constructs validated on a sample chosen to model Air Force enlisted recruits. Additional psychometric data from enlistees and cyber trainees will be gathered once COVID-19 restrictions are lifted.

We present initial structural validity evidence for a serious game designed for personnel selection and classification for cybersecurity roles in the US Air Force (USAF). Based on literature review and input from USAF cybersecurity subject-matter-experts, we targeted six constructs for assessment. We describe the development process used to build a game to assess individual differences in these constructs, while also being engaging and motivating for players. We attend to the challenge of avoiding an overall game performance factor that dominates variance of multiple constructs scored from the same gameplay episodes and report steps taken to enhance discriminant validity of the scores. We apply factor analysis and item response theory models to develop scores that are reliable, show discriminant validity, and show modest education/gender group differences.

Advances at the intersection of artificial intelligence (AI) and education and training are occurring at an ever-increasing pace. On the education and training side, psychological and performance constructs play a central role in both theory and application. It is essential, therefore, to accurately determine the dimensionality of a construct, as it is often employed during both the assessment and development of theory, and its practical application. Traditionally, both exploratory and confirmatory factor analyses have been employed to establish the dimensionality of data. Due in part to inconsistent findings, methodologists recently resurrected the bifactor approach for establishing the dimensionality of data. The bifactor model is pitted against traditional data structures, and the one with the best overall fit (according to chi-square, root mean square error of approximation (RMSEA), comparative fit index (CFI), Tucker–Lewis index (TLI), and standardized root mean square residual (SRMR)) is preferred. If the bifactor structure is preferred by that test, it can be further examined via a suite of emerging coefficients (e.g., omega, omega hierarchical, omega subscale, H, explained common variance, and percent uncontaminated correlations), each of which is computed from standardized factor loadings. To examine the utility of these new statistical tools in an education and training context, we analyze data where the construct of interest is trust. We chose trust as it is central, among other things, to understanding human reliance upon and utilization of AI systems. We utilized the above statistical approach and determined the two-factor structure of widely employed trust scale is better represented by one general factor. Findings like this hold substantial implications for theory development and testing, prediction as in structural equation modeling (SEM) models, as well as the utilization of scales and their role in education, training, and AI systems. We encourage other researchers to employ the statistical measures described here to critically examine the construct measures used in their work if those measures are thought to be multidimensional. Only through the appropriate utilization of constructs, defined in part by their dimensionality, are we to advance the intersection of AI and simulation and training.

We describe a development process for serious games to create psychometrically rigorous measures of individual aptitudes (abilities, skills) and traits (habits, tendencies, behaviors). We begin with a discussion of serious games and how they can instantiate appropriate cognitive states for relevant aptitudes and traits to manifest. This can have numerous advantages over traditional assessment mo-dalities. We then describe the iterative approach to aptitude and trait measure-ment that emphasizes (1) careful definition and specification of the traits and ap-titudes to be measured, (2) rigorous assessment of reliability and validity, and (3) revision of gameplay elements and metrics to improve measurement properties.

Objective: To examine the utility of equal-variance signal detection theory (EVSDT) for evaluating and understanding human detection of phishing and spear-phishing e-mail scams. Background: Although the majority of cybersecurity breaches are due to erroneous responses to deceptive phishing e-mails, it is unclear how best to quantify performance in this context. In particular, it is unclear whether equal variances can safely be assumed in the SDT model, or, relatedly, whether degree of targeting, or threat level, primarily affects mean separation or evidence variability. Method: Through an online inbox simulation, the present research found that differences in susceptibility to phishing and spear-phishing e-mails could be carefully quantified with respect to detection accuracy and response bias through the use of an EVSDT framework. Results: The results indicated that EVSDT-based point metrics are effective for modeling and measuring phishing susceptibility in the inbox task, without the need for parameter estimation or model comparison involving unequal-variance SDT (UVSDT). Threat level modulated mean separation, with no effects on signal variances. Conclusion: These findings support the viability of using EVSDT to initially assess and subsequently monitor training effectiveness for phishing susceptibility, thereby providing measures that are superior to more intuitive metrics, which typically confound an individual's bias and accuracy. Effects of threat level mapped clearly onto distribution means with no effect on variances, suggesting phishing susceptibility primarily reflects temporally stable discriminative characteristics of observers. Notably, results indicated that people are particularly poor at identifying spear-phishing e-mail threats (demonstrating only 40% accuracy).

The persistently changing landscape of cyberspace and cybersecurity has led to a call for organizations' increased attention toward securing information and systems. Rapid change in the cyber environment puts it on a scale unlike any other performance environment typically of interest to industrial and organizational (I-O) psychologists and related disciplines. In this article, we reflect on the idea of keeping pace with cyber, with a particular focus on the role of practicing I-O psychologists in assisting individuals, teams, and organizations. We focus on the unique roles of I-O psychologists in relation to the cyber realm and discuss the ways in which they can contribute to organizational cybersecurity efforts. As highlighted throughout this article, we assert that the mounting threats within cyberspace amount to a "looming crisis." Thus, we view assisting organizations and their employees with becoming resilient and adaptive to cyber threats as an imperative, and practicing I-O psychologists should be at the forefront of these efforts.

Trust plays a central role in the effectiveness of work groups and teams. This is the case for both face-to-face and virtual teams. Yet little is known about the development of trust in virtual teams. We examined cognitive and affective trust and their relationship to team effectiveness as reflected through satisfaction with one’s team and task performance. Latent growth curve analysis reveals both trust types start at a significant level with individual differences in that initial level. Cognitive trust follows a linear growth pattern while affective trust is overall non-linear, but becomes linear once established. Latent change score models are utilized to examine change in trust and also its relationship with satisfaction with the team and team performance. In examining only change in trust and its relationship to satisfaction there appears to be a straightforward influence of trust on satisfaction and satisfaction on trust. However, when incorporated into a bivariate coupling latent change model the dynamics of the relationship are revealed. A similar pattern holds for trust and task performance; however, in the bivariate coupling change model a more parsimonious representation is preferred.

Serious games are an attractive tool for education and training, but their utility is even broader. We argue serious games provide a unique opportunity for research as well, particularly in areas where multiple players (groups or teams) are involved. In our paper we provide background in several substantive areas. First, we outline major constructs and challenges found in team research. Secondly, we discuss serious games, providing an overview and description of their role in education, training, and research. Thirdly, we describe necessary characteristics for game engines utilized in team research, followed by a discussion of the value added by utilizing serious games. Our goal in this paper is to argue serious games are an effective tool with demonstrated reliability and validity and should be part of a research program for those engaged in team research. Both team researchers and those involved in serious game development can benefit from a mutual partnership which is research focused.

Objective: Information technology has rapidly changed work in the United States in the 21st century. Healthcare, however, is one industry that has lagged behind in IT investment for a variety of reasons. Recent federal initiatives to encourage IT adoption in the healthcare industry provide an ideal context to study factors that influence technology acceptance.Method: Data from 261 practicing pediatricians were collected to evaluate an extended Technology Acceptance Model. We employed structural equation modeling to statistically test three theoretically plausible models.Results: Results indicated that individual, organizational, and device characteristics collectively influence pediatricians' intention to adopt tablet computers in their medical practice. Subjective norms, compatibility, and reliability explain 72% of the variance in perceived usefulness. Additionally, compatibility and reliability explain 38% of the variance in perceived ease of use.Conclusions: These results extend the literature on technology adoption by modeling determinants of the two core attitudinal constructs in the Technology Acceptance Model. Understanding these constructs will facilitate the adoption of technology and the management of health policies. (C) 2016 Fellowship of Postgraduate Medicine. Published by Elsevier Ltd. All rights reserved.

Problem. Teams or groups of individuals working together to achieve a shared goal, make up today’s world of work. Although the literature is rife with issues concerning teams, there is no coherent structure to guide researchers wishing to gain a deeper understanding into those factors leading to positive team outcomes. Question. This is due in part to two factors; one being the methods (e.g., observation; self report) typically employed to study teams often do not apply rigorous standards for reliability and validity. The second is it is difficult to construct data gathering situations that realistically approach the context in which teams operate. Approach. Addressing the first issue, we present a framework for the type of data that should be gathered to reliably and validly evaluate team performance. We believe the second issue is best addressed through the application of serious games, where realistic scenarios are delivered representing problems similar to those faced by teams in the real world. Finally, we describe a study whereby we demonstrate the approach utilizing a serious game to gather the data, which is then analyzed to assess the reliability and validity of the measures. Conclusion. With reliability and validity being established, a latent change score model is presented to illustrate how rich models of team interaction can be stated, investigated, and statistically assessed; since the serious game ensures the quality of the team interaction and the superior quality of the data.

BACKGROUND Medical residents receive both medical education and clinical skills training. New technologies and pedagogies are being developed to address each of these phases. Our research focuses on the efficacy of an iPad(®) (Apple, Cupertino, CA) for clinical skills training. MATERIALS AND METHODS For a period of 3 years, the University of South Florida provided incoming pediatric residents (n=94) with an iPad. At the end of the 3-year program, we surveyed the residents, measuring perceptions and satisfaction of iPad use in clinical training. RESULTS Sixty percent of the residents responded to the survey. Ninety-three percent reported at least some iPad usage per day on clinical activities. We classified 13 facets of clinical training into three conceptual areas and provided figures detailing iPad use for each facet relative to other facets in the same cluster. The obtaining, management, and display of information are primary uses of iPad applications in clinical training. Finally, we provide information relative to perceived obstacles in clinical training, with weight of the device being the most frequently cited. CONCLUSIONS The role of graduate medical education is changing with the introduction of new technologies. These technologies can differentially impact the various aspects of residency education and training. Residents reported using an iPad extensively in their clinical training. We argue that in addition to impacting traditional educational strategies, iPads can successfully facilitate aspects of clinical training in medical education.

Today’s medical students are digital natives who, for their entire life, have been surrounded by digital technology. Our research focuses on a tablet computer’s usability in medical education, and the subsequent transfer from the classroom to the work environment. For a period of three years, all incoming pediatric residents at a large southeastern university were provided an iPad. At the end of the 3-year program, we surveyed the residents measuring perceptions of iPad use and satisfaction. Fifty-six (60%) of the residents responded to the survey. A statistically significant number reported an increased amount of time spent with the tablet throughout their medical education. Similarly, a significant difference exists between those who believe the device to be a necessary part of medical education versus those stating it would be nice but not necessary. We present figures detailing how three conceptual areas: receiving information, inputting information, and collaboration (consisting of ten different facets of the tablet’s use) impacted their medical education. Residents throughout their medical education use the tablet extensively. There is variance in the areas where the tablet is the preferred tool versus a smartphone or computer. A clear majority of students expect to transition the tablet into their workplace upon completing residency. We argue a tablet is a useful tool for graduate medical education and later medical practice.

This report examines effects of a coparenting intervention designed for and delivered to expectant unmarried African American mothers and fathers on observed interaction dynamics known to predict relationship adjustment. Twenty families took part in the six-session "Figuring It Out for the Child" (FIOC) dyadic intervention offered in a faith-based human services agency during the third trimester of the mother's pregnancy, and completed a postpartum booster session 1 month after the baby's arrival. Parent referrals for the FIOC program were received from a county Health Department and from OBGYNs and Pregnancy Centers in the targeted community. All intervention sessions were delivered by a trained male-female paraprofessional team whose fidelity to the FIOC manualized curriculum was independently evaluated by a team of trained analysts. At both the point of intake ("PRE") and again at an exit evaluation completed 3 months postpartum ("POST"), the mothers and fathers were videotaped as they completed two standardized "revealed differences" conflict discussions. Blinded videotapes of these sessions were evaluated using the System for Coding Interactions in Dyads. Analyses documented statistically significant improvements on 8 of 12 variables examined, with effect sizes ranging from moderate to large. Overall, 14 families demonstrated beneficial outcomes, 3 did not improve, and 3 showed some signs of decline from the point of intake. For most interaction processes, PRE to POST improvements were unrelated to degree of adherence the paraprofessional interventionists showed to the curriculum. However, better interventionist competence was related to decreases in partners' Coerciveness and Negativity and Conflict, and to smaller increases in partner Withdrawal. Implications of the work for development and delivery of community-based coparenting interventions for unmarried parents are discussed.