Despite their intentioned focus on process, macrocognitive models run the risk of advancing a perspective on human behavior that can be called “composite/constraints”—that is, the various, long-prevailing views that people are nothing but composites of factors and variables who behave according to their particular makeup and/or their circumstances. The perspective neglects a key feature of the human experience, primarily because it is mostly invisible to both the researcher and their participants. This paper introduces this invisible feature and explores its integration with a core macrocognitive model, Klein’s Recognition-Primed Decision model. It also suggests implications of this analysis for the theoretical and methodological concerns of the NDM paradigm, applications of NDM models, and for an understanding of artificial general intelligence.

This is the first in a series of articles covering advancements in eAssessment. The series will feature educators, developers, and researchers from around the world who are innovating how learning is assessed while meeting the challenges of efficiency, scalability, usability, and accessibility.

This chapter reveals how concept mapping has provided insight into learner thinking and progress since 1972. Despite its broad appeal, however, concept mapping has not yet reached ubiquity as a technique for assessment – primarily due to several challenges of implementation and limitations on further innovation of the original approach. In its 50th anniversary, a critical question remains: How can the process of concept mapping as a data collection technique enable the assessment of cognitive structures so that teaching may proceed accordingly, at scales that create genuine benefit? This chapter aims to answer this question by highlighting four challenges and presenting innovations designed to overcome them that have resulted in compelling findings and the development of a software tool to enable large-scale assessment based on concept mapping.

This is the first in a series of articles covering advancements in eAssessment. The series will feature educators, developers, and researchers from around the world who are innovating how learning is assessed while meeting the challenges of efficiency, scalability, usability, and accessibility.

Authors for this special issue presented their work, perspectives, and recommendations that look to move learning assessment at all levels "beyond multiple-choice."

The U.S. Army sought to develop capabilities that allow for the automated or semi-automated, with greatly reduced human involvement, creation of tests and assessments. In recognizing the potential for an assessment approach that goes beyond multiple-choice, the Army chose our team to introduce and evaluate automated capabilities to author concept mapping-based assessments. This paper describes our initial approaches toward introducing efficiencies into the authoring process for concept map-based assessments. We are developing and evaluating methods to automatically generate concept maps from a knowledge domain and convert the maps into assessments for formative and summative purposes. Our initial work has sought to overcome challenges as we introduced artificial intelligence into the authoring process. In this paper, we describe our emergent approach and the challenges we have faced in seeking efficiencies in the conversion of text to concept maps.

Given the substantial increase in interest and activity in the virtual care space, the Department of Veterans Affairs (VA) is making a concerted effort to increase opportunities for Veterans to adopt virtual care applications. We investigated Veteran attitudes, interests, and needs for virtual care applications, including privacy considerations and specific types of data they would be comfortable sharing with their providers through various Veteran-facing tools. We administered a 28-item questionnaire to 40 Veterans and performed follow-up interviews with a subset of nine Veterans. Study results revealed broad support by Veterans for using virtual care applications, including wearable devices, for sharing all types of health information with their providers. Further, Veterans expressed a desire for consolidation across VA-provided virtual care tools. Our findings will inform a strategy for Veteran-facing tools. Other health care systems may be interested in exploring these topics, based on our study results.

A insatisfação de professores e alunos com o formato atual do processo de ensino-aprendizagem tem gerado a busca por inovações pedagógicas. Este trabalho apresenta os mapas conceituais com erros (MCE) como uma atividade de avaliação diagnóstica de rápida aplicação, permitindo ao professor oferecer devolutivas imediat as e personalizadas aos alunos. Isso pode estimular a ressonância pedagógica e favorecer a aprendizagem significativa. As atividades usando MCE podem ser facilmente incorporadas aos ambientes virtuais de aprendizagem, potencializando sua utilização em larga escala.

Concept maps (Cmaps) have been successfully used to make knowledge structures visible. During Cmap task elaboration, novice students are likely to suffer cognitive overload, and they might avoid coping with difficult contents staying in his semantic safe territory. The authors have developed an innovative approach using Cmaps with embedded errors applied on Sero! – a cloud-based knowledge assessment platform. This chapter presents a case study involving the current use of Cmaps with errors as an assessment task capable of identifying misconceptions about the advances of molecular biology. Undergraduate students (n=86) were asked to find the errors hidden into the propositional network. The results confirmed the task challenged the students to go beyond their safe semantic territory. Misconceptions were readily identified from the students' answers providing good insights for the development of a bespoke feedback. The current data available is enough to foresee a broad range of research opportunities to readers interested in concept mapping, instruction and learning analytics.

Mental models have long been considered an important enabler of cognitive performance and a key to understanding educational progress. Yet assessing mental models using straightforward, valid, reliable, and efficient methods remains an elusive challenge. Research suggests that concept mapping holds the promise of the direct analysis of mental models. We stress mapping to emphasize a goal of assessing a dynamic thing (mental models) through the use of a process (concept mapping). Our supposition is that concept mapping-based methods can be used to assess mental models. However, the practicality and feasibility of conducting concept mapping-based assessment (CMA) has to date stifled widespread application of the approaches. To meet this challenge, we have developed a system, Sero!, that implements a robust CMA process in a cloud-based platform. This paper presents an overview and advantages of mental model assessment, a rationale in support of the use of CMA, our development of Sero!, and makes a case for the use of Sero!’s specific instantiation of a CMA as a scalable approach that can support many applications in the assessment of mental models. A demonstration of the implementation in a large training event is presented to bolster the case.

Training and development programs within corporate environments have seen a significant expansion in recent years. Yet there is little research regarding workplace learning assessments. Learning assessments could be used as a tool to gauge employee knowledge in a specific domain area, which can be useful in identifying areas of strengths and weaknesses and indicate where improvement is needed. One way to effectively and efficiently assess workplace learning could be with Concept Maps, a diagramming technique of knowledge representation that has been used to capture the mental models of domain experts. Concept Maps can in turn be used to assess the progress of learners through training and learning curricula. Although the actual set-up and analysis for Concept Maps tend to be rather time consuming and laborious, Sero!, a developing learning assessment platform, holds promise for realizing efficiencies in their use to enable deeper learning assessment. This paper explores the literature about workplace learning assessment, reviews the applicability of Concept Maps for such assessment, summarizes the user-centered design and development of Sero!, and reflects on the boundary conditions for its use.

After nearly two decades of knowledge preservation activity, relatively little work has explored the organizational use of Concept Maps elicited from experts. This paper describes an attempt to get back to the roots of Concept Mapping as a means of both representing and evaluating knowledge, in the context of professional work. It describes a pilot project in which the authors used Concept Mapping to elicit and represent knowledge from domain experts, then demonstrated the use of Concept Maps for assessing the mental models of other professionals. The authors introduce Sero! – a prototype Concept Map-based learning assessment platform, and a general model for the organizational use of Concept Mapping and Concept Maps.

Purpose: Computerized clinical decision support systems (CDSS) are an emerging means for improving healthcare safety, quality and efficiency, but meta-analyses findings are mixed. This meta-synthesis aggregates qualitative research findings as possible explanations for variable quantitative research outcomes.Inclusion criteria: Qualitative studies published between 2000 and 2013 in English, involving physicians, registered and advanced practice nurses' experience of CDSS use in clinical practice were included. Search strategy: PubMed and CINAHL databases were searched. Study titles and abstracts were screened against inclusion criteria. Retained studies were appraised against quality criteria. Findings were extracted iteratively from studies in the 4th quartile of quality scores. Two reviewers constructed themes inductively. A third reviewer applied the defined themes deductively achieving 92% agreement.Results: 3798 unique records were returned; 56 met inclusion criteria and were reviewed against quality criteria. 9 studies were of sufficiently high quality for synthetic analysis. Five major themes (clinician-patient-system integration; user interface usability; the need for better 'algorithms'; system maturity; patient safety) were defined.Conclusions: Despite ongoing development, CDSS remains an emerging technology. Lack of understanding about and lack of consideration for the interaction between human decision makers and CDSS is a major reason for poor system adoption and use. Further high-quality qualitative research is needed to better understand human-system interaction issues. These issues may continue to confound quantitative study results if not addressed. (C) 2015 Published by Elsevier Ireland Ltd.

A key aspect of a website or any artifact is its usability—the ability of the artifact’s target audience to carry out tasks safely, effectively, efficiently, even joyfully. One class of usability evaluation methods is inspection methods, in which the usability professional systematically inspects the user interface to discern potential usability problems. Here the article proposes employing Concept Mapping, a proven method of knowledge elicitation and representation, as a new, structured usability inspection method. Nineteen students in a master’s-level usability class each generated a Concept Map (Cmap) of 1 of 5 websites. These Cmaps were shared with the sites’ webmasters, and the webmasters completed a questionnaire giving us feedback on the value of the Cmaps for subsequent site redesigns. The article presents those data, infers what improvements need to be made in the new Concept Mapping Usability Evaluation method, and invites others to join us in investigating the potential value of this method.

Naturalistic Decision Making (NDM) offers perspectives and methods for understanding cognitive performance, especially expertise. Knowledge management (KM) is a multi-disciplinary field professing to improve organizational performance by making the best use of knowledge. Since expertise is one of the most valuable assets in any organization, it stands to reason that adopting the perspectives and methods of NDM as a KM paradigm – i.e., Expertise Management – should enable organizations to realize performance improvements. Yet, Expertise Management has not achieved recognition as a KM strategy, and attempts to implement it have been met with significant methodological, practical, and competitive challenges. This paper examines the case for NDM-based Expertise Management as a core KM strategy, and the methodological, practical and competitive challenges for adoption. The authors draw on their collective professional experience in attempting to implement Expertise Management at a diverse range of organizations, and conclude with recommendations for future directions.

Ensuring that unmanned aerial systems’ (UAS) control stations include a tight coupling of systems engineering with human factors, cognitive analysis, and design is key to their success. We describe a combined cognitive task analysis (CTA) and design thinking effort to develop interfaces for an operator controlling an autonomous helicopter, a prototype system that the Office of Naval Research is developing. We first conducted CTA interviews with subject-matter experts having expertise in UAS flight operations, helicopter resupply, military ground forces, and marine airspace control. Data informed the development of analysis products, including human–system interface requirements, which drove the creation of design concepts through ideation sessions using design thinking methods. We validated and refined the design concepts with UAS pilots. We provide an overview of our process, illustrated by details of a timeline display development. Significant aspects of our work include close integration of CTA and design thinking efforts, designing for an “envisioned world” of interaction with highly autonomous helicopter systems, and the importance of knowledge elicitation early in system design. This effort represents a successful demonstration of an innovative design process in developing UAS interfaces.

This paper reports on a research project that combined cognitive task analysis (CTA) methods with innovative design processes to develop a handheld device application enabling a non-aviator to interact with a highly autonomous resupply helicopter. In recent military operations, unmanned helicopters have been used to resupply U.S. Marines at remote forward operating bases (FOBs) and combat outposts (COPs). This use of unmanned systems saves lives by eliminating the need to drive through high-risk areas for routine resupply. The U.S. Navy is investing in research to improve the autonomy of these systems and the design of interfaces to enable a non-aviator Marine to safely and successfully interact with an incoming resupply helicopter using a simple, intuitive handheld device application. In this research, we collected data from multiple stakeholders to develop requirements, use cases, and design storyboards that have been implemented and demonstrated during flight tests in early 2014.

The growing interest of rotary wing UAVs, for military and civilian applications, has encouraged designers to consider miniaturized configurations, more efficient in terms of endurance, payload capability and maneuverability. The purpose of this paper is to study a new configuration of coaxial rotor as applied to a micro aerial vehicle (MAV) with the intention to guarantee the vehicle maneuverability while removing unnecessary control surfaces which would increase wind gust sensitivity. Coaxial rotor configurations maximize the available rotor disk surface and allow for torque cancelation. Tilting rotors may allow for the vehicle control.

Cognitive task analysis (CTA) methods are most widely known for their contributions to military, nuclear power plant, and aviation research. In recent years, however, these methods have been adapted and applied with increasing frequency to address issues in healthcare. CTA methods have been used in the context of designing and integrating health information technology, in pursuit of improved patient safety, and as a means to improve education and training. This panel will 1) reflect on strategies for tailoring CTA methods for use in a range of healthcare settings, 2) highlight challenges to conducting CTA in healthcare settings, and 3) highlight important contributions of CTA in addressing challenging issues in healthcare today.