Modern artificial intelligence (AI) and machine learning (ML) systems have become more capable and more widely used, but often involve underlying processes their users do not understand and may not trust. Some researchers have addressed this by developing algorithms that help explain the workings of the system using ‘Explainable’ AI algorithms (XAI), but these have not always been successful in improving their understanding. Alternatively, collaborative user-driven explanations may address the needs of users, augmenting or replacing algorithmic explanations. We evaluate one such approach called “collaborative explainable AI” (CXAI). Across two experiments, we examined CXAI to assess whether users’ mental models, performance, and satisfaction improved with access to user-generated explanations. Results showed that collaborative explanations afforded users a better understanding of and satisfaction with the system than users without access to the explanations, suggesting that a CXAI system may provide a useful support that more dominant XAI approaches do not.

This Report is a companion to the Report titled "Requirements for the Evaluation of Human-AI Work Systems." Whereas that Report focused on the minimum necessary empirical requirements for the assessment of AI systems, this Report provides additional recommendations and technical details to assist the developers of AI systems. Recommendations are presented covering study design, research methods, measurement, statistical analyses, and online experimentation. This guidance should be applicable to all research intended to evaluate the effectivity of AI systems.

The development of AI systems represents a significant investment. But to realize the promise of that investment, performance assessment is necessary. Empirical evaluation of Human-AI work systems must adduce convincing empirical evidence that the work method and its AI technology are learnable, usable, and useful. The theme to this Report is the notion that AI assessment must be effective but must also be efficient. Bench testing of a prototype of an AI system cannot require extensive series of experiments with complex designs. Thus, the empirical requirements that are presented in this Report involve escaping some of the constraints that are imposed in traditional laboratory research. Also, there is a recognition of new constraints that are unique to AI evaluation contexts. Empirical requirements are presented covering study design, research methods, statistical analyses, and online experimentation. The 15 requirements presented in this Report should be applicable to all research intended to evaluate the effectivity of AI systems.

Trust in automation, is of concern in computer science and cognitive systems engineering, as well as the popular media (e.g., Chancey et al., 2015; Hoff and Bashir 2015; Hoffman et al., 2009; Huynh et al., 2006; Naone, 2009; Merritt and Ilgen, 2008; Merritt et al. 2013, 2015a; Pop et al. 2015; Shadbolt, 2002; Wickens et al. ,2015; Woods and Hollnagel 2006). Trust is of particular concern as more AI systems are being developed and tested (Schaefer et al., 2016).

The key to effective 21st century intelligence is our sensemaking process. In this article, we present a case for why Cold War-era reductive intelligence models have become obsolete, and we show th...

This chapter describes the "state of the art" of the psychology of proverbs. It reviews the empirical work on proverbs. This work was conducted in the absence of a well developed conception of proverbs. The extensional meaning of a proverb consists not simply of already encountered instances, but of many possible instances, as these are coded by the interpretation. In general, the interpretations of proverbs take the form of abstract structures (conceptual bases). These structures function as theories. The chapter suggests directions for a theory of proverb understanding. Toward this end, and to highlight the issues involved, proverbs will be compared with metaphors. The comparison of metaphors and proverbs breaks down into two not unrelated problems—how they are identified and how they are understood. Both metaphors and proverbs are expressed through a large variety of structures.

Weather forecasting is a highly technical domain, relying on many kinds of technologies and many kinds of data. This chapter summarizes four research programs ranging from organizational to individual analyses to provide unique, complementary insights about expertise. Forecaster learning was highly dependent on the forecasters’ context, and whether they gained expertise depended on having a strong personal identity as a forecaster. How forecasters coped with risk and uncertainty suggests expertise is not just a deeper causative understanding, but an ability to provide an accurate forecast that serves users well. Novice forecasters employ rule-based knowledge, while experts employ a fluid and flexible application of knowledge. Expert forecasters have extensive, complex knowledge about each type of weather process they forecast, a knowledge that may be lost if not captured and passed on to the next generation. This empirical work on professional activity in context has the potential to invigorate studies of expertise.

Abstract Proficiency scaling in the domain of intelligence analysis converges on an answer to the question of what counts as expertise in this domain. Proficiency scales in the domain are based on what are called essential competencies. There are many distinct analytical roles, entailing a specialization of expertise. This chapter discusses macrocognitive models of analyst reasoning and knowledge as a function of proficiency level, including recognition-primed decision making and intuition. This chapter also considers individual differences and conceptualize the different styles to be relatively stable and distinctive approaches to critical thinking. Proficiency scaling entails the issue of whether intelligence analysts are prone to cognitive biases. Analysts must cope with the problem of indeterminate causation, that is, the understanding of events for which there is no single cause, and causal forces include human agency and motivations. Directives in the intelligence community call for robust performance measures, but measuring analyst procedural skills is non-trivial. Finally, the implications for training are discussed.

Mentoring and coaching are effective and sometimes necessary strategies to foster the development of expertise. However, these learning enhancement methods are multifactorial and often ambiguously characterized. The goal of this chapter is to unpack this complex interactional relationship to more fully describe what it means to be an effective coach/mentor, and to better understand their role in developing expertise. Specifically, this chapter provides: (1) a detailed description of the different components and functions of mentoring, coaching, and preceptorship; (2) a summary of the meta-analytic evidence supporting the effectiveness of developmental support roles on a range of outcome measures (e.g., job performance); (3) a review of the empirical evidence supporting the development of expertise in mentoring-type roles; and (4) concluding remarks summarizing some of the major issues, and suggestions for how to advance this area of research.

This introduction presents an overview of the key concepts discussed in the subsequent chapters of this book. The book describes the trends of thought in some of the disciplines, with a focus on experimental psychology, psychodynamic psychology, linguistic theory, and philosophy. It discusses certain "landmark papers" within these disciplines that appear to be relevant to the Zeitgeist in cognitive psychology in the 1970s. The book reviews treatments of figurative language in linguistics and philosophy and presents the major theories of metaphor in philosophy from Plato and I. Kant to modern times, including the logical positivists' view that metaphor possesses connotative value but not truth value. It also reviews standard theories of metaphor comprehension from the perspective of attitudes about epistemology, such as Phenomenalism and Realism. The book argues that there may be a trade-off between abstract understanding and imaging, and that their results define boundary conditions on the role of imagery as described by the dual-coding view.

This chapter begins with summaries of the views of some philosophers and scientists—pro and con—about the status and use of metaphors in scientific theories. It provides an examination of some specific scientific metaphors, their character and foibles, which leads to an attempt at a description of the place of a metaphor in a theory. The chapter discusses the example metaphors from many sciences, especially psychology, but it also relies on some well-worn examples from physics. Some in philosophy of science and in the philosophy of psychology feel that metaphor is no good when used in a theory. In a broader psychological context of science, general problem-solving, many treatments of problem-solving strategies describe the use of metaphor in the creation of solutions. Metaphor may be inevitable and necessary to science, and cognitively prior to scientific description, because of psychological factors in learning, inference-making, symbol-formation, and explanation.

This chapter is a historical review of a variety of conceptual approaches that system developers have adduced to guide the design of human-machine systems. The goal of this chapter is to convince cognitive systems engineers, human factors engineers, and systems developers more broadly that macrocognitive work systems must not be designed around methods of task allocation and schemes based on levels of automation. An alternative approach is proposed that regards the human and the machine as operating in a number of different interdependence relations.

This chapter takes a historical perspective on examining the utility and validity of various introspective-type methods. First, specific contributions of some of the pioneers of introspective methods are considered, highlighting key motivations and arguments that have spurred methodological evolution over the past 100+ years. Next, current methods of thinking aloud are reviewed. Then the types of verbal reports of thinking used to study expertise are described. In the penultimate section, some guidance on using these methods is offered. In the concluding part, a summary of the key recommendations is provided, and some thoughts on the future of introspection methodology that, it is hoped, will improve the state-of-the-science and escape the legacies of behaviorism are offered.

Interpreting remote sensing imagery , Interpreting remote sensing imagery , کتابخانه دیجیتال جندی شاپور اهواز

The identification of experts is crucial in many research projects and application areas for intelligent systems, including the development of rational algorithms and the creation of knowledge bases. This essay addresses the question of how to identify experts, offering a method that is more robust and scientifically grounded than the common reliance on the so-called "ten-year" or "10,000 hours" rules for deciding who is, and who is not an expert.

Abstract In this chapter, we reflect on the themes that emerged throughout this Handbook. First, we review why expertise is not always revered and ask whether this relates to the way expertise has been defined and measured. We then re-examine definitions of expertise presented throughout the Handbook as well as the idea that expertise is, in part, about increasing one’s cognitive ability to adapt to complexity. Next, we take a look at where we have been, as a community of communities of expertise researchers, and whether we are heading in good directions, placing an especial focus on how expertise is and should be measured. In the penultimate section, we present some ideas about future areas of research recommended by chapter authors. Finally, we present a potential way forward for researchers to continue to move the field of expertise studies in a positive direction and, ultimately, to better prepare individuals to operate effectively in tomorrow’s workplace.