In complex crises, coordination between organizations is challenging. Knowledge needed to coordinate, like responsibilities, capabilities and interdependencies between tasks are often not known or not communicated systematically. As a result, coordination develops gradually and causes confusion. In this paper we describe an approach and tool called 'Profiler', that focuses on quickly increasing knowledge and understanding about the participating organizations while preparing for, or at the beginning of a crisis. Profiler was evaluated during an exercise of 1 Civil Military Coordination Battalion (1CIMICbat). Teams consisting of functional specialists performed a damage and needs assessments after a flooding. The results show that participants that used Profiler increased their knowledge and integrated understanding, when this was initially lacking. Further, participants with improved knowledge and integrated understanding, coordinated more within and between teams, when they perceived to be interdependent. Our results point in the direction that coordination effectiveness and efficiency may be improved with our approach. et al.; Information Systems for Crisis Response and Management (ISCRAM); Iterop; KIMOCE; Safe Cluster; Vinovalie - Les Vignerons Dovalie

In disaster recovery, responding professional organizations traditionally assess the needs of communities following a disaster. Recent disasters have shown that volunteer capacities within the community are not yet integrated in recovery activities. To improve the efficiency of responding professionals and utilize the potential capacity from within the community, a platform is needed that identifies needs and capacities and provides situational overviews of recovery activities for different stakeholders. The proposed COBACORE platform aims to 1) bring community needs and capacities directly together, 2) allow professionals to better maintain awareness of recovery activities and to better deploy their capacities and 3) facilitate collaboration between professionals and responding communities. For each function and feature, the Human-Computer Interaction (HCI) challenges are outlined. In ongoing work, a first prototype of this platform is implemented and evaluated with stakeholders in simulated disaster recovery activities.

This study presents a framework for understanding task and psychological factors affecting reliance on advice from decision aids. The framework describes how informational asymmetries in combination with rational, motivational and heuristic factors explain human reliance behavior. To test hypotheses derived from the framework, 79 participants performed an uncertain pattern learning and prediction task. They received advice from a decision aid either before or after they expressed their own prediction, and received feedback about performance. When their prediction conflicted with that of the decision aid, participants had to choose to rely on their own prediction or on that of the decision aid. We measured reliance behavior, perceived and actual reliability of self and decision aid, responsibility felt for task outcomes, understandability of one's own reasoning and of the decision aid, and attribution of errors. We found evidence that (1) reliance decisions are based on relative trust, but only when advice is presented after people have formed their own prediction; (2) when people rely as much on themselves as on the decision aid, they still perceive the decision aid to be more reliable than themselves; (3) the less people perceive the decision aid's reasoning to be cognitively available and understandable, the less people rely on the decision aid; (4) the more people feel responsible for the task outcome, the more they rely on the decision aid; (5) when feedback about performance is provided, people underestimate both one's own reliability and that of the decision aid; (6) underestimation of the reliability of the decision aid is more prevalent and more persistent than underestimation of one's own reliability; and (7) unreliability of the decision aid is less attributed to temporary and uncontrollable (but not external) causes than one's own unreliability. These seven findings are potentially applicable for the improved design of decision aids and training procedures.

Naval tactical picture compilation is a task for which allocation of attention to the right information at the right time is crucial. Performance on this task can be improved if a support system assists the human operator. However, there is evidence that benefits of support systems are highly dependent upon the systems' tendency to support. This paper presents a study into the effects of different levels of support conservativeness (i.e., tendency to support) and human competence on performance and on the human's trust in the support system. Three types of support are distinguished: fixed, liberal and conservative support. In fixed support, the system calculates an estimated optimal decision and suggests this to the human. In the liberal and conservative support types, the system estimated the important information in the problem space in order to make a correct decision and directs the human's attention to this information. In liberal support, the system attempts to direct the human's attention using only the assessed task requirements, whereas in conservative support, the this attempt is done provided that it has been estimated that the human is not already paying attention (more conservative). Overall results do not confirm our hypothesis that adaptive conservative support leads to the best performances. Furthermore, especially highcompetent humans showed more trust in a system when delivered support was adapted to their specific needs.

Computational models of attention can be used as a component of decision support systems. For accurate support, a computational model of attention has to be valid and robust. The effects of task performance and task complexity on the validity of three different computational models of attention were investigated in an experiment. The gaze-based model uses gaze behavior to determine where the subject's attention is, the task-based model uses information about the task and the combined model uses both gaze behavior and task information. While performing a tactical compilation task, participants had to indicate to what set of objects their attention was allocated. The indications of the participants were compared with the estimations of the three models. The results show that overall, the estimation of the combined model was better than that of the other two models. Contrary to what was expected, the performance of the models was not different for good and bad performers and was not different for a simple and complex scenario. The difference in complexity and performance might not have been strong enough. Further research is needed to determine if improvement of the combined model is possible with additional features and if computational models of attention can effectively be used in decision support systems. This can be done using a similar validation methodology as presented in this paper.

Several challenges can be identified for work on future naval platforms. Information volumes for navigation, system monitoring, and tactical tasks will increase as the complexity of the internal and external environment also increases. The trend of reduced manning is expected to continue as a result of economic pressures and humans will be responsible for more tasks, tasks with increased load, and tasks with which they will have little experience. Problems with attention allocation are more likely to occur when more has to be done with less. To avoid these attention allocation problems, in this paper it is proposed that humans are supported by cooperative agents capable of managing their own and the human’s allocation of attention. It is expected that these attention managers have a significant impact: when attentional switches between tasks or objects are often solicited, where the human’s lack of experience with the environment makes it harder for them to select the appropriate attentional focus, or where an inappropriate selection of attentional focus may cause serious damage. In domains like air traffic control or naval tactical picture compilation these properties are found. The main contribution of the present paper is the description of the combined approach of design and validation for the development of applied cooperative agent-components. The design requirements are given of an agent-component called Human Attention-Based Task Allocator (HABTA). This component enables the agent to support the human-agent team by managing attention allocation of the human and the agent. The HABTA-component does this by reallocation of the human’s and agent’s focus of attention to tasks or objects based on an estimation of the current human allocation of attention and by comparison of this estimation with normative rules. In Figure 1 the design overview of a HABTA-component is shown that corresponds to the above mentioned design requirements. The setting in this particular overview is a naval officer behind an advanced future integrated command and control workstation and compiling a tactical picture of the situation. If the agent cooperatively assists the officer, than the agent should have a descriptive (Requirement 1) and normative model (Requirement 2). When the operator allocates his attention to certain objects or tasks that also require to receive attention, the outcome of both models should be comparable. This means that output of the models should not differ more than a certain threshold. The output of the two models in the example shown in Figure 1 are clearly different: in the top-left image, the operator is attending to different objects and corresponding tasks than the top-right image indicates as being required (see red arrows). Because of this discrepancy, which the HABTA-component should be able to determine (Requirement 3), an adaptive reaction by the agent is triggered (Requirement 4). This means that, for instance, the agent either will draw attention to the proper region or task through the workstation, or it will allocate its own attention to this region and starts executing the tasks related to that region, for the given situation. Requirements 1–4 constrain the HABTA-component architecture, but in order to make it effective, two types of questions should be answered: 1) are the descriptive and prescriptive models accurate enough, and 2) does the use of the HABTA-component based on those models lead to significant improvements? These two questions can be translated into two complementary validation experiments. The first experiment

This paper addresses the development of an adaptive cooperative agent in a domain that suffers from human error in the allocation of attention. The design is discussed of a component of this adaptive agent, called Human Attention-Based Task Allocator (HABTA), capable of managing agent and human attention. The HABTA-component reallocates the human’s and agent’s focus of attention to tasks or objects based on an estimation of the current human allocation of attention and by comparison of this estimation with certain normative rules. The main contribution of the present paper is the description of the combined approach of design and validation for the development of such components. Two complementary experiments of validation of HABTA are described. The first experiment validates the model of human attention that is incorporated in HABTA, comparing estimations of the model with those of humans. The second experiment validates the HABTA-component itself, measuring its effect in terms of human-agent team performance, trust, and reliance. Finally, some intermediary results of the first experiment are shown, using human data in the domain of naval warfare.

This paper argues that it is important to study issues concerning trust and reliance when developing systems that are intended to augment cognition. Operators often under-rely on the help of a support system that provides advice or that performs certain cognitive tasks autonomously. The decision to rely on support seems to be largely determined by the notion of relative trust. However, this decision to rely on support is not always appropriate, especially when support systems are not perfectly reliable. Because the operator's reliability estimations are typically imperfectly aligned or calibrated with the support system's true capabilities, we propose that the aid makes an estimation of the extent of this calibration (under different circumstances) and intervenes accordingly. This system is intended to improve overall performance of the operator-support system as a whole. The possibilities in terms of application of these ideas are explored and an implementation of this concept in an abstract task environment has been used as a case study.

One of the goals of augmented cognition is creation of adaptive human-machine collaboration that continually optimizes performance of the human-machine system. Augmented Cognition aims to compensate for temporal limitations in human information processing, for instance in the case of overload, cognitive lockup, and underload. Adaptive behavior, however, may also have undesirable side effects. The dynamics of adaptive support may be unpredictable and may lead to human factors problems such as mode errors, 'out-of-the-loop' problems, and trust related issues. One of the most critical challenges in developing adaptive human-machine collaboration concerns system mitigations. A combination of performance, effort and task information should be taken into account for mitigation strategies. This paper concludes with the presentation of an iterative cognitive engineering framework, which addresses the adaptation strategy of the human and machine in an appropriate manner carefully weighing the costs and benefits.

This paper involves a human-agent system in which there is an operator charged with a pattern recognition task, using an automated decision aid. The objective is to make this human-agent system operate as effectively as possible. Effectiveness is gained by an increase of appropriate reliance on the operator and the aid. We studied whether it is possible to contribute to this objective by, apart from the operator, letting the aid as well calibrate trust in order to make reliance decisions. In addition, the aid's calibration of trust in reliance decision making capabilities of both the operator and itself is also expected to contribute, through reliance decision making on a metalevel, which we call metareliance decision making. In this paper we present a formalization of these two approaches: a reliance (RDMM) and metareliance decision making model (MetaRDMM), respectively. A combination of laboratory and simulation experiments shows significant improvements compared to reliance decision making solely done by operators.

It is often assumed that two heads are better than one, but reliance on decision aids is often inappropriate. Decisions to rely on an aid are thought to be based on a comparison between the perceived reliability of own performance and that of the decision aid. Unfortunately, perceived reliabilities are unlikely to be perfectly calibrated. This may result in inappropriate decisions to rely on advice. In a laboratory experiment with 40 participants, we studied whether calibration improves after practice, whether calibration of own reliability differs from calibration of the aid's reliability and whether unreliability of the aid is attributed differently. Under-trust in own reliability disappears after practice but under-trust in the aid's reliability persists. Unreliability of the decision aid is less likely to be attributed to temporary, external and uncontrollable factors. This asymmetry in attribution and calibration may explain under-reliance on decision aids.

Contemporary operations are characterised by more local conflicts, more dynamic circumstances and a less predictable enemy. Besides that, commanders must be able to be present at various locations, because of the involvement of different parties. This requires a more agile command post, supporting distributed command and control. A command post with reachback facilities, in which the commander can co-operate with his staff while being on the move, could meet these demands. However, such a concept has major consequences for aspects of team work, leadership, information support and organisation. This paper describes the results of an experiment with a reachback concept within the Royal Netherlands Army (RNLA) 43 Brigade staff where the staff was equipped with collaborative tools in a changed organisational context. The results show that the reachback concept is promising, enabling the commander to operate at a distance while maintaining accurate situational awareness. However, improvements, especially in the communication with other staff members, have to be made to be successful during operations. The next step will be to enhance the concept and focus on reachback concepts enabling staff members to co-operate from outside the operational area.

An important issue in research on human-machine cooperation concerns how tasks should be dynamically allocated within a human-machine team in order to improve team performance. The ability to support humans in task allocation decision making requires a thorough understanding of its underlying cognitive processes, and that of relative trust more specifically. This paper presents a computational agent-based model of these cognitive processes and proposes an experiment design that can be used to validate theoretical aspects of this model.

Good memorability of licence plates is important in those cases where licence plates are viewed for a brief period of time and the information is essential for police investigations. The purpose of the current study was to design a new Dutch licence plate that could be remembered well. A memory experiment was conducted, in which 16 different character arrangements were presented for both 450 and 550 ms to 48 participants with ages varying between 20 and 57 years. Participants had to rehearse the stimuli for 6 s, after which they had to be written down. Based on literature on short-term memory for serial order, character arrangements differed on three dimensions: 1) number of alternations between letters and digits; 2) letter to digit ratio; and 3) equality of group size. Results showed that number of alternations between characters of different categories affected memory performance the most. Letter to digit ratio and equality of group size affected memory performance to a lesser, but still significant, extent. A significant interaction between the latter two factors indicates that equal groups only lead to fewer memory errors when more than three letters are used. With three or fewer letters, group size is not a significant factor any more. Based on these results, a new licence plate for Dutch vehicles was recommended, which was subsequently adopted.

Future platforms are envisioned in which human-machine teams are able to share and trade tasks as demands in situations change. It seems that human-machine coordination has not received the attention it deserves by past and present approaches to task allocation. In this paper a simple way to make coordination requirements explicit is proposed and for dynamic task allocation a dual-route approach is suggested. Advantages of adaptable automation, in which the human adjusts the way tasks are divided and shared, are complemented with those of adaptive automation, in which the machine allocates tasks. To be able to support design for dynamic task allocation, a theory about task allocation decision making by means of modeling of trust is proposed. It is suggested that dynamic task allocation is improved when information about situational abilities of agents is provided and the cost of observing and re-directing agents is reduced.

Building up situation understanding is one of the most difficult tasks in the beginning stages of largescale accidents. As ambiguous information about the events becomes available, decision-makers are often tempted to quickly develop a particular story to explain the observed events. As the accident evolves, decision-makers can fail to revise their initial assessments despite contradicting information. Our approach is to reduce fixation errors and confirmation bias by providing critical thinking support. In a laboratory experiment with 60 participants, we compared the effect on decision making of a critical thinking tool, which requires the explication of evidence-conclusion relations in situation assessment, with two control conditions. Participants acted as crisis managers determining the likely cause of accidents. The results show a positive impact of the tool on both the decision-making process and decision making effectiveness. Participants did, however, take more time to arrive at a conclusion using the tool.