The criterion validity of the NASA Task Load indeX (TLX) and a previously developed subjective Task Complexity Index (TCI) were studied. A measure of subjective task complexity was hypothesized to complement the NASA TLX subjective workload measure on prediction of performance on complex problem-solving tasks. Twelve licensed nuclear power plant operators participated in a study with 16 simulated realistic scenarios. Although the study indicated some validity for both the TLX and the TCI, the results indicated stronger criterion validity for the subjective task complexity measure than for the TLX. Factor analysis of the task complexity ratings and TLX ratings together indicated that the 2 measures represent different constructs. Implications of the research include the recommendation of assessing subjective task complexity when studying human-machine configurations dealing with complex problem-solving tasks.
(2001). The Effects of Visual Information on Users' Mental Models: An Evaluation of Pathfinder Analysis as a Measure of Icon Usability. International Journal of Cognitive Ergonomics: Vol. 5, No. 1, pp. 59-84.
This article sets out to test the hypothetical COtext and COntrol Model (COCOM) developed by Hollnagel (1993). Essentially, Hollnagel develops the argument that team behavior should be analyzed at a macro, rather than micro, level. He proposes 4 principal models of team activity: strategic, tactical, opportunistic, and scrambled. These modes of team behavior vary in terms of the degree of forward planning (highest in the strategic mode) and reactivity to the environment (highest in the scrambled mode). He further hypothesizes a linear progression through the modes from strategic to tactical to opportunistic to scrambled, depending on context, and vice versa. To test the COCOM model, we placed teams of people in a simulated energy distribution system. Our results confirm Hollnagel's hypothesized model in 2 main ways. First, we show that the team behavior could be categorized reliably into the 4 control modes and this provided a useful way of distinguishing between experimental conditions. Second, the progression between control modes conformed to the linear progression as predicted. This research provided the first independent test of the COCOM model and lends empirical support to the hypotheses.
Cognitive work analysis (CWA) techniques are the primary methods available for designers to obtain the knowledge required to create interfaces to complex systems involving cognitive work. There are a wide and growing variety of analysis methods available with a variety of claims for their relative strengths and weaknesses, but it is extremely rare for anyone actually to apply different analytic techniques to the same analysis problem. The work reported here begins to address this gap by directly comparing the information requirements produced by what are probably the 2 most commonly used analysis techniques-Rasmussen's (1985) Abstraction-Decomposition Space (ADS) or "Abstraction Hierarchy" and Shepherd's (1989) Hierarchical Task Analysis (HTA) technique. These 2 approaches were selected because each is well known in the literature, yet they rarely have been directly compared on a common problem. Our comparison shows that the techniques produce different yet complementary information about the interaction needs that human users of a system will have. Both approaches have strengths and weaknesses, but ultimately they reflect different perspectives on (and different avenues to) the knowledge needed for good system and interface design.
Analytical measurement, the process by which the chemical composition of a substance is determined, is carried out in chemical laboratories throughout the world. It has been recognized for some time that results obtained from different laboratories are not as consistent as would be expected, but until recently little attention has been paid to the human factors and human error aspects of the task. However, analytical measurement is a multistage task that places high manual and cognitive demands on the operator, which are likely to be the source of considerable human error, as it is likely that the reliability of the task could be improved by the application of appropriate cognitive ergonomics. Task demands range from accuracy in manual, skill-based activities to complex, knowledge-based activities. This paper presents a hierarchical task analysis as a methodological framework for looking at the process of analytical measurement, followed by an account of the potential errors that can occur. These errors are then placed within a broader cognitive and human error framework. A series of recommendations as to how to reduce error at each of the stages of analytical measurement is described.
Model-based approaches to cognitive task analysis rely on abstracting from the problem domain, rather than a description of task performance. Various classes of work analysis models can be distinguished: task models, work domain structure models, and work domain goal models. In this article, we discuss 3 specific model-based approaches: the Abstraction Hierarchy (AH), Multilevel Flow Modeling (MFM), and the Decision Ladder (DL). The AH creates a model of the work domain structure, MFM of work domain goals, and DL of actor tasks. The 3 approaches are compared in the context of a common example-that of a power plant boiler-to show clearly the differences among the 3 classes of techniques.
The design of team training programs and other team interventions could benefit from an understanding of team cognition. The research presented in this article evaluates methods for eliciting and assessing team knowledge during acquisition of a complex task. Knowledge measures are evaluated in terms of their ability to predict team performance and also in terms of how they reflect skill acquisition. The study was conducted in the context of a synthetic 3-person team task that involved operating an uninhabited air vehicle (UAV). Eleven teams of 3 Air Force ROTC cadets participated in 3 experimental sessions lasting from 3 to 6 hr. During these sessions, teams were trained on the task and were observed as they performed ten 40-min missions. During the missions, team performance and team process behaviors were measured, as well as the fleeting team knowledge associated with situation awareness. In addition, long-term team knowledge regarding both taskwork and teamwork were measured offline in four sessions. ...
This article focuses on the variations of the human information processing (HIP) performance, error occurrence, and HIP time that occurred under the constraints of working memory. The characteristics of the variations were analyzed by carrying out 2 experiments. It was revealed in Experiment 1 that the variation of the error occurrence existed and the time series of the error occurrence had the characteristic fluctuation of 1/f in all subjects even though the experimental conditions and the protocol for each subject were fixed. It was also revealed in Experiment 2 that the variation of the HIP time in the working memory existed and the time series of the HIP time in the working memory had the characteristic fluctuation of 1/f0 in all subjects even though the experimental conditions and the protocol for each subject were fixed. It was difficult to explain these variations and the characteristic fluctuations of the performance by the experimental conditions as well as by applying the precedent concept on the working memory resource capacity of the conventional working memory models. In the new theoretical hypothesis, called FLuctuation Of Working Memory (FLOWM), it was proposed that the working memory resource capacity had 1/f0 fluctuation. It was revealed in Experiments 1 and 2 that the variations of the error occurrence and the HIP time and 1/f0 fluctuation of the HIP time could be explained without any contradiction by applying FLOWM. Through the experiments and by conducting the simulation it was also revealed that 1/f fluctuation of the error occurrence could be explained by applying FLOWM.
A proposed feedback model of driving implicates vehicle feedback as an important variable affecting driver cognition. This naturalistic study employs an on-road paradigm to begin investigating the effects of vehicle feedback on drivers. Whilst performing a specially designed concurrent verbal protocol, 12 drivers drove their own cars around a predetermined 14 mile test route. This was designed to elicit the information that drivers were gaining from the environment and the vehicle, and how this information was being put to use. Prerun questionnaire measures featured driving style and locus of control, whereas postrun measures included self-assessment of mental workload and situational awareness. The vehicles were divided into 2 groups contingent upon their mechanical and engineering specifications into high and low vehicle feedback status, anecdotally, driver's cars versus average cars. A content analysis showed key differences in driver cognition contingent upon the vehicles feedback status. High-feedback vehicles are related to better situational awareness for drivers, coupled with lower workload. Drivers of low-feedback cars used their vehicle's instruments more often (despite having less of them), and it appears overall from self-assessment of situational awareness that drivers are not particularly aware of their own levels of Situational Awareness (SA), or indeed, any shortfall in it. These findings all correspond to feedback model predictions, and suggest a fruitful avenue for further simulator-based research.
A proper understanding of human performance characteristics is a prerequisite for designers of complex systems. Although human factors texts provide some insights into basic performance issues, the emergence of highly automated computing systems have fundamentally altered the way humans work. The purpose of this article is to present a research approach to quantify and analyze human performance within a complex, time-critical system. The approach is centered on a measurement construct, called a time window, which enables a functional relation between constraints on operator activities and time availability. A blackboard model is developed as the mechanism to generate, maintain, and complete time windows. Moreover, an object-oriented methodology is described that implements the blackboard model within a realistic task context. To demonstrate the utility of time windows, an existing implementation in a real-time human-in-the-loop simulation is also described. Using time window outcomes, some cursory analyses are completed to exhibit the potential of the construct.
Experiments were conducted to examine whether information refresh rate and moving objects have an effect on saccade latency times. The ratio of the number of moving objects to the number of static objects in a display was used to define dynamic object ratio (DOR), an independent variable in the study. The results of the experiment reveal the following: (a) dynamicity and information refresh rates have effects on saccade latency; (b) it was observed that when DOR increases, participants took longer time to make their initial saccades; and (c) as the DOR increases beyond 2 dynamic objects, latency times seem to be the same irrespective of the information refresh rate. The results suggest that the movement of objects and the information presentation refresh rates in a display environment are important variables to consider in visual cognition studies. We hope to expand the experiment to include effects on visuospatial attention, information recall, and information contents to be analyzed by human participants.
In this case study, operators' cardiovascular activity was observed during the operation of modern forestry machinery. In addition to heart rate, power spectral analysis was used to indicate how the variance distributes as a function of frequency. As has been pointed out in previous studies, changes in relative power, especially at medium frequencies (MF), have been found to indicate the amount of mental workload. The results illustrated that operating a modern single-grip harvester causes relatively high mental workload. Moreover, significant differences in cardiovascular activity during different work tasks were observed.
The use of alarms in process control is a complex affair, because alarms can have many different functions over and above that of alerting the operator to a new event. Alarms are potentially a rich source of information that can be used in all phases of responding to a disturbance and should therefore be considered as an aspect of the general process information. From the point of view of cognitive systems engineering, alarms provide information that enables the operators to maintain control of the process and to be proactive as well as reactive. An important issue is the time and effort needed to evaluate events and new information (time to evaluate: TE) and the time and effort needed to select an appropriate response or line of action (time to select: TS). The former (TE) can be supported by more effective presentation of information, specifically a more comprehensive presentation of alarms. The latter (TS) can be supported by various kinds of computerized support, ranging from planning systems to adaptive and computerized procedures.
A usability engineering (UE) method has been developed for the European Space Agency to assist the development teams of payload user interfaces in the International Space Station (ISS). It attempts to synchronize the activities of 2 traditionally separated groups-the operations group involved in procedure development and the software group involved in display development-to realize adequate online procedural operation support (instead of the traditional paper-based support). An example application of the UE method resulted in a prototype user interface with the online procedures integrated into the operation display, and with consistent navigation support in procedure and operation interface components. A controlled experiment showed that the integration and the navigation support enhance the efficiency of payload operations substantially as a first validation of this example interface design.
Because of substantial advances of automatic technology in human-machine systems, the operator has changed to a "decision maker" in a control room. Modeling operator's decision behavior has attracted a major interest in research fields such as human factors and reliability engineering. However, traditional reasoning models proposed by psychologists are "static" so that they are deficient in accounting for the interactive nature of modern human-machine systems. Concerning this issue, a conceptual framework of decision making is suggested to distinguish between dynamic and static decision making. Hence, dynamic and static decision processes are defined considering the decision stages in a decision-making task. Reasoning is regarded as a dominant mental activity in each decision stage and the inadequacy of static reasoning models in handling dynamic decision making is discussed. Considering human cognitive biases and the requirements of engineering applications, an event-driven approach, called the Dynamic Boolean Expression (DBE), is developed to model dynamic reasoning. Based on the deterministic nature of human decision making, a quantitative "causality" is derived to describe the degree to which the decision outcome is driven by its driving events. An experiment based on the popular tic-tac-toe game was performed to verify the validity of the DBE. Finally, an engineering application of the DBE and its further researches on dynamic decision making are suggested in the concluding section of this article.
When viewing a multimedia document on the web, 2 parameters comprise the user perception of the document: user satisfaction/enjoyment of the document and user assimilation/understanding of the information contained in the document. Existing communication architectures and protocols do not cater to the user's experience with multimedia. In this article we present a communication architecture based on microprotocols that adapt so as to maintain user perception even as the quality of the multimedia presentation degrades due to network congestion. We compare our protocols with 2 legacy protocols by testing real users for perception as the quality of the multimedia presentation deteriorates. Our results show that despite the overheads for adaptation, our protocol performed the best from a user's perspective. Our research shows that not only is it desirable but feasible to tailor communication architectures and protocols to the benefit of the user.
Usability is paramount for the success of Web sites. This study aims to contribute towards Web design guidelines for usability through an empirical investigation into two Web page design parameters. Using a 2 x 4 mixed measures experimental design, we studied the effects of frame layout and background contrast on visual search performance. One hundred and eighty-nine undergraduates carried out a visual search task using mock Web pages. Analysis showed an effect of layout both on accuracy and speed measures, with frames located at the top or left of the screen leading to better performance. No main effect of contrast was found, although there was an interaction between layout and contrast in reaction time for hits. Preference for frame layout was dependent on background contrast. The results are discussed in terms of visual search processes and design recommendations are given. (C) 2001 Elsevier Science B.V. All rights reserved.
