As organizations continue to invest in phishing awareness training programs, many chief information security officers (CISOs) are concerned when their training exercise click rates are high or variable, as they must justify training budgets to organization officials who question the efficacy of awareness training when click rates are not declining. We argue that click rates should be expected to vary based on the difficulty of the phishing email for a target audience. Past research has shown that when the premise of a phishing email aligns with a user's work context, it is much more challenging for users to detect a phish. Given this, we propose a Phish Scale, so CISOs and phishing training implementers can easily rate the difficulty of their phishing exercises and help explain associated click rates. We base our scale on past research in phishing cues and user context, and apply the scale to previously published and new data from enterprise-based phishing exercises. The Phish Scale performed well with the current phishing dataset, but future work is needed to validate it with a larger variety of phishing emails. The Phish Scale shows great promise as a tool to help frame data sharing on phishing exercise click rates across sectors.

In public safety and homeland security, it is important to understand technology users' primary goals, the characteristics of the users, and the context in which they are operating. New and emerging technologies present opportunities and challenges for public safety standards. As part of the Public Safety Communications Research (PSCR) effort, the National Institute of Standards and Technology (NIST) usability team is currently researching the usability of communications technology for first responders. This is a large, multiphase research project, with the ultimate goal of developing a standardized usability testing and evaluation methodology for public safety communications technology. In this document, we discuss our application of a user-centered research methodology to investigate first responders' technology and communication needs. We present the details of the initial phase of the project, including our methodological approach and results from working with subject matter experts (SMEs) in fire, emergency medical services (EMS), and law enforcement fields. We conducted in-depth interviews with 133 SMEs to understand their perceptions and attitudes towards communications technologies and their information and data needs during incident response. We found that information and data needs can vary based on user role, tasks, and the context and scope of an incident. User needs and requirements have been organized into five categories of technology opportunities. Further analysis identified six user-centered design guidelines for technology development. Data resulting from this user-centered research can help inform usability testing methodology and standards for homeland security and public safety.

As phishing continues to evolve, what’s your organization doing to stay off the hook?

Extensive research has been performed to examine the effectiveness of phishing defenses, but much of this research was performed in laboratory settings. In contrast, this work presents 4.5 years of workplace-situated, embedded phishing email training exercise data, focusing on the last three phishing exercises with participant feedback. The sample was an operating unit consisting of approximately 70 staff members within a U.S. government research institution. A multiple methods assessment approach revealed that the individual’s work context is the lens through which email cues are interpreted. Not only do clickers and non-clickers attend to different cues, they interpret the same cues differently depending on the alignment of the user’s work context and the premise of the phishing email. Clickers were concerned over consequences arising from not clicking, such as failing to be responsive. In contrast, non-clickers were concerned with consequences from clicking, such as downloading malware. This finding firmly identifies the alignment of user context and the phishing attack premise as a significant explanatory factor in phishing susceptibility. We present additional findings that have actionable operational security implications. The long-term, embedded and ecologically valid conditions surrounding these phishing exercises provided the crucial elements necessary for these findings to surface and be confirmed. Keywords—decision-making, embedded phishing awareness training, user-centered approach, survey instrument, long-term assessment, operational data, trial deployment, network security, security defenses

The public safety community is transitioning from land mobile radios to a communications technology ecosystem including a variety of broadband data sharing platforms. Successful deployment and adoption of new communications technology relies on efficient and effective user interfaces based on understanding first responder needs, requirements, and contexts of use; human factors research is needed to examine these factors. As such, this paper presents initial qualitative research results via semi-structured interviews with 133 first responders across the U.S. While there are similarities across disciplines, results show there is no easy “one size fits all” communications technology solution. To facilitate trust in new communications technology, solutions must be dependable, easy to use for first responders, and meet their communication needs through the application of user-centered design principles. During this shift in public safety communications technology, the time is now to leverage existing human factors expertise to influence emerging technology for public safety.

Password policies – documents which regulate how users must create, manage, and change their passwords – can have complex and unforeseen consequences on organizational security. Since these policies regulate user behavior, users must be clear as to what is expected of them. Unfortunately, current policies are written in language that is often ambiguous. To tackle ambiguity, we previously developed a formal language for stating what behavior is and is not allowed regarding password management. Unfortunately, manual translation of the policy to this formal language is time consuming and error prone. This work focuses on providing an interface for policy users to generate accurate models of their interpretations of a password policy. This will aid password policy research, formalization, and ultimately more usable password policies. This paper describes the requirements, design, high-level application features, application validation, user testing, and includes a discussion of how this work is expected to progress.

We conducted a two-part study to understand the impact of authentication on employees' behaviour and productivity in a US governmental organisation. We asked 23 participants to keep a diary of all their authentication events within a 24-hour period, and subsequently interviewed them about their experience with authentication. We found that the authentication tasks employees have to perform not only carry significant workload, but that the way in which authentication disrupts primary tasks reduces productivity and creates frustration. Our participants reported a range of coping strategies, including use of tools and re-organising their work to avoid security. Avoidance meant they logged in less frequently, stopped using certain devices and services. They also reported not pursing innovative ideas because of "the battle with security" that would be required. Our case study paints a picture of chronic 'authentication fatigue' resulting from current policies and mechanisms, and the negative impact on staff productivity and morale. We propose that organisations need to urgently re-think how they authenticate users in a pervasive technology requirement, and advocate a paradigm shift from explicit to implicit authentication.

Users have developed various coping strategies for minimizing or avoiding the friction and burden associated with managing and using their portfolios of user IDs and passwords or personal identification numbers (PINs). Many try to use the same password (or different versions of the same password) across different systems. Others use memory aids or technological assistants such as password management software. We were interested in these coping strategies and the ,friction pointsS that prompt people to use them. More broadly, we wanted to address a pressing research need by gathering data for user-centered models of how people interact with security as part of their daily life, as empirical research in that area is currently lacking.

Password policies – documents which regulate how users must create and manage their passwords – can have complex and unforeseen consequences on organizational security. Since these policies attempt to govern user behavior, users must be clear as to what is expected of them for a policy to be effective. While a culprit of misinterpretation, policy ambiguity also prevents researchers from comparing and contrasting policy statements. To tackle ambiguity, we developed a formal language for stating what behavior is and is not allowed when creating, managing, and changing passwords. This formal language lends itself to policy analysis and visualization. A corpus of 41 password policies was translated into the formal language and analyzed. Having these clear, unambiguous policy statements enables us to explore password policies in much greater detail, discuss the relative merits of different statements, compare and contrast policies, and begin to examine the interplay between usability and security in password policies.

One of the most difficult challenges that military personnel face when operating in foreign countries is clear and successful communication with the local population. To address this issue, the Defense Advanced Research Projects Agency (DARPA) is funding academic institutions and industrial organizations through the Spoken Language Communication and Translation System for Tactical Use (TRANSTAC) program to develop practical machine translation systems. The goal of the TRANSTAC program is to demonstrate capabilities to rapidly develop and field free-form, two-way, speech-to-speech translation systems that enable speakers of different languages to communicate with one another in real-world tactical situations without an interpreter. Evaluations of these technologies are a significant part of the program and DARPA has asked the National Institute of Standards and Technology (NIST) to lead this effort. This article presents the experimental design of the TRANSTAC evaluations and the metrics, both quantitative and qualitative, that were used to comprehensively assess the systems' performance.

Under the United States Visitor and Immigrant Status Indicator Technology (US-VISIT) program, the Department of Homeland Security (DHS) collects biometric information from foreign nationals entering the country. Foreign nationals are required to – among other things – undergo a ten-fingerprint scanning process, which is conducted by a customs agent. This scan is only performed upon entry into the United States. DHS is investigating the feasibility of deploying an unassisted " self-service " fingerprint scanning solution as well for when foreign nationals exit the country. To that end, DHS requested assistance from NIST to examine the affordances of the fingerprint scanner when used with the US-VISIT instructional poster, both of which were in use at U.S. ports of entry when this study was conducted (Feb-Mar 2010), to learn how travelers interpret these features. For example, whether they help travelers use the scanner in such a way that the scanner can capture clear pictures of travelers' prints, and whether a person can present a usable fingerprint sample to the scanner without assistance. Findings from this study will inform future efforts to design a self-service fingerprint solution that can be effectively used by the vast majority of people regardless of age, nationality, or level of technical proficiency. In this study, 62 participants used the same instructional poster and fingerprint scanner employed by the US-VISIT program and underwent the same process employed in that program, except that they did not receive any guidance from the person operating the fingerprint scanner. We observed their behavior during the process, evaluated how well they completed the fingerprinting task, i.e., whether they got the scanner to take a clear picture of some or all of their fingerprints, and afterwards we asked them questions about their experience using the scanner. Our findings indicate that the US-VISIT poster and scanner, as implemented, will not constitute a workable self-service fingerprinting solution. The US-VISIT poster provided our study participants with a general idea of what to do, but left them unsure of specifics such as how long to hold their fingers on the platen or how to tell whether the scanner had managed to get a clear image of their fingerprints – a finding consistent with a previous study involving a similar instructional poster [11]. The affordances on the scanner itself, consisting of instructional icons and light emitting diodes (LEDs), are designed to provide this information; however, many participants either failed to …