This study sought to understand the factors influencing voters to verify their paper ballots produced by an electronic Ballot Marking Device (BMD). To tackle this question, 60 undergraduate students from Rice University participated in mock elections where their votes on the electronic BMD displayed their selections accurately, but then a subset of their votes on the printed ballots were altered. After the system printed the paper ballot, two user interface (UI) interventions aimed at improving ballot verification rates were displayed in the following sequence: (1) a digital-based prompt instructing voters to “Carefully check your printed ballot. Take your time. Make sure everything is correct” and asked the question, “Are your printed selections correct?” requiring an on-screen response of “Yes” or “No,” and (2) a paper-based prompt requiring voters to handscribe their signature, a checkmark, or a sentence on the paper ballot. The results revealed that the paper-based intervention did not influence ballot verification performance, as all participants who detected at least one anomaly destroyed their compromised ballots after encountering the digital-based intervention. The overall detection rate on the digital-based intervention was high, with 88% of voters detecting changes in their ballots, suggesting that this ballot verification intervention was an effective countermeasure to encourage voters to check their paper ballots. The findings from this study can inform voting system design guidelines that motivate voters to independently audit their own ballot for anomalies.

Absentee voting presents a unique challenge for U.S. uniformed service members, as they often struggle to request and return absentee ballots while deployed, sometimes stationed far from their registered voting area. This research evaluates the usability of a proposed absentee voting system for military voters, which allows instant ballot requests and enables voters to verify their own votes, focusing on whether the user interface supports effective use among military personnel. Our evaluation revealed that military voters frequently relied on external assistance to navigate the electronic voting system, revealing opportunities for its improvement and design recommendations to facilitate absentee voting for U.S. military personnel.

Findings from previous research that assessed the usability of single-page and multipage digital interfaces in purely digital interactions indicated that the single-page format is more efficient than its multipage counterpart. This research expands on previous work by applying the findings from these digital-only interactions to a paper-digital interaction. Specifically, this study assessed the usability of single-page and multipage instructional interfaces to guide voters through the paper-based ballot mailing process embedded in the prototype of an electronic voting system designed for overseas military voters. A detailed classification of errors and requests for assistance revealed that the multipage format had fewer occurrences of both than the single-page format. Statistical analyses revealed no statistically significant differences between the single-page and multipage interfaces in efficiency, contrary to previous research, as well as no differences in effectiveness, satisfaction, and workload. To conclude, we provide arguments in favor of utilizing the multipage format for the digital display of the ballot mailing instructions on electronic voting systems moving forward. These findings can reveal best practices for the design of digital instructions in emerging paper-digital systems.

Ranked choice voting (RCV) is a method of voting where individuals rank their choices for each race in an election, rather than selecting a single candidate. For paper ballot implementations, there are three basic formats: bubble grid, column, and handwritten. This study aimed to understand which format produces the best outcomes with the fewest errors. Using a between-subjects design, we measured voting time, errors, success rates, and overall subjective usability using the System Usability Scale (SUS). Results showed that the handwritten format took significantly longer to complete. However, the handwritten ballot errors were recoverable and generally did not invalidate the ballots. The column and bubble grid ballots had significant errors, with fewer errors on the bubble grid format. There was no significant difference in SUS scores, nor any contribution of demographics across ballot types. Ballot format directly dictates the time taken to vote, usability, and overall voter success.

Endovascular surgery involves minimally invasive sur- gical techniques that can result in significantly shorter operation times and hospital stays, lower complication rates, less blood loss, and lower rates of postoperative mechanical ventilation and atrial fibrillation than the equivalent open procedures [1], [2]. Repeated practice is central to skill acquisition, and minimally invasive procedures like endovascular surgery may require more or specialized practice compared to traditional surgery. For example, despite known benefits of endovascular aortic valve replacement compared to traditional surgical methods, Smith et al. attributed observations of higher rates of stroke, transient ischemic attacks, and major vascular complications to a protracted learning curve for the endovascular approach [3]. Virtual reality endovascular surgical simulators can be loaded with a patient’s pre-operative CT scan, enabling rehearsal of difficult cases before operating. Simulators are also accessible to trainees, giving opportunities for additional practice in navigating to hard-to-reach vas- cular structures, or exposure to rare procedures. Still, surgical simulators lack the provision of real-time and objective performance feedback. Instead, feedback is only available after the completion of a surgical task, and often does not provide the trainee with insight into how they should change their task performance strategies to achieve performance goals. Objective measures of skill derived from endovascular guidewire movement kinematics that characterize tool tip movement smoothness have been shown to correlate with expertise [4], [5]. Such metrics have not yet been used during training as real-time performance feedback, despite evidence that providing feedback can improve training outcomes [6]. Our approach to providing real-time performance feed- back during surgical skill training is intended to address this gap. We propose to use estimates of spectral arc length (SPARC), idle time, and average velocity to quantify task performance, then encode these measures as vibrotactile cues displayed to trainees in a wearable haptic device (see Fig. 1).

There is still debate on whether voters can detect malicious changes in their printed ballot after making their selections on a Ballot Marking Device (BMD). In this study, we altered votes on a voter's ballot after they had made their selections on a BMD. We then required them to examine their ballots for any changes from the slate they used to vote. Overall accuracy was exceptionally high. Participants saw 1440 total contests, and of those 1440, there were a total of 4 errors, so total accuracy was 99.8 perform with near-perfect accuracy regardless of ballot length, ballot type, number of altered races, and location of altered races. Detection performance was extremely robust. We conclude that with proper direction and resources, voters can be near-perfect detectors of ballot changes on printed paper ballots after voting with a BMD. This finding has significant implications for the voting community as BMD use continues to grow. Research should now focus on identifying administrative and behavioral methods that will prompt and encourage voters to check their BMD-generated ballots before they drop them in the ballot box.

BACKGROUND:From the project's inception, STAR-Vote was intended to be one of the first usable, end-to-end (e2e) voting systems with sophisticated security. To realize STAR-Vote, computer security experts, statistical auditors, human factors (HF)/human-computer interaction (HCI) researchers, and election officials collaborated throughout the project and relied upon a user-centered, iterative design and development process, which included human factors research and usability testing, to make certain the system would be both usable and secure.OBJECTIVE:While best practices in HF/HCI methods for design were used and all apparent usability problems were identified and fixed, summative system usability assessments were conducted toward the end of the user-centered design process to determine whether STAR-Vote is in fact easy to use.METHOD AND RESULTS:After collecting efficiency, effectiveness, and satisfaction measurements per ISO 9241-11's system usability criteria, an analysis of the data revealed that there is evidence for STAR-Vote being the most usable, cryptographically secure voting system to date when compared with the previously tested e2e systems: Helios, Prêt à Voter, and Scantegrity.CONCLUSION AND APPLICATION:STAR-Vote being one of the first e2e voting systems that is both highly usable and secure is a significant accomplishment, because tamper-resistant voting systems can be used in U.S. elections to ensure the integrity of the electoral process, while still ensuring that voter intent is accurately reflected in the cast ballots. Moreover, this research empirically shows that a complex, secure system can still be usable-meaning that implemented security is not an excuse for poor usability.

The question of whether or not voters actually verify ballots produced by ballot marking devices (BMDs) is presently the subject of some controversy. Recent studies (e.g., Bernhard et al. 2020) suggest the verification rate is low. It is unclear if this is because voters cannot do this accurately or whether it is because voters simply choose not to attempt verification in the first place. In order to answer this question, we conducted an experiment in which 108 participants participated in a mock election where the BMD displayed the voters' true choices, but then changed a subset of those choices on the printed ballot. The design of the printed ballot, its length, the number and location of changes that were made to the ballot, and the instructions provided to the voters were manipulated as part of the experiment. Results indicated that of those voters who chose to examine the printed ballot, 76% detected anomalies, indicating that voters can reliably detect errors on their ballot if they simply review it. This suggests that administrative remedies, rather than attempts to alter fundamental human perceptual capabilities, could be employed to encourage voters to check their ballots, which could prove as an effective countermeasure.

Endovascular navigation proficiency requires a significant amount of manual dexterity from surgeons. Objective performance measures derived from endovascular tool tip kinematics have been shown to correlate with expertise; however, such metrics have not yet been used during training as a basis for real-time performance feedback. This paper evaluates a set of velocity-based performance measures derived from guidewire motion to determine their suitability for online performance evaluation and feedback. We evaluated the endovascular navigation skill of 75 participants using three metrics (spectral arc length, average velocity, and idle time) as they steered tools to anatomical targets using a virtual reality simulator. First, we examined the effect of navigation task and experience level on performance and found that novice performance was significantly different from intermediate and expert performance. Then we computed correlations between measures calculated online and spectral arc length, our “gold standard” metric, calculated offline (at the end of the trial, using data from the entire trial). Our results suggest that average velocity and idle time calculated online are strongly and consistently correlated with spectral arc length computed offline, which was not the case when comparing spectral arc length computed online and offline. Average velocity and idle time, both time-domain based performance measures, are therefore more suitable measures than spectral arc length, a frequency-domain based metric, to use as the basis of online performance feedback. Future work is needed to determine how to best provide real-time performance feedback to endovascular surgery trainees based on these metrics.

Carotid artery stenting (CAS) is a minimally invasive endovascular procedure used to treat carotid artery disease and is an alternative treatment option for carotid artery stenosis. Robotic assistance is becoming increasingly widespread in these procedures and can provide potential benefits over manual intervention, including decreasing peri- and post-operative risks associated with CAS. However, the benefits of robotic assistance in CAS procedures have not been quantitatively verified at the level of surgical tool motions. In this work, we compare manual and robot-assisted navigation in CAS procedures using performance metrics that reliably indicate surgical navigation proficiency. After extracting guidewire tip motion profiles from recorded procedure videos, we computed spectral arc length (SPARC), a frequency-domain metric of movement smoothness, average guidewire velocity, and amount of idle tool motion (idle time) for a set of CAS procedures performed on a commercial endovascular surgical simulator. We analyzed the metrics for two procedural steps that influence post-operative outcomes. Our results indicate that during advancement of the sheath to the distal common carotid artery, there are significant differences in SPARC (F(1, 223) = 6.12, p = .021) and idle time (F(1, 22.6) = 6.26, p = .02) between manual and robot-assisted navigation, as well as a general trend of lower SPARC, lower average velocity, and higher idle time values associated with robot-assisted navigation for both procedural steps. Our findings indicate that significant differences exist between manual and robot-assisted CAS procedures. These are quantitatively detectable at the granular-level of physical tool motion, improving the ability to evaluate robotic assistance as it grows in clinical use.

Objective assessment of surgical skill is gaining traction in a number of specialty fields. In robot-assisted surgery in particular, the availability of data from the operating console and patient-side robot offers the potential to derive objective metrics of performance based on tool movement kinematics. While these techniques are becoming established in the laparoscopic domain, current assessment techniques for robotic endovascular surgery are based primarily on observation, checklists, and grading scales. This work presents an objective and quantitative means of measuring technical competence based on analysis of the kinematics of endovascular tool tip motions controlled with a robotic interface. We designed an experiment that recorded catheter tip movement from 21 subjects performing fundamental endovascular robotic navigation tasks on a physical model. Motion-based measures of smoothness (spectral arc length and number of submovements) were computed and tested for correlation with subjective scores from a global rating scale assessment tool that has been validated for use when performing manual catheterization. Results show that the smoothness metrics that produced significant correlations with the global rating scale for manual catheterization show similar correlations for robotic catheterization. This finding is notable, since with the robotic interface, tool tip motion is commanded discretely via a control button interface, while in manual procedures the tools are controlled through continuous movements of the surgeon's hands. Logistic regression analysis using a single motion metric was capable of classifying subjects by expertise with better than 90% accuracy. These objective and quantitative metrics that capture movement quality could be incorporated into future training protocols to provide detailed feedback on trainee performance.

Abstract This chapter provides an introduction to and overview of both foundational and contemporary research using computational modeling to aid in the scientific understanding of human expertise. The authors note the distinction between computational models constructed within some molar or unified cognitive architecture and models that are more domain or task specific in their psychological assumptions, and present numerous examples of each type. The authors also provide their assessment of this body of research, one that highlights the need for extensive analysis, and even expert-level knowledge of both tasks and the environments in which expert behavior is manifest as a key requirement for successfully modeling high levels of skill or expert performance. Finally, the authors provide their thoughts about promising future directions for research using computational modeling, together with other emerging methodological techniques such as neuroimaging, to provide a comprehensive approach to advance the scientific understanding of human expertise.

Due to the proliferation of online services such as social networking, online banking, and cloud computing, more personal data are potentially exposed than ever before. Efforts such as two factor authentication (2FA) aim to make these services more secure; however, existing research efforts suggest this may come at the expense of usability. We conducted a usability evaluation of Google's 2FA setup process that confirms this concern, and extends previous efforts by identifying several problem areas and specific usability issues that affect human performance in 2FA setup processes. Future research should include more diverse populations but also continue efforts in improving the usability of 2FA setup processes. This will hopefully lead to increased adoption of 2FA systems.

Surgery is a challenging domain for motor skill acquisition, and compounding this difficulty is the often delayed and qualitative nature of feedback that is provided to trainees. In this paper, we explore the effectiveness of providing real-time feedback of movement smoothness, a characteristic associated with skilled and coordinated movement, via a vibrotactile cue. Subjects performed a mirror-tracing task that requires coordination and dexterity similar in nature to that required in endovascular surgery. Movement smoothness, measured by spectral arc length, a frequency-domain measure of movement smoothness, was encoded in a vibrotactile cue. Performance of the mirror tracing task with smoothness-based feedback was compared to position-based feedback (where the subject was alerted when they moved outside the path boundary) and to a no-feedback control condition. Although results of this pilot study failed to indicate a statistically significant effect of smoothness-based feedback on performance, subjects receiving smoothness-based feedback altered their task completion strategies to improve speed and accuracy, while those receiving position-based feedback or no feedback only improved in terms of increased accuracy. In tasks such as surgery where both speed and accuracy are vital to positive patient outcomes, the provision of smoothness-based feedback to the surgeon has the potential to positively influence performance.

About half of Americans have limited confidence that their vote will be properly counted. These fears have focused attention on voting system reliability, security, and usability. Over the last decade, substantial research on voting systems has demonstrated that many systems are less usable and secure than they should be. Producing truly reliable voting systems demands more than just following the federal guidelines enacted in 2005 (which, although well intentioned, have failed to substantially improve current systems) or simply updating voting systems to electronic voting computers using monies allocated by the 2002 Help America Vote Act (HAVA). In fact, HAVA has inadvertently led to the purchase of systems that may have actually increased the vote error rate. Key reforms needed to deliver reliable voting systems include substantial testing for usability, especially regarding the accurate capture of voter intent and the reduction of voter error rates, and measures to ensure the integrity of elections, such as election officials' ability to secure ballots.

Surgery is a challenging domain for motor skill acquisition. A critical contributing factor in this difficulty is that feedback is often delayed from performance and qualitative in nature. Collection of highdensity motion information may offer a solution. Metrics derived from this motion capture, in particular indices of movement smoothness, have been shown to correlate with task outcomes in multiple domains, including endovascular surgery. The open question is whether providing feedback based on these metrics can be used to accelerate learning. In pursuit of that goal, we examined the relationship between a motion metric that is computationally simple to compute—spectral arc length—and performance on a simple but challenging motor task, mirror tracing. We were able to replicate previous results showing that movement smoothness measures are linked to overall performance, and now have performance thresholds to use in subsequent work on using these metrics for training.

Ensuring the integrity of elections is one of the most important elements in maintaining democracy. While it is commonly believed that threats to election integrity are primarily due to security issues, the reality is that voting systems that are not designed to support human perceptual and cognitive limitations also pose a serious and immediate threat. This mismatch between system design and human capabilities can cause tremendous difficulty for voters who are trying to cast a ballot, and has almost certainly altered the outcome of elections in the United States. This article describes the psychological issues that can impact the ability of a voter to cast a vote as intended.

Current performance assessment techniques in endovascular surgery are subjective or limited to grading scales based solely on an expert's observation of a novice's task execution. Since most endovascular procedures involve performing fine motor control tasks that require complex dexterous movements, this paper evaluates objective and quantitative metrics of performance that capture movement quality through the computation of tool tip movement smoothness. An experiment was designed that involved recording the catheter tip movement from 20 subjects performing four fundamental endovascular tasks in each of three sessions using manual catheterization on a physical model and in a simulation environment. Several motion-based performance measures that have been shown to reliably assess skill in other domains were computed and tested for correlation with subjective data that were simultaneously obtained from the global rating scale assessment tool. Metrics that captured movement smoothness produced statistically significant correlations with the observation-based assessment metrics and were able to differentiate skill among participants. In particular, submovement analysis led to metrics that captured statistically significant differences across ability group, session, experimental platform, and task. Objective and quantitative metrics that capture movement smoothness could be incorporated into future training protocols to provide detailed feedback on trainee performance.