OBJECTIVE:The public, regulators, and domain experts alike seek to understand the effect of deployed SAE level 4 automated driving system (ADS) technologies on safety. The recent expansion of ADS technology deployments is paving the way for early stage safety impact evaluations, whereby the observational data from both an ADS and a representative benchmark fleet are compared to quantify safety performance. METHODS:In January 2024; a working group of experts across academia, insurance, and industry came together in Washington, DC to discuss the current and future challenges in performing such evaluations. A subset of this working group then met, virtually, on multiple occasions to produce this paper. RESULTS:This paper presents the RAVE (Retrospective Automated Vehicle Evaluation) checklist, a set of fifteen recommendations for performing and evaluating retrospective ADS performance comparisons. The recommendations are centered around the concepts of (1) quality and validity, (2) transparency, and (3) interpretation. CONCLUSION:Over time, it is anticipated there will be a large and varied body of work evaluating the observed performance of these ADS fleets. Establishing and promoting good scientific practices benefits the work of stakeholders, many of whom may not be subject matter experts. This working group's intentions are to: i) strengthen individual research studies and ii) make the at-large community more informed on how to evaluate this collective body of work.

There is a need for a large-scale, real world, diverse, and context rich vehicle acceleration catalog that can be used to design, analyze, and compare various intelligent transportation systems. This paper fulfills three primary objectives. First, it provides such a catalog through the Surface Accelerations Reference, which is openly available as an interactive analytics tool as well as an open and downloadable dataset. The Surface Accelerations Reference statistically describes the driving profiles of about 3,500 individuals contributing 34 million miles of continuous driving data collected in the Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS). These profiles were created by summarizing billions of longitudinal and lateral acceleration epochs experienced by the participants. Second, this paper introduces a standardized methodology for creating such a catalog so that similar acceleration profiles can be produced for other human cohorts or automated driving systems. Finally, the data are used to analyze the effect of roadway speed category on the rates of lateral and longitudinal acceleration epochs at various thresholds. It is observed that, for the median driver, the rates of epochs are up to three orders of magnitude higher on low-speed roads as compared to high-speed roads. This catalog will facilitate intelligent vehicle system designers to compare and tune their systems for safer driving experiences. It will also allow agencies with similar data to create comparable catalogs facilitating safety and behavioral comparisons between populations.

The purpose of this study is to understand and quantify the simultaneous effects of roadway speed category, driver age, driver gender, vehicle class, and location on the rates of longitudinal and lateral acceleration epochs. The rate of usual as well as harsh acceleration epochs are used to extract insights on driving risk and driver comfort preferences. However, an analysis of acceleration rates at multiple thresholds incorporating various effects while using a large-scale and diverse dataset is missing. This analysis will fill this research gap. Data from the 2nd Strategic Highway Research Program Naturalistic Driving Study (SHRP2 NDS) was used for this analysis. The rate of occurrence of acceleration epochs was modeled using negative binomial distribution based generalized linear mixed effect models. Roadway speed category, driver age, driver gender, vehicle class, and location were used as the fixed effects and the driver identifier was used as the random effect. Incidence rate ratios were then calculated to compare subcategories of each fixed effect. Roadway speed category has the strongest effect on longitudinal and lateral accelerations of all magnitudes. Acceleration epoch rates consistently decrease as the roadway speed category increases. The difference in the rates depends on the threshold and is up to three orders of magnitude. Driver age is another significant factor with clear trends for longitudinal and lateral acceleration epochs. Younger and older drivers experience higher rates of longitudinal accelerations and decelerations. However, the rate of lateral accelerations consistently decreases with age. Vehicle class also has a significant effect on the rate of harsh accelerations with minivans consistently experiencing lower rates.

Takayuki Kondoh, Nissan Motor Co., Ltd. / Virginia Tech Transportation Institute, t_kondoh@mail.nissan.co.jp Shane McLaughlin, Virginia Tech Transportation Institute, smclaughlin@vtti.vt.edu Tomohiro Yamamura, Nissan Motor Co., Ltd., t-yamamura@mail.nissan.co.jp Nobuyuki Kuge, Nissan Motor Co., Ltd., n-kuge@mail.nissan.co.jp Miguel Perez, Virginia Tech Transportation Institute, mperez@vtti.vt.edu Takashi Sunda, Nissan Motor Co., Ltd., t-sunda@mail.nissan.co.jp

Introduction: Accurate motorcyclist mileage estimates are important because self-evaluation of riding experience is related to riding behavior, the relationship of self-reported to actual or future mileage is necessary in targeting training and considering survey responses, and motorcycle crash statistics require accurate travel data. Method: This study collected real-world data from motorcyclists over the course of two months to two years per rider. This paper explores motorcyclists' self-reported annual riding mileage (obtained via pre-study surveys) and the actual amount of riding during the study (based upon odometer readings and GPS data). Results: Of the 91 riders who had been riding for at least a year before the study, significantly more (73%) rode less the following year than reported for the previous year. The recorded annualized mileage averaged 89% of the reported mileage from the previous year. Analyses based on estimated average annual mileage were similar to those using the previous year estimation, and the pattern held regardless of age group, motorcycle type, or gender. The exception was novice or returning riders, who tended to either significantly underestimate or increase actual mileage as they began (or continued) to ride. Conclusions: Motorcyclists' estimation of riding experience expressed as mileage may not be indicative of current or future mileage. Practical applications: Reliance on self-reported mileage during training to categorize groups, for interpretation of studies, or to develop motorcycle travel data and safety statistics may be unrealistic. Certainly any use of self-reported mileage should incorporate the concept that mileage overestimation seems likely. Because questions about previous year and average annual mileage may elicit similar responses, motorcyclist surveys should be constructed to prompt the most thoughtful responses in terms of mileage estimations. In general, reported mileage should not be relied upon as an accurate predictor of future actual mileage. (C) 2017 National Safety Council and Elsevier Ltd. All rights reserved.

Crash warning systems have been deployed in the high-end vehicle market segment for some time and are trickling down to additional motor vehicle industry segments each year. The motorcycle segment, however, has no deployed crash warning system to date. With the active development of next generation crash warning systems based on connected vehicle technologies, this study explored possible interface designs for motorcycle crash warning systems and evaluated their rider acceptance and effectiveness in a connected vehicle context. Four prototype warning interface displays covering three warning mode alternatives (auditory, visual, and haptic) were designed and developed for motorcycles. They were tested on-road with three connected vehicle safety applications - intersection movement assist, forward collision warning, and lane departure warning - which were selected according to the most impactful crash types identified for motorcycles. Combined auditory and haptic displays showed considerable promise for implementation. Auditory display is easily implemented given the adoption rate of in-helmet auditory systems. Its weakness of presenting directional information in this study may be remedied by using simple speech or with the help of haptic design, which performed well at providing such information and was also found to be attractive to riders. The findings revealed both opportunities and challenges of visual displays for motorcycle crash warning systems. More importantly, differences among riders of three major motorcycle types (cruiser, sport, and touring) in terms of rider acceptance of a motorcycle crash warning system were revealed. Based on the results, recommendations were provided for an appropriate crash warning interface design for motorcycles and riders in a connected vehicle environment.

This project characterized the performance of Connected Vehicle Systems (CVS) on motorcycles based on two key components: global positioning and wireless communication systems. Considering that Global Positioning System (GPS) and 5.9 GHz Dedicated Short-Range Communications (DSRC) may be affected by motorcycle rider occlusion, antenna mounting configurations were investigated. In order to assess the performance of these systems, the Virginia Tech Transportation Institute’s (VTTI) Data Acquisition System (DAS) was utilized to record key GPS and DSRC variables from the vehicle’s CVS Vehicle Awareness Device (VAD). In this project, a total of four vehicles were used where one motorcycle had a forward mounted antenna, another motorcycle had a rear mounted antenna, and two automobiles had center-mounted antennas. These instrumented vehicles were then subject to several static and dynamic test scenarios on closed test track and public roadways to characterize performance against each other. Further, these test scenarios took into account motorcycle rider occlusion, relative ranges, and diverse topographical roadway environments. From the results, both rider occlusion and approach ranges were shown to have an impact on communications performance. In situations where the antenna on the motorcycle had direct line of sight with another vehicle’s antenna, a noticeable increase in performance can be seen in comparison to situations where the line of sight is occluded. Further, the forward-mounted antenna configuration provided a wider span of communication ranges in open-sky. In comparison, the rear-mounted antenna configuration experienced a narrower communication range. In terms of position performance, environments where objects occluded the sky, such as deep urban and mountain regions, relatively degraded performance when compared to open sky environments were observed.

Crash warning systems have been deployed in the high-end vehicle market segment for some time and are trickling down to additional motor vehicle industry segments each year. The motorcycle segment, however, has no deployed crash warning system to date. With the active development of next generation crash warning systems based on connected vehicle technologies, this study explored possible interface designs for motorcycle crash warning systems and evaluated their rider acceptance and effectiveness in a connected vehicle context. Four prototype warning interface displays covering three warning mode alternatives (auditory, visual, and haptic) were designed and developed for motorcycles. They were tested on-road with three connected vehicle safety applications - intersection movement assist, forward collision warning, and lane departure warning - which were selected according to the most impactful crash types identified for motorcycles. It showed that a combination of warning modalities was preferred to a single display by 87.2% of participants and combined auditory and haptic displays showed considerable promise for implementation. Auditory display is easily implemented given the adoption rate of in-helmet auditory systems. Its weakness of presenting directional information in this study may be remedied by using simple speech or with the help of haptic design, which performed well at providing such information and was also found to be attractive to riders. The findings revealed both opportunities and challenges of visual displays for motorcycle crash warning systems. More importantly, differences among riders of three major motorcycle types (cruiser, sport, and touring) in terms of riders’ acceptance of a crash warning interface were revealed. Based on the results, recommendations were provided for an appropriate crash warning interface design for motorcycles and riders in a connected vehicle environment.

This paper describes a 100-rider naturalistic motorcycle study in terms of participant demographics, riding preferences, and behaviors collected through five questionnaires (NEO-FFI-3, Dula Dangerous Driving Index, Frequency of Risky Behavior, Barkley Adult ADHD Rating Scale–IV, and the Motorcycle Safety Foundation Rider Survey). The analysis of the questionnaire data is divided into three bike Types (Touring, Cruiser, Sport). The results indicate that self-reported mileage traveled was highest among Touring bike riders, followed by Sport riders, then Cruiser riders. Differences in Personality factors between Motorcycle Types was generally not found, with the exception of lower levels of Neuroticism in riders of the Touring bikes compared to riders of Cruisers and Sport. The survey based measures indicate that the participant sample does include some measurable heterogeneity in neuroticism scores purely based on questionnaires.

On-board driver state monitoring is considered one of technologies that could reduce traffic accidents caused by human errors, particularly in cases of higher workload due to driver distraction. An approach to assessing driver workload using a real-time steering entropy (RSE) method has been proposed by the authors. RSE quantifies the nature of a driver's corrective steering with an index of relative entropy (RHp) from information theory. The higher the driver workload, the higher the RHp. In this paper, the RSE method was applied to naturalistic driving database gathered from 18 drivers in the United States under joint research between Virginia Tech Transportation Institute (VTTI) and Nissan Motor. RHp was calculated in off line simulation, and cases where the diver's state indicated higher RHp were reviewed in through using video. The result indicates that the RSE method detected over 80% of the cases within 300 sec from when the handheld call started. There may also be differences in the onset of RHp behavior between calls initiated by the driver (outgoing) and incoming calls.

This paper introduces the preliminary categorization of motorcycle crash data collected from 100 riders as they rode for a period between 2 months and 2 years. These riders resided in California, Florida, Virginia, and Arizona, and both video and motorcycle kinematic data were collected for every ride. The videos of incidents were reviewed, and the events were described with motorcycle-specific categories for event severity, event nature, incident type, precipitating event, rider reaction, and postmaneuver control. Within the data set of more than 38,000 trips, 22 incidents defined as crashes (involving 18 riders) were identified and categorized. The majority of the crashes (15) were single-vehicle, low-speed crashes. Of the remaining crashes, one was a single-vehicle (higher speed) crash, two were rear-end collisions, and four involved one vehicle turning into or across the path of another at an intersection. Rider response to the precipitating event for more than half of the crashes involved no visible front braking or lateral input. Most crashes occurred within 20 min of the beginning of a trip, during daylight, and in favorable weather conditions. There was no overriding hypothesis implying a relationship of crash occurrence to a specific demographic group in terms of location, age, gender, or motorcycle class. This research applied motorcycle incident categorization to 22 crashes. The categorization terminology and definitions are also applicable to near-crashes. This information is useful in providing an unbiased understanding of what occurs during such incidents and offering a basis for the structured cataloging of crashes, as well as future categorization of near-crashes and crash-relevant events.

This report details the methodology used to link the second Strategic Highway Research Program (SHRP 2) Naturalistic Driving Study (NDS) data to the SHRP 2 Roadway Information Database (RID), the final critical step in completing the SHRP 2 Safety database. The NDS data set contains extensively detailed data collected continually from more than 5.5 million trips taken by the instrumented vehicles of 3,147 volunteer drivers in six sites. The RID contains extensively detailed data on 25,000 centerline miles of roadways in these six sites, less detailed data on 200,000 centerline miles of roadways in the six states in which the sites were located, and supplemental data on topics such as crash histories, travel volumes, construction, and weather in the six states. The true power of the NDS and the RID comes when they are linked—when each trip is matched to the roadway segments that were traveled and each roadway segment is matched to the trips that traveled on it. The matching methodology documented in this report uses as input the GPS position data collected once per second by the NDS instrumentation and the NAVTEQ network of road segments of all public roads in the continental United States over which the NDS vehicles could travel. The Matching Algorithm associates each GPS point of an NDS trip with the road segment on which a vehicle traveled. The principal challenges overcome by the algorithm were to accommodate GPS readings that may drift far from the correct roadway and to be operationally efficient in comparing the 3.7 billion GPS readings with the 2.6 million NAVTEQ road segments that were traversed. The algorithm’s results are stored in a very large table that associates trip timestamps with road segments.

This paper explores differences in the temperature and precipitation in which individuals were found to ride within a large naturalistic riding data set. The data included in this analysis describe approximately 363,000 miles of riding by 98 participants. Trips include travel in over 40 states, as individuals rode for both transportation and pleasure. GPS location data from the motorcycles, combined with historic weather information from the National Oceanic and Atmospheric Administration databases, permitted the investigation of gender, motorcycle type, and installation locations, and their relationship to observed weather conditions during rides. Analyses showed motorcyclists prefer to ride in temperatures between 48°F and 82°F and under dry conditions when possible. If one lives in an area that experiences those temperatures year-round, then it can be expected that trips will continue year-round. If a rider lives in an area with distinctive seasons, those months where the temperature stays within that average range will contain the most rides.