Four different versions of the HAILCAST hail model have been tested as part of the 2014–16 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiments. HAILCAST was run as part of the National Severe Storms Laboratory (NSSL) WRF Ensemble during 2014–16 and the Community Leveraged Unified Ensemble (CLUE) in 2016. Objective verification using the Multi-Radar Multi-Sensor maximum expected size of hail (MRMS MESH) product was conducted using both object-based and neighborhood grid-based verification. Subjective verification and feedback was provided by HWT participants. Hourly maximum storm surrogate fields at a variety of thresholds and Storm Prediction Center (SPC) convective outlooks were also evaluated for comparison. HAILCAST was found to improve with each version due to feedback from the 2014–16 HWTs. The 2016 version of HAILCAST was equivalent to or exceeded the skill of the tested storm surrogates across a variety of thresholds. The post-2016 version of HAILCAST was found to improve 50-mm hail forecasts through object-based verification, but 25-mm hail forecasting ability declined as measured through neighborhood grid-based verification. The skill of the storm surrogate fields varied widely as the threshold values used to determine hail size were varied. HAILCAST was found not to require such tuning, as it produced consistent results even when used across different model configurations and horizontal grid spacings. Additionally, different storm surrogate fields performed at varying levels of skill when forecasting 25- versus 50-mm hail, hinting at the different convective modes typically associated with small versus large sizes of hail. HAILCAST was able to match results relatively consistently with the best-performing storm surrogate field across multiple hail size thresholds.

One primary goal of annual Spring Forecasting Experiments (SFEs), which are coorganized by NOAA's National Severe Storms Laboratory and Storm Prediction Center and conducted in the National Oceanic and Atmospheric Administration's (NOAA) Hazardous Weather Testbed, is documenting performance characteristics of experimental, convection-allowing modeling systems (CAMs). Since 2007, the number of CAMs (including CAM ensembles) examined in the SFEs has increased dramatically, peaking at six different CAM ensembles in 2015. Meanwhile, major advances have been made in creating, importing, processing, verifying, and developing tools for analyzing and visualizing these large and complex datasets. However, progress toward identifying optimal CAM ensemble configurations has been inhibited because the different CAM systems have been independently designed, making it difficult to attribute differences in performance characteristics. Thus, for the 2016 SFE, a much more coordinated effort among many collaborators was made by agreeing on a set of model specifications (e.g., model version, grid spacing, domain size, and physics) so that the simulations contributed by each collaborator could be combined to form one large, carefully designed ensemble known as the Community Leveraged Unified Ensemble (CLUE). The 2016 CLUE was composed of 65 members contributed by five research institutions and represents an unprecedented effort to enable an evidence-driven decision process to help guide NOAA's operational modeling efforts. Eight unique experiments were designed within the CLUE framework to examine issues directly relevant to the design of NOAA's future operational CAM-based ensembles. This article will highlight the CLUE design and present results from one of the experiments examining the impact of single versus multicore CAM ensemble configurations.

Led by NOAA's Storm Prediction Center and National Severe Storms Laboratory, annual spring forecasting experiments (SFEs) in the Hazardous Weather Testbed test and evaluate cutting-edge technologies and concepts for improving severe weather prediction through intensive real-time forecasting and evaluation activities. Experimental forecast guidance is provided through collaborations with several U.S. government and academic institutions, as well as the Met Office. The purpose of this article is to summarize activities, insights, and preliminary findings from recent SFEs, emphasizing SFE 2015. Several innovative aspects of recent experiments are discussed, including the 1) use of convection-allowing model (CAM) ensembles with advanced ensemble data assimilation, 2) generation of severe weather outlooks valid at time periods shorter than those issued operationally (e.g., 1-4 h), 3) use of CAMs to issue outlooks beyond the day 1 period, 4) increased interaction through software allowing participants to create individual severe weather outlooks, and 5) tests of newly developed storm-attribute-based diagnostics for predicting tornadoes and hail size. Additionally, plans for future experiments will be discussed, including the creation of a Community Leveraged Unified Ensemble (CLUE) system, which will test various strategies for CAM ensemble design using carefully designed sets of ensemble members contributed by different agencies to drive evidence-based decision-making for near-future operational systems.

A proposed new method for hazard identification and prediction was evaluated with forecasters in the National Oceanic and Atmospheric Administration Hazardous Weather Testbed during 2014. This method combines hazard-following objects with forecaster-issued trends of exceedance probabilities to produce probabilistic hazard information, as opposed to the static, deterministic polygon and attendant text product methodology presently employed by the National Weather Service to issue severe thunderstorm and tornado warnings. Three components of the test bed activities are discussed: usage of the new tools, verification of storm-based warnings and probabilistic forecasts from a control-test experiment, and subjective feedback on the proposed paradigm change. Forecasters were able to quickly adapt to the new tools and concepts and ultimately produced probabilistic hazard information in a timely manner. The probabilistic forecasts from two severe hail events tested in a control-test experiment were more skillful than storm-based warnings and were found to have reliability in the low-probability spectrum. False alarm area decreased while the traditional verification metrics degraded with increasing probability thresholds. The latter finding is attributable to a limitation in applying the current verification methodology to probabilistic forecasts. Relaxation of on-the-fence decisions exposed a need to provide information for hazard areas below the decision-point thresholds of current warnings. Automated guidance information was helpful in combating potential workload issues, and forecasters raised a need for improved guidance and training to inform consistent and reliable forecasts.

Wildfires pose a serious threat to life, property and the nation's natural forests. In 2012, according to the National Interagency Fire Center, 9,443 wildfires were started by lightning, burning over 6 million acres of land. Forecasts for fire weather are performed on a routine basis by the National Weather Service (NWS) Storm Prediction Center (SPC). Of their product suite, the SPC produces outlooks of critical fire weather conditions for dry thunderstorms within the continental United States. Dry thunderstorm events (hereafter, dry thunder) are a significant source of lightning caused wildfires given that little to no rainfall reaches the ground. The standard definition for dry thunder assumes one or more cloud-to-ground (CG) lightning flashes occurring in conjunction with no more than 0.1 inches of precipitation. However, it is unclear that the standard definition represents all or even most cases of lightning caused wildfires. Given the existing ambiguities in defining an important event, more precise specifications for dry thunder are required for forecasters and also for purposes of verification of forecasts. The purpose of this study is to better identify the existence of dry thunder events using observational data by examining a variety of classification methods. The findings will hopefully increase understanding of what constitutes dry thunder events by creating a more consistent, realistic, and accurate means of documenting occurrence. If results are promising, any suggested refinements to the standard definition could be implemented at SPC in the future. The following section will describe the methodology used along with the various thresholds included in the study. Section 3 will discuss the results and section 4 will summarize the results for the study. ________________________________ *Corresponding author address: Paul X. Flanagan, School of Meteorology Office 5234, 120 David L. Boren Blvd, Norman, OK 73072; E-mail: pxf11@ou.edu 2. Data and Methodology 2.1 Data

The 2014 Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE) operated for a 5-week period (5 May – 6 June) at the National Weather Center in Norman, OK. The Storm Prediction Center (SPC) and National Severe Storms Laboratory (NSSL) jointly conduct the SFE every spring to test emerging concepts and technologies for improving the prediction of hazardous convective weather. More importantly, efforts to bridge the gap between research and operations continue as a key component of the HWT with each year designed to build successful collaborations. For more background details, historical summaries of the annual SFE since 2000 can be found in both Kain et al. (2003) and Clark et al. (2012). Objective forecast verification of experimental severe weather forecasts was conducted for the third consecutive year during the 2014 SFE. The next-day subjective evaluations by SFE participants have been found to be more complete by incorporating forecast verification metrics in near real-time as opposed to performing statistical assessments only in a post-experiment fashion. As discussed in Melick et al. (2013), the subjective evaluations from the participants were generally consistent and agreed with the statistical results. The comparisons were facilitated by creating time-matched spatial plots of forecasts and observations for display on webpages linked from the SFE website. Skill scores could then be viewed for each forecast time period with the appropriate images and/or examined via table summaries. Preliminary local storm reports (LSR) have traditionally served as the primary verification dataset when computing objective performance metrics. _______________________________________ *Corresponding author address: Christopher J. Melick, NOAA/NWS/NCEP Storm Prediction Center, 120 David L. Boren Blvd, Norman, OK 73072; E-mail: chris.melick@noaa.gov Subjective assessments of the probabilistic forecast products created by the participants during the 2014 SFE were similar to what had been done in previous years. For the first time, however, individual hazard (tornado, wind, hail) probabilistic forecasts were produced by the Severe Desk led by the SPC instead of a single probabilistic forecast for total severe. This study addresses the performance of the experimental probabilistic severe hail forecasts in exploring additional verification datasets instead of solely using LSRs. Images from multiple observation sources were made available for next-day subjective comparisons during the 2014 SFE. Of these, radarderived maximum expected size of hail (MESH; Witt et al. 1998) from NSSL served as a valuable surrogate to document the occurrence of hail, especially in low-density population areas where there may be a scarcity of LSRs. A key goal is to further explore gridded MESH fields as an alternative dataset to verify the experimental hail forecasts. This also provides the opportunity to compare results in the objective forecast verification to those obtained from traditional LSRs.

The 2011 Spring Forecasting Experiment in the NOAA Hazardous Weather Testbed (HWT) featured a significant component on convection initiation (CI). As in previous HWT experiments, the CI study was a collaborative effort between forecasters and researchers, with equal emphasis on experimental forecasting strategies and evaluation of prototype model guidance products. The overarching goal of the CI effort was to identify the primary challenges of the CI forecasting problem and to establish a framework for additional studies and possible routine forecasting of CI. This study confirms that convection-allowing models with grid spacing ~4 km represent many aspects of the formation and development of deep convection clouds explicitly and with predictive utility. Further, it shows that automated algorithms can skillfully identify the CI process during model integration. However, it also reveals that automated detection of individual convection cells, by itself, provides inadequate guidance for the disruptive potential of deep convection activity. Thus, future work on the CI forecasting problem should be couched in terms of convection-event prediction rather than detection and prediction of individual convection cells.

Objective forecast verification was conducted for the second year in near real-time during the 2013 Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (2013 SFE). As part of the daily activities, experimental probabilistic forecasts for severe thunderstorms were created. These forecasts were then evaluated the next day via webpages with preliminary local storm reports (LSR) serving as the verification dataset. The idea was to further explore the value of incorporating verification metrics by comparing various scores to subjective evaluations from the participants. In addition to the forecast verification metrics examined in the 2012 SFE, the relative skill score was introduced since it was designed with a baseline reference capable of measuring skill of rare-event forecasts (i.e. severe thunderstorms). Results suggested that the relative skill scores were generally better on days with more severe weather reports. Further, the participants generated skillful forecasts at the lower probability thresholds, as the relative skill scores were predominately positive in accordance with favorable subjective ratings.

Wildfires are a serious threat to life and property in the United States and many of these wildfires in the U.S. are started by lightning. According to the National Interagency Fire Center, in 2011 alone, there were 10,249 wildfires started by lightning, resulting in over 3 million acres burned. Typically, lightning strikes which ignite wildfires are spawned from dry thunderstorms (hereafter, dry thunder). A storm is typically classified as dry thunder when lightning occurs with less than 0.1 inches of rain in a given location. Even with the advent of convection allowing models, though, errors still remain with respect to the guidance accurately predicting placement and timing of thunderstorms (e.g., Weisman et al. 2008). In addition, the forecasting challenge becomes more substantial when lightning is required to be co-located with minimal precipitation, especially since large amounts of precipitation are often a by-product of deep convection. Fire weather forecasting is one of the duties of the National Weather Service (NWS) Storm Prediction Center (SPC). The SPC is charged with creating national fire weather guidance to be utilized by the NWS Weather Forecast Offices, as well as other entities. The SPC creates dry thunder outlooks out to three days that highlight critical areas at risk for fire weather. These critical areas are typically issued when widespread dry thunder is expected to occur where dried fuels exist.

The annual Spring Forecasting Experiment (SFE) is conducted in the NOAA Hazardous Weather Testbed during the climatological peak of severe convective weather in the U.S. (Clark et al. 2011). This experiment is uniquely designed to bring together meteorological scientists and practitioners to work on emerging problems of mutual interest. In 2011, a major component of the SFE (hereafter SFE2011) was a pilot study on convective initiation (CI). The focus on CI was motivated by a growing awareness that its prediction is one of the weak links in forecasts of thunderstorm activity, presenting a significant challenge for forecasters, including those who specialize in prediction of severe convective weather, flash flooding, hazards to aviation, and other specific threats.

Objective forecast verification was conducted for the first time in near real-time during the 2012 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (2012 SFE). One of the daily activities was to test the value of verification metrics by comparing the scores to subjective impressions of the participants. For this purpose, 1-km simulated reflectivity from high-resolution model and ensemble guidance was selected for examination. The evaluation was conducted via web pages using spatial plots for distinct time frames as well as a table which summarized statistical results. Feedback from the five-week period of the 2012 SFE indicated that the neighborhood techniques applied were more useful than grid point verification methods in the evaluation process, with the fractions skill score often rated as the most preferred metric.

The NOAA Hazardous Weather Testbed (HWT) conducts annual spring forecasting experiments organized by the Storm Prediction Center and National Severe Storms Laboratory to test and evaluate emerging scientific concepts and technologies for improved analysis and prediction of hazardous mesoscale weather. A primary goal is to accelerate the transfer of promising new scientific concepts and tools from research to operations through the use of intensive real-time experimental forecasting and evaluation activities conducted during the spring and early summer convective storm period. The 2010 NOAA/HWT Spring Forecasting Experiment (SE2010), conducted 17 May through 18 June, had a broad focus, with emphases on heavy rainfall and aviation weather, through collaboration with the Hydrometeorological Prediction Center (HPC) and the Aviation Weather Center (AWC), respectively. In addition, using the computing resources of the National Institute for Computational Sciences at the University of Tennessee, the Center for Analysis and Prediction of Storms at the University of Oklahoma provided unprecedented real-time conterminous United States (CONUS) forecasts from a multimodel Storm-Scale Ensemble Forecast (SSEF) system with 4-km grid spacing and 26 members and from a 1-km grid spacing configuration of the Weather Research and Forecasting model. Several other organizations provided additional experimental high-resolution model output. This article summarizes the activities, insights, and preliminary findings from SE2010, emphasizing the use of the SSEF system and the successful collaboration with the HPC and AWC. A supplement to this article is available online (DOI:10.1175/BAMS-D-11-00040.2)

The 2014 Spring Forecasting Experiment (SFE2014) was conducted from 5 May – 6 June by the Experimental Forecast Program (EFP) of the NOAA/Hazardous Weather Testbed (HWT). SFE2014 was organized by the Storm Prediction Center (SPC) and National Severe Storms Laboratory (NSSL) with participation from numerous forecasters, researchers, and developers from around the world to test emerging concepts and technologies designed to improve the prediction of hazardous convective weather. SFE2014 aimed to address several primary goals:

Since 2007, the University of Oklahoma (OU) Center for Analysis and Prediction of Storms (CAPS) has provided the NOAA Hazardous Weather Testbed (HWT) Spring Experiment with data from their 4-km grid length storm scale ensemble forecast (SSEF) system. In 2010, the SSEF was composed of several configurations of the Weather and Research Forecasting (WRF) model as well as the Advanced Regional Prediction System (ARPS; Xue et al. 2003). To better understand the temporal and domain-wide behavior of the 26-member SSEF, data from the 5-week period encompassing the 2010 Spring Experiment (Weiss et al. 2010) were examined to create an ensemble climatology for selected meteorological parameters related to convective development, including 2-m temperature, 2-m dew point, surfacebased CAPE (SBCAPE), and 0-6 km AGL bulk wind shear. Results from the statistics will be illustrated through 2D field analyses (e.g., domain wide geographic displays at different forecast hours) to help augment the standard graphical results from the forecast climatology. Finally, some skill-related comparisons with hourly objective mesoscale analyses are presented.