NOAA Storm Prediction Center (SPC) and National Severe Storms Laboratory (NSSL) have a collaborative testbed facility called the Hazardous Weather Testbed. The NOAA Hazardous Weather Testbed (HWT) has conducted Spring Experiments since 2000. The 2009 Spring Experiment took place over a 5 week period from 4 May – 5 June. A main focus of recent Spring Experiments is to gain an understanding of how to better use the output of near-cloud resolving configurations of numerical models to predict convective storms. HWT Spring Experiment participants have found through subjective evaluation that high resolution convective storm predictions are at times difficult for operational forecasters to reconcile, in part because many solutions appear to be plausible for a given mesoscale environment. The Development Testbed Center (DTC) collaborated with HWT in 2009 to help evaluate performance of three models during the Spring Experiment. The goal of the 2009 objective evaluation was to assess the impact of radar assimilation on the forecasts of strong convection. Generally speaking, the objective evaluation provided by MET supported the subjective evaluation performed by the forecasters. The results from both the traditional and spatial methods will be presented in this paper.

Convection-allowing configurations of the Weather Research and Forecast (WRF) model were evaluated during the 2004 Storm Prediction Center-National Severe Storms Laboratory Spring Program in a simulated severe weather forecasting environment. The utility of the WRF forecasts was assessed in two different ways. First, WRF output was used in the preparation of daily experimental human forecasts for severe weather. These forecasts were compared with corresponding predictions made without access to WRF data to provide a measure of the impact of the experimental data on the human decision-making process. Second, WRF output was compared directly with output from current operational forecast models. Results indicate that human forecasts showed a small, but measurable, improvement when forecasters had access to the high-resolution WRF output and, in the mean, the WRF output received higher ratings than the operational Eta Model on subjective performance measures related to convective initiation, evolution, and mode. The results suggest that convection-allowing models have the potential to provide a value-added benefit to the traditional guidance package used by severe weather forecasters.

Systematic subjective verification of precipitation forecasts from two numerical models is presented and discussed. The subjective verification effort was carried out as part of the 2001 Spring Program, a seven-week collaborative experiment conducted at the NOAA/National Severe Storms Laboratory (NSSL) and the NWS/Storm Prediction Center, with participation from the NCEP/Environmental Modeling Center, the NOAA/Forecast Systems Laboratory, the Norman, Oklahoma, National Weather Service Forecast Office, and Iowa State University. This paper focuses on a comparison of the operational Eta Model and an experimental version of this model run at NSSL; results are limited to precipitation forecasts, although other models and model output fields were verified and evaluated during the program.By comparing forecaster confidence in model solutions to next-day assessments of model performance, this study yields unique information about the utility of models for human forecasters. It is shown that, when averaged over many forecasts, subjective verification ratings of model performance were consistent with preevent confidence levels. In particular, models that earned higher average confidence ratings were also assigned higher average subjective verification scores. However, confidence and verification scores for individual forecasts were very poorly correlated, that is, forecast teams showed little skill in assessing how "good'' individual model forecasts would be. Furthermore, the teams were unable to choose reliably which model, or which initialization of the same model, would produce the "best'' forecast for a given period.The subjective verification methodology used in the 2001 Spring Program is presented as a prototype for more refined and focused subjective verification efforts in the future. The results demonstrate that this approach can provide valuable insight into how forecasters use numerical models. It has great potential as a complement to objective verification scores and can have a significant positive impact on model development strategies.

Collaborative activities between operational forecasters and meteorological research scientists have the potential to provide significant benefits to both groups and to society as a whole, yet such collaboration is rare. An exception to this state of affairs is occurring at the National Severe Storms Laboratory (NSSL) and Storm Prediction Center (SPC). Since the SPC moved from Kansas City to the NSSL facility in Norman, Oklahoma in 1997, collaborative efforts between researchers and forecasters at this facility have begun to flourish. This article presents a historical background for this interaction and discusses some of the factors that have helped this collaboration gain momentum. It focuses on the 2001 Spring Program, a collaborative effort focusing on experimental forecasting techniques and numerical model evaluation, as a prototype for organized interactions between researchers and forecasters. In addition, the many tangible and intangible benefits of this unusual working relationship are discussed.