Abstract The Multi-University Consortium for Advanced Data Assimilation Research and Education (CADRE) is a new initiative recently funded by the National Oceanic and Atmospheric Administration (NOAA) to accelerate data assimilation (DA) research, education, and workforce development. Unlike previous initiatives, CADRE fosters end-to-end, direct, and comprehensive collaboration between university faculty and government agencies. It supports innovative DA research, prepares the next-generation DA workforce, and facilitates the transition of DA research to operational applications. CADRE performs a broad scope of cutting-edge research to address multiscale, nonlinear, and coupled Earth system DA challenges to improve short-range (sub-hourly) to seasonal predictions. It achieves this task through innovative data assimilation algorithm development, novel applications of machine learning (ML) in DA, and optimizing the utilization of existing and new in-situ and remotely sensed observations. Beyond research, CADRE establishes a comprehensive education, workforce development, and community building program, which includes a novel graduate student advising model, new university class curriculum development, public training courses, community scientific workshops, an international exchange program, trans-disciplinary partnerships, an outreach program, and promotion of the open sharing of data, code, and educational materials. Through this unique holistic approach, CADRE is set to strengthen both the intellectual and software infrastructure in the broad community for DA research, increase the number of DA scientists with expanded skill sets, and revolutionize forecasting capabilities.

Abstract The assimilation of all-sky infrared brightness temperatures (BTs) into short-term convective-scale forecasts has led to forecast improvements. While most studies employ an ensemble-based approach to achieve this, some limitations, such as the treatment of sampling errors and insufficient model spread, have been addressed in different ways. With the ultimate goal of informing more flexible cloud-adaptive treatments of these issues, the present study aims to better understand the vertical structure of systematic relationships between different atmospheric state variables and BT under different cloud and precipitation conditions. In particular, results are evaluated for high-altitude clouds, both with and without robust precipitation present, and for midaltitude clouds with cloud tops both above and below 400 mb (1 mb = 1 hPa). The systematic relationships to BTs are described across the different cloud categories defined within the study, and case studies are used to connect the systematic results to physical processes or features. In the high-cloud categories, the conclusions made came from a mesoscale convective system. For high-precipitating high clouds, BTs were related to the overall strength of convection and the proximity to the localized updrafts. In nonprecipitating high clouds, BT anomalies were related to the depth and hydrometeor concentration of anvil-like clouds as well as midlevel temperature and dewpoint anomalies associated with organized convection. Results in the midlevel cloud categories appeared to be generally dominated by synoptic-scale features. When cloud tops were above 400 mb, BTs were mainly related to ice-phase hydrometeors. When cloud tops were below 400 mb, BTs were mainly related to liquid-phase hydrometeors and included a contribution from what is considered the clear-air regime above the cloud.

There has been an increasing interest in recent years in the use of machine learning (ML) algorithms such as random forests (RFs) in the context of severe weather prediction. However, there is a need to better understand the impacts of large-scale flow patterns on RF model performance to further improve their use of multiscale information. This work leverages real-time forecasts from convection-allowing Finite-Volume Cubed-Sphere Dynamical Core (FV3)-based ensemble forecasts produced by the University of Oklahoma Multiscale Data Assimilation and Predictability Laboratory during the Hazardous Weather Testbed (HWT) Spring Forecasting Experiments. Results show that by using an RF trained on all forecast cases, forecast cases that have relatively high importance for certain predictors have different discernible flow patterns compared to low-importance forecast cases for the same predictors through composite differences in different environmental variables. Maximum updraft helicity storm attribute predictors were associated with strong synoptic ascent, u500 had compact shortwaves within northwesterly flow, and MUCAPE was associated with forcing displaced north of the region of severe weather. RF models trained on forecast cases with similar domain-averaged CAPE/shear were statistically more skillful than the baseline model trained irrespective of CAPE/shear patterns in forecasting the occurrence of severe weather. However, selecting training forecast cases based on spatial patterns or principal components of CAPE/shear using EOF analysis did not further improve the RF ability to forecast severe weather compared to the baseline model. The benefits of training on forecast cases based on domain-averaged CAPE/shear were maintained, and some of the benefits of training based on spatial patterns of CAPE/shear were maintained as the sample size increased.

This study aims to quantify and better understand the impact of an upgrade to the configuration of an FV3 (Finite Volume cubed-sphere) LAM (Limited Area Model) convection-allowing ensemble on the skill of the RF models trained on cases before the upgrade and forecast on cases after the upgrade. Specifically, Random Forest (RF) models were used to produce probabilistic forecasts of severe weather, significant severe weather, and individual hazards of wind, hail, and tornado for the purpose of day-1 convective outlook guidance. The RF models are trained and forecast on different subsets of the available data set of forecast cases from the spring seasons of 2019 and 2021 (before the FV3 LAM upgrade) and 2022 (after the upgrade) and evaluated both quantitatively and qualitatively. It is found for most predictands that the RF models forecasting 2022 (2019/2021) cases are statistically significantly more skillful when trained on other cases from the 2022 (2019/2021) data set using a leave-one-out approach. However, within the 2019/2021 data set, training on cases from a different year than the year being forecast also leads to statistically significant degradations of skill, apparently at least in part due to the different sample climate between 2019 and 2021. For this particular NWP (Numerical Weather Prediction) model configuration change, the consistency in sample climate between training and forecast cases is at least as important as consistency in model configuration. Finally, increases in skill resulting from increasing the number of forecast cases used to train the RF levels off around 30 forecast cases.

The impact of assimilating in situ temperature, moisture and wind observations from a WindBorne Systems balloon, with long duration and adjustable ballast, is evaluated using a case study from the 2022 THINICE field campaign. A case is selected wherein the WindBorne balloon directly sampled a jet streak associated with a tropopause polar vortex (TPV). The observed TPV merged with another TPV at the same time as a downstream Arctic cyclone (AC) redeveloped eastward. The case is used to better understand the role of the observed TPV in the evolution of the downstream AC. The assimilation of the WindBorne observations improves the forecast track and amplitude of the TPV during the similar to 1 day forecast period that the TPV can be tracked as a distinct feature. The root mean square error of temperature at 350 hPa is improved by the WindBorne assimilation throughout the 2.5 day forecast period. The surface cyclone track forecast is improved by the WindBorne assimilation during the period of eastward redevelopment of the surface cyclone, and the sea level pressure RMSE in the surrounding region is reduced during the first similar to 1.5 days of forecast lead time. Results demonstrate the capability of the long-duration controllable WindBorne balloons to improve the analysis of the TPV associated with the jet streak, leading to improved forecast of both tropopause-level and surface pressure features. Additionally, the importance of observing the upstream TPV amplitude for improving forecasts of the process of TPV merging and its impact on the evolution and longevity of the mature AC is confirmed. During a recent field campaign, WindBorne Systems deployed novel weather balloons that remain in the atmosphere for weeks at a time, adjusting their altitude to follow winds towards features of interest. These balloons provide an opportunity to study the impact of the merging of two upper-level disturbances on the performance of an Arctic cyclone (AC) forecast. WindBorne observations of one of the two merged disturbances allowed for an improved forecast of the eastward redevelopment of the AC. These experiments confirm the importance of observing such features for AC prediction and the importance of the merging process in the maintenance and evolution of long-lived Arctic cyclones. WindBorne balloons can improve in situ sampling of small scale features critical for Arctic predictability Tropopause polar vortex (TPV)-related jet streak is important for predictability of subsequent TPV merging and Arctic cyclone (AC) redevelopment Merging of TPVs without associated surface cyclone should be included in conceptual models of AC longevity

The Great Arctic Cyclone 2012 (AC12) is used to understand the role of initial condition errors in the predictability at 2–3‐day forecast range of a high‐impact summer Arctic Cyclone (AC). Ensemble sensitivity analysis (ESA) is first performed to identify potentially sensitive regions of the cyclone evolution using an ensemble baseline forecast with conventional in situ observations assimilated. A pseudo‐observation method is then introduced to investigate impacts of hypothetical observations in these sensitive but unobserved regions. In the baseline experiments with in situ observations assimilated, the forecasted AC12 reaches its peak intensity 18 hr earlier than in the verifying Global Forecast System Analysis (GFS‐ANL) and the cyclone track is biased toward the southwest. Using ESA, the time of peak intensity and the cyclone track error are identified to be sensitive to the upstream trough, downstream ridge, and the tropopause polar vortex (TPV) to the northeast (NE TPV) of the AC12. These features were not observed by the in situ observation networks. To examine the impact of the observation gaps, pseudo‐observations drawn from GFS‐ANL are assimilated. Pseudo‐observations sample the three features separately to study the impact of the initial condition error on the predictability of AC12. The cyclone peak intensity timing error and track error are greatly reduced when the initial condition error is reduced near the NE TPV. A southward expansion of the NE TPV and the corresponding southward shifting low‐level front lead the forecasted AC12 to progress to the east, which better agrees with the verifying GFS‐ANL.

The impact of assimilating in-situ observation inside a Tropopause Polar Vortex (TPV) from a novel weather balloon system, the WindBorne, on the predictability of the TPV and the coupled Arctic Cyclone (AC) is investigated using an AC case from the THINICE filed campaign. Two WindBornes continuously sampled inside and near the center of the TPV at various vertical levels for 27 hr before the cyclogenesis. The 27 hr were divided into three phases based on the different vertical levels sampled by WindBornes. The Root Mean Square Error (RMSE) of the forecasted cyclone is reduced from 12-hr to 36-hr forecast lead time as more phases of WindBornes are assimilated. This period corresponds to when the surface cyclone becomes superimposed with the TPV and rapidly deepens. Comparing the experiment with three phases of WindBorne observations assimilated versus the baseline, the cyclone expands southwestwards toward the TPV and forms a stronger coupled structure. Two analysis improvements leading to the improved AC predictability are revealed. First, a stronger circulation at and below the tropopause of the TPV is observed, leading to stronger coupling between the TPV and the surface cyclone in the forecast. Second, a mesoscale shortwave embedded in the synoptic trough is better constrained as more phases of WindBorne are assimilated. The north part of the TPV, which evolves from the shortwave trough, is therefore forecasted to be stronger. The sensitivity of forecast performance to horizontal localization parameter of the WindBorne observations is also studied. Larger localization parameters result in larger RMSE reduction compared to smaller values.

The THINICE field campaign, based in Svalbard in August 2022, provided unique observations of summertime Arctic cyclones, their coupling with cloud cover, and their interactions with tropopause polar vortices and sea ice conditions. THINICE was motivated by the need to advance our understanding of these processes and to improve coupled models used to forecast weather and sea ice, as well as long-term projections of climate change in the Arctic. Two research aircraft were deployed with complementary instrumentation. The Service des Avions Fran & ccedil;ais Instrument & eacute;s pour la Recherche en Environnement (Safire) Aerei da Trasporto Regionale 42 (ATR42) aircraft, equipped with the radar-lidar (RALI) remote sensing instrumentation and in situ cloud microphysics probes, flew in the midtroposphere to observe the wind and multiphase cloud structure of Arctic cyclones. The British Antarctic Survey Meteorological Airborne Science Instrumentation (MASIN) aircraft flew at low levels measuring sea ice properties, including surface brightness temperature, albedo and roughness, and the turbulent fluxes that mediate exchange of heat and momentum between the atmosphere and the surface. Long-duration instrumented balloons, operated by WindBorne Systems, sampled meteorological conditions within both cyclones and tropospheric polar vortices across the Arctic. Several novel findings are highlighted. Intense, shallow low-level jets along warm fronts were observed within three Arctic cyclones using the Doppler radar and turbulence probes. A detailed depiction of the interweaving layers of ice crystals and supercooled liquid water in mixed-phase clouds is revealed through the synergistic

A series of convection-allowing 36-h ensemble forecasts during the 2018 spring season are used to better understand the impacts of ensemble configuration and blending different sources of initial condition (IC) perturbation. Ten- and forty-member ensemble configurations are initialized with the multiscale IC perturbations generated as a product of convective-scale data assimilation (MULTI) and initialized with the MULTI IC perturbations blended with IC perturbations downscaled from coarser-resolution ensembles (BLEND). The forecast performance of both precipitation and nonprecipitation variables is consistently improved by the larger ensemble size. The benefit of the larger ensemble is largely, but not entirely, due to compensating for underdispersion in the fixed-physics ensemble configuration. A consistent improvement in precipitation forecast skill results from blending in the 10-member ensemble configuration, corresponding to a reduction in the ensemble calibration error (i.e., reliability component of Brier score). In the 40-member ensemble configuration, the advantage of blending is limited to the -18-22-h lead times at all precipitation thresholds and the -35-36-h lead times at the lowest threshold, both corresponding to an improved resolution component of the Brier score. The advantage of blending in the 40-member ensemble during the diurnal convection maximum of -18-22-h lead times is primarily due to cases with relatively weak synoptic-scale forcing, while advantages at later lead times beyond -30-h lead time are most prominent on cases with relatively strong synoptic-scale forcing. The impacts of blending and ensemble configuration on forecasts of nonprecipitation variables are generally consistent with the impacts on the precipitation forecasts.

This study aims to investigate how assimilating GOES-16 ABI infrared brightness temperature (BT) with different microphysics schemes affects the convection-allowing analysis and prediction of the 3 May 2020 bow echo case. The Gridpoint Statistical Interpolation-based Ensemble Kalman Filter (GSI-EnKF) system and Weather Research and Forecasting (WRF) model are utilized to conduct data assimilation (DA) experiments using Thompson, WDM6, NSSL, and Morrison microphysics schemes. Correlation structures between BT and model state variables indicate that assimilating infrared BT can adjust bowing MCS dynamics via latent cooling and the rear inflow jet. Such corrections during DA cycling enhance the rear inflow jet and bow echo size, primarily for microphysics schemes featuring faster hydrometeor fall velocity and stronger latent cooling. The improved analyses lead to better forecasts of the bow echo's shape, size, timing of the bowing process, and wind speeds. Substituting a larger microphysics-dependent effective radius for a constant default value increases prior BT, the magnitude of BT innovations, and accumulated impact on the rear inflow jet, especially for the WDM6 and Morrison schemes. In the subsequent forecasts, incorporating microphysics-dependent effective radius further improves the experiment using the Morrison scheme but degrades it when using the WDM6 scheme. Past studies have shown that incorporating infrared radiance observations, sensitive to water vapor and cloud top, into weather prediction models enhances the analysis of current weather conditions and the forecast accuracy for severe weather events. However, previous studies focused only on a single method to parameterize cloud particles and did not explore how cloud particle characteristics-such as falling speed, phase changes, and scattered size-affect the assimilation of these infrared radiance observations into model states. This study investigates how different cloud parameterization methods affect the analysis and forecast of a severe weather event on 3 May 2020. Our findings illustrate that using infrared radiance observations from the GOES-16 satellite adjusts the dynamics of a bow echo, enhancing evaporation and circulation within the storm. These adjustments notably improve the forecast accuracy for the bow echo's shape, size, timing, and wind speeds. Employing parameterization methods with faster cloud falling speeds and stronger evaporation intensifies the storm's circulation, resulting in a wider bow echo and a subsequent area of severe wind damage in the forecast. Furthermore, assigning large scattered particle sizes based on the applied cloud parameterization methods can either enhance or degrade the severe storm forecast. Assimilation of all-sky infrared radiance improves analysis and forecast of bow echo dynamics Microphysics schemes affect assimilating all-sky radiance through latent cooling and hydrometeor fall velocity assumption Improving consistency in effective radius between forward operator and microphysics scheme also affects assimilation results of all-sky radiances depending on schemes

This study evaluates simulated radiance forecasts from a series of controlled experiments consisting of FV3-LAM forecasts with different configurations of model physics and vertical resolution. The forecasts were produced during the 2020 Hazardous Weather Testbed Spring Forecasting Experiments on the same forecast cases. The evaluation includes grid-point, neighborhood-based and object-based verification. The experiments include forecasts that were identical except for the physics (EMC-LAM vs. EMC-LAMx), vertical resolution (EMC-LAMx vs. NSSL-LAM), or combined initial conditions, physics and vertical resolution (GSL-LAM). It is found that the EMC-LAM generally provided better simulated radiance forecasts than the other three configurations at most forecast lead times, due to its unique physics configuration. All configurations generally over-forecasted high level clouds. EMC-LAM reduced the over-forecasting of high clouds, but also under-forecasted the coverage of mid-level clouds. In contrast, at early lead times the EMC-LAM had relatively poor performance relative to the other forecasts. Furthermore, EMC-LAM was an outlier in terms of the vertical structure of clouds. It is also found that the NSSL-LAM consistently improved upon the EMC-LAMx, which had fewer vertical levels than NSSL-LAM. Compared to EMC-LAMx, NSSL-LAM had less cloud over-forecasting bias, especially with small cloud objects, and less overall error. The differences between EMC-LAMx and GSL-LAM were generally much smaller than the differences between EMC-LAMx and EMC-LAM/NSSL-LAM. Finally, it is found that a non-linear bias correction conditioned on symmetric brightness temperature reduced the overall root-mean-square error by about a factor of 2 while improving the unrealistic vertical structure of clouds in the EMC-LAM.

Given the large range of resolvable space and time scales in large-domain convection-allowing for ensemble forecasts, there is a need to better understand optimal initial-condition perturbation strategies to sample the forecast uncertainty across these space and time scales. This study investigates two initial-condition perturbation strategies for CONUS-domain ensemble forecasts that extend into the two-day forecast lead time using traditional and object-based verification methods. Initial conditions are perturbed either by downscaling perturbations from a coarser resolution ensemble (i.e., LARGE) or by adopting the analysis perturbations from a convective-scale, EnKF system (i.e., MULTI). It was found that MULTI had more ensemble spread than LARGE across all scales initially, while LARGE’s perturbation energy surpassed that of MULTI after 3 h and continued to maintain a surplus over MULTI for the rest of the 36h forecast period. Impacts on forecast bias were mixed, depending on the forecast lead time and forecast threshold. However, MULTI was found to be significantly more skillful than LARGE at early forecast hours for the meso-gamma and meso-beta scales (1–9h), which is a result of a larger and better-sampled ensemble spread at these scales. Despite having a smaller ensemble spread, MULTI was also significantly more skillful than LARGE on the meso-alpha scale during the 20–24h period due to a better spread-skill relation. MULTI’s performance on the meso-alpha scale was slightly worse than LARGE’s performance during the 6–12h period, as LARGE’s ensemble spread surpassed that of MULTI. The advantages of each method for different forecast aspects suggest that the optimal perturbation strategy may require a combination of both the MULTI and LARGE techniques for perturbing initial conditions in a large-domain, convection-allowing ensemble.

The unprecedentedly high space and time resolution of infrared radiance observations from GOES‐16 Advanced Baseline Imager (ABI) present an opportunity to improve analyses and short‐term forecasts of rapidly evolving convective‐scale processes such as the initiation and organization of severe supercell thunderstorms. Such a case is used for experiments aimed at better understanding the assimilation of ABI all‐sky radiance observations in GSI‐EnKF. Experiments assimilating ABI channel 10 are used to demonstrate and understand the impacts of implementing additive inflation and adaptive observation error for ABI radiance assimilation in GSI‐EnKF. The experiments using channel 10 are then compared to experiments using ABI channel 9 and both channels together. The impact of additive inflation is to increase lead time of one of the two supercells by about 20 min, and to enable improved prediction of the second supercell. The improvement occurs because the development of deepening cumulus is accelerated where ABI observes clouds to appear that are not present in the ensemble background forecast. The impact of the adaptive observation error is to increase the strength and persistence of the developing supercells in forecasts initialized from ensemble mean analyses. Compared to the ABI channel 10 radiance, ABI channel 9 provides 10 min of additional lead time for the second supercell, likely because it better constrains an upper‐level shortwave in clear air.

Abstract The present study introduces the online non‐linear bias correction for the assimilation of all‐sky GOES‐16 Advanced Baseline Imager (ABI) channel 9 (6.9 μm) radiances in a rapidly cycled EnKF for convective scale data assimilation (DA). This study is the first to explore the use of the radar reflectivity as the anchoring observation for ABI all sky radiance assimilation. The online and offline nonlinear bias correction methods are compared and evaluated for a case of rapidly developing supercells over Oklahoma and Texas. The analysis and background of the online bias correction perform better than the offline approach during the suppression of spurious clouds and the establishment of non‐precipitating and precipitating regions when the supercell storms are observed to develop. The online approach not only improves the analysis and background over the radar anchored region but also the unanchored non‐precipitating regions compared to the offline approach. Both quantitative and subjective verification of the deterministic forecasts showed consistent superior performance from the online bias correction over the offline approach. Diagnostics reveal that the online bias correction retains useful information in the innovation, which in turn improves subsequent analysis, background and background ensemble spread for both the thermodynamic and dynamic fields. The effect is accumulated during the DA cycling that is responsible for the superior analysis and forecast of the supercells.

There is a growing interest in the use of ground-based remote sensors for numerical weather prediction, which is sparked by their potential to address the currently existing observation gap within the planetary boundary layer. Nevertheless, open questions still exist regarding the relative importance of and synergy among various instruments. To shed light on these important questions, the present study examines the forecast benefits associated with several different ground-based profiling networks using 10 diverse cases from the Plains Elevated Convection at Night (PECAN) field campaign. Aggregated verification statistics reveal that a combination of in situ and remote sensing profilers leads to the largest increase in forecast skill, in terms of both the parent mesoscale convective system and the explicitly resolved bore. These statistics also indicate that it is often advantageous to collocate thermodynamic and kinematic remote sensors. By contrast, the impacts of networks consisting of single profilers appear to be flow-dependent, with thermodynamic (kinematic) remote sensors being most useful in cases with relatively low (high) convective predictability. Deficiencies in the data assimilation method as well as inherent complexities in the governing moisture dynamics are two factors that can further limit the forecast value extracted from such networks.

A case study characterized by Arctic cyclogenesis following a tropopause polar vortex (TPV)-induced Rossby wave initiation event is used to better understand how well existing observations constrain analyses of processes influencing Arctic cyclone predictive skill. Complementary techniques of observation system experiments (OSE) and ensemble sensitivity analysis (ESA) are used to investigate the impacts of existing observation networks on predictions for this case. The ESA reveals that the large-scale Rossby wave structure is correlated with both Arctic cyclone track and amplitude errors. The ensemble analyses of midlevel moisture in the warm conveyor belt region were correlated with forecast cyclone amplitude, but this feature was poorly sampled in existing observations. There is also a sensitivity of Arctic cyclone forecast amplitude error to low-level temperature in the air mass of the cyclogenesis region at analysis time and a sensitivity of Arctic cyclone forecast track error to low-level temperature in the region of an Arctic cold front and a coastal front at the analysis time. The OSEs for this case reveal that Arctic cyclone track error is more sensitive to denial of existing observations than amplitude error. While lower-level (below 700 hPa) observations had the greatest impact on the surface cyclone during the early stages, upper-level (above 500 hPa) observations had the dominant impact during its later evolution. Denying temperature from just three well-placed sondes substantially increased track error by degrading analyses of the TPV amplitude and its interaction with the waveguide and developing Rossby wave packet. These results are encouraging for further Arctic cyclone forecast improvements through addition of even a small number of well-placed observations.

Convection-allowing model (CAM) ensembles contain a distinctive ability to predict convective initiation location, mode, and morphology. Previous studies on CAM ensemble verification have primarily used neighborhood-based methods. A recently introduced object-based probabilistic (OBPROB) framework provides an alternative and novel framework in which to re-evaluate aspects of optimal CAM ensemble design with an emphasis on ensemble storm mode and morphology prediction. Herein, we adopt and extend the OBPROB method in conjunction with a traditional neighborhood-based method to evaluate forecasts of four differently configured 10-member CAM ensembles. The configurations include two single-model/single-physics, a single-model/multi-physics, and a multi-model/multi-physics configuration. Both OBPROB and neighborhood frameworks show that ensembles with more diverse member-to-member designs improve probabilistic forecasts over single-model/single-physics designs through greater sampling of different aspects of forecast uncertainties. Individual case studies are evaluated to reveal the distinct forecast features responsible for the systematic results identified from the different frameworks. Neighborhood verification, even at high reflectivity thresholds, is primarily impacted by mesoscale locations of convective and stratiform precipitation across scales. In contrast, the OBPROB verification explicitly focuses on convective precipitation only and is sensitive to the morphology of similarly located storms.

This study investigates impacts on convection-permitting ensemble forecast performance of different methods of generating the ensemble IC perturbations in the context of simultaneous physics diversity among the ensemble members. A total of 10 convectively active cases are selected for a systematic comparison of different methods of perturbing IC perturbations in 10-member convection-permitting ensembles, both with and without physics diversity. These IC perturbation methods include simple downscaling of coarse perturbations from a global model (LARGE), perturbations generated with ensemble data assimilation directly on the multiscale domain (MULTI), and perturbations generated using each method with small scales filtered out as a control. MULTI was found to be significantly more skillful than LARGE at early lead times in all ensemble physics configurations, with the advantage of MULTI gradually decreasing with increasing forecast lead time. The advantage of MULTI, relative to LARGE, was reduced but not eliminated by the presence of physics diversity because of the extra ensemble spread that the physics diversity provided. The advantage of MULTI, relative to LARGE, was also reduced by filtering the IC perturbations to a commonly resolved spatial scale in both ensembles, which highlights the importance of flow-dependent small-scale (<~10 m) IC perturbations in the ensemble design. The importance of the physics diversity, relative to the IC perturbation method, depended on the spatial scale of interest, forecast lead time, and the meteorological characteristics of the forecast case. Such meteorological characteristics include the strength of synoptic-scale forcing, the role of cold pool interactions, and the occurrence of convective initiation or dissipation.

An object-based probabilistic (OBPROB) forecasting framework is developed and applied, together with a more traditional neighborhood-based framework, to convection-permitting ensemble forecasts produced by the University of Oklahoma (OU) Multiscale data Assimilation and Predictability (MAP) laboratory during the 2017 and 2018 NOAA Hazardous Weather Testbed Spring Forecasting Experiments. Case studies from 2017 are used for parameter tuning and demonstration of methodology, while the 2018 ensemble forecasts are systematically verified. The 2017 case study demonstrates that the OBPROB forecast product can provide a unique tool to operational forecasters that includes convective-scale details such as storm mode and morphology, which are typically lost in neighborhood-based methods, while also providing quantitative ensemble probabilistic guidance about those details in a more easily interpretable format than the more commonly used paintball plots. The case study also demonstrates that objective verification metrics reveal different relative performance of the ensemble at different forecast lead times depending on the verification framework (i.e., object versus neighborhood) because of the different features emphasized by object- and neighborhood-based evaluations. Both frameworks are then used for a systematic evaluation of 26 forecasts from the spring of 2018. The OBPROB forecast verification as configured in this study shows less sensitivity to forecast lead time than the neighborhood forecasts. Both frameworks indicate a need for probabilistic calibration to improve ensemble reliability. However, lower ensemble discrimination for OBPROB than the neighborhood-based forecasts is also noted.