A verification system has been developed for the ensemble prediction system (EPS) at the Canadian Meteorological Centre (CMC). This provides objective criteria for comparing two EPSs, necessary when deciding whether or not to implement a new or revised EPS. The proposed verification methodology is based on the continuous ranked probability score (CRPS), which provides an evaluation of the global skill of an EPS. Its reliability/resolution partition, proposed by Hersbach, is used to measure the two main attributes of a probabilistic system. Also, the characteristics of the reliability are obtained from the two first moments of the reduced centered random variable (RCRV), which define the bias and the dispersion of an EPS. Resampling bootstrap techniques have been applied to these scores. Confidence intervals are thus defined, expressing the uncertainty due to the finiteness of the number of realizations used to compute the scores. All verifications are performed against observations to provide more independent validations and to avoid any local systematic bias of an analysis. A revised EPS, which has been tested at the CMC in a parallel run during the autumn of 2005, is described in this paper. This EPS has been compared with the previously operational one with the verification system presented above. To illustrate the verification methodology, results are shown for the temperature at 850 hPa. The confidence intervals are computed by taking into account the spatial correlation of the data and the temporal autocorrelation of the forecast error. The revised EPS performs significantly better for all the forecast ranges, except for the resolution component of the CRPS where the improvement is no longer significant from day 7. The significant improvement of the reliability is mainly due to a better dispersion of the ensemble. Finally, the verification system correctly indicates that variations are not significant when two theoretically similar EPSs are compared.

The Canadian Meteorological Centre (CMC) started running an Ensemble Prediction System in January 1996 with an ensemble of eight members (Houtekamer et al. 1996; Lefaivre et al. 1997). This set using eight different versions of the spectral model (SEF) was extended to sixteen members in September 1999 by adding eight different versions of the Global Environmental Multi-scale model (GEM). The models differ in their physical parameterizations and their dynamical cores. The horizontal resolution was increased in June 2001: the SEF models went from TL95 to TL149 with an equivalent increase from 1.5 degrees to 1.2 degrees for the grid point GEM model (Pellerin et al. 2003).

An ensemble Kalman filter (EnKF) has been implemented for atmospheric data assimilation. It assimilates observations from a fairly complete observational network with a forecast model that includes a standard operational set of physical parameterizations. To obtain reasonable results with a limited number of ensemble members, severe horizontal and vertical covariance localizations have been used.It is observed that the error growth in the data assimilation cycle is mainly due to model error. An isotropic parameterization, similar to the forecast-error parameterization in variational algorithms, is used to represent model error. After some adjustment. it is possible to obtain innovation statistics that agree with the ensemble-based estimate of the innovation amplitudes for winds and temperature. Currently, no model error is added for the humidity variable, and, consequently, the ensemble spread for humidity is too small. After about 5 days of cycling. fairly stable global filter statistics are obtained with no sign of filter divergence.The quality of the ensemble mean background field, as verified using radiosonde observations. is similar to that obtained using a 3D variational procedure. In part, this is likely due to the form chosen for the parameterized model error. Nevertheless, the degree of similarity is surprising given that the background-error statistics used by the two procedures are rather different, with generally larger background errors being used by the variational scheme.A set of 5-day integrations has been started from the ensemble of initial conditions provided by the EnKF. For the middle and lower troposphere. the growth rates of the perturbations are somewhat smaller than the growth rate of the actual ensemble mean error. For the upper levels, the perturbation patterns decay for about 3 days as a consequence of diffusive model dynamics. These decaying perturbations lend to severely underestimate the actual error that grows rapidly near the model top.

The present paper summarizes the methodologies used at the European Centre for Medium-Range Weather Forecasts (ECMWF), the Meteorological Service of Canada (MSC), and the National Centers for Environmental Prediction (NCEP) to simulate the effect of initial and model uncertainties in ensemble forecasting. The characteristics of the three systems are compared for a 3-month period between May and July 2002. The main conclusions of the study are the following:(.) the performance of ensemble prediction systems strongly depends on the quality of the data assimilation system used to create the unperturbed (best) initial condition and the numerical model used to generate the forecasts;(.) a successful ensemble prediction system should simulate the effect of both initial and model-related uncertainties on forecast errors; and(.) for all three global systems, the spread of ensemble forecasts is insufficient to systematically capture reality, suggesting that none of them is able to simulate all sources of forecast uncertainty.The relative strengths and weaknesses of the three systems identified in this study can offer guidelines for the future development of ensemble forecasting techniques.

Introduction The ensemble Kalman filter (EnKF) is a 4D data assimilation method that uses a Monte-Carlo ensemble of short-range forecasts to estimate the covariances of the forecast error (Evensen 1994; Burgers et al. 1998; Houtekamer and Mitchell 1998). It is a close approximation to the standard Kalman filter. The approximation becomes more accurate as the ensemble size increases. The EnKF is conceptually simple. It does not depend strongly on the validity of hypotheses about the linearity of the model dynamics and requires neither a tangent linear model nor its adjoint. In addition, it parallelizes well. Like most modern data assimilation methods, the EnKF directly assimilates observed radiance data. This aspect of the EnKF, and in particular the assimilation of AMSU-A microwave radiances, is the focus of this presentation. First, the EnKF and the experimental environment are briefly described. Then we focus on how the EnKF assimilates the AMSU-A microwave radiances and show some results indicating their impact with the EnKF (including a comparison with similar results from a 3D-Var system). The present text ends with some concluding remarks and a brief outline of our future plans in this area.

The present paper summarizes the methodologies used at the European Centre for Medium-Range Weather Forecasts (ECMWF), the Meteorological Service of Canada (MSC), and the National Centers for Environmental Prediction (NCEP) to simulate the effect of initial and model uncertainties in ensemble forecasting. The characteristics of the three systems are compared for a 3-month period between May and July 2002. The main conclusions of the study are that: The performance of ensemble prediction systems strongly depends on the quality of the data assimilation system used to create the unperturbed (best) initial condition, and the numerical model used to generate the forecasts; A successful ensemble prediction system should simulate the effect of both initial and model related uncertainties on forecast errors; and For all three global systems, the spread of ensemble forecasts are insufficient to systematically capture reality, suggesting that none of them is able to simulate all sources of forecast uncertainty. The relative strengths and weaknesses of the three systems identified in this study can offer guidelines for the future development of ensemble forecasting techniques.

Ensemble forecasts are run operationally since February 1998 at the Canadian Meteorological Centre, with outputs up to ten days. The ensemble size was increased from eight to sixteen members in August 1999. The method of producing the perturbed analyses consists of running independent assimilation cycles that use perturbed sets of observations and are driven by eight different models, mainly different in their physical parameterizations. Perturbed analyses are doubled by taking opposite pairs. A multi-model approach is then used to obtain the forecasts. The ensemble output has been used to generate several products. In view of increasing computing facilities, the ensemble prediction system horizontal resolution was increased to TL149 in June 2001. Heights at 500 hPa and mean sea-level pressure maps are regularly used. Charts of precipitation with the probability of precipitation being above various thresholds are also produced at each run. The probabilistic forecast of the 24-h accumulated precipitation has shown skill as demonstrated by the relative operating characteristic (ROC). Verifications of the ensemble forecasts will be presented.

The ensemble Kalman filter (EnKF) has been proposed for operational atmospheric data assimilation. Some outstanding issues relate to the required ensemble size, the impact of localization methods on balance, and the representation of model error.To investigate these issues, a sequential EnKF has been used to assimilate simulated radiosonde, satellite thickness, and aircraft reports into a dry, global, primitive-equation model. The model uses the simple forcing and dissipation proposed by Held and Suarez. It has 21 levels in the vertical, includes topography, and uses a 144 x 72 horizontal grid. In total, about 80 000 observations are assimilated per day.It is found that the use of severe localization in the EnKF causes substantial imbalance in the analyses. As the distance of imposed zero correlation increases to about 3000 km, the amount of imbalance becomes acceptably small.A series of 14-day data assimilation cycles are performed with different configurations of the EnKF. Included is an experiment in which the model is assumed to be perfect and experiments in which model error is simulated by the addition of an ensemble of approximately balanced model perturbations with a specified statistical structure. The results indicate that the EnKF, with 64 ensemble members, performs well in the present context.The growth rate of small perturbations in the model is examined and found to be slow compared with the corresponding growth rate in an operational forecast model. This is partly due to a lack of horizontal resolution and partly due to a lack of realistic parameterizations. The growth rates in both models are found to be smaller than the growth rate of differences between forecasts with the operational model and verifying analyses. It is concluded that model-error simulation would be important, if either of these models were to be used with the EnKF for the assimilation of real observations.

A new, higher-resolution version of the regional forecast system was implemented into operations at the Canadian Meteorological Centre during 1995. The new version of the regional finite-element forecast model is run at 35-km resolution in the horizontal and 28 sigma levels in the vertical (instead of 50-km and 25 levels in the previous operational version), with a more advanced physics package. The improved physical parametrizations feature the following: 1) modifications to the treatment of surface surface processes; 2) changes to the surface layer formulation; 3) an explicit cloud scheme following Sundqvist for stratiform precipitation; 4) Fritsch-Chappell scheme for deep convection. The new regional forecast system also includes a pseudo-analysis of initial soil moisture content based on model error feed-back.Both objective and subjective evaluations on case studies and in the parallel runs showed improved performance with the new 35-km model. The main points from these verifications indicated a significant reduction of the moist bias near the surface, improvements in the predicted surface temperatures and the diurnal cycle, better forecasts of convective precipitation, and more realistic surface wind forecasts, especially over complex orography due to the increased resolution. The new predicted cloud parameters permit a better representation of the cloud-radiation interactions that are important for the atmospheric energy balance, especially at the surface.

On 3 November 1993 a new higher-resolution version of the regional forecast system was implemented into operations at the Canadian Meteorological Centre. The changes include modifications to the regional data assimilation system and to the regional finite-element (RFE) forecast model. The main features of the new version of the RFE model include an increase in resolution from 100 to 50 km and to 25 sigma levels in the vertical. The fields that describe the surface characteristics are generated directly on the 50-km grid of the model from high-resolution global geophysical datasets, yielding more derails and a much better definition of the orography and coastlines. The new RFE model also includes an improved package of physical parametrizations, notably for condensation and radiation processes. The major changes to the data assimilation are a higher-resolution analysis, and the assimilation of humidity profiles retrieved from satellite imagery.The new system is evaluated using performance statistics, and case studies are presented to highlight some of the benefits. These include more accurate analyses with a better fit to the data, and more detailed and precise forecasts, particularly for frontal zone structures, jet streams, moisture distribution and precipitation. The new physics package reduces the spinup of the model and the systematic errors in precipitation amounts, and gives better thermal and hydrologic balances.