Observation Impact Metrics In Nwp: A Theoretical Study. Part I: Optimal Systems

Two methods are widely used to assess the impact of observations in global numerical weather prediction (NWP): data denial experiments (DDEs) and the forecast sensitivity-based observation impact (FSOI) method. A DDE measures the impact on forecast accuracy of removing an observation type from the system, whereas FSOI measures the amount by which an observation type reduces the short-range forecast error (within a system containing all observation types). It is usually found, in global NWP experiments in which both methods are used, that the FSOI impact of a given observation type is greater than its DDE impact. The aim of this article is to present the theory behind the DDE and FSOI metrics that are commonly used and to explore factors that cause DDE and FSOI metrics to give different results. The general theory is presented for an optimal analysis and forecast system and then applied to a simple model with two state variables. To explain the commonly found result of FSOI impact greater than DDE impact, the following system properties are found to be important: mixing of information (and error) by the forecast model between state variables, the rate of forecast error growth, the presence of forecast model error, the way in which observational information is distributed between state variables and denied from them, and the presence of error in the data used for forecast verification. These results provide insight into why NWP systems are resilient to the removal of observational information; they are shown to be resilient if information is denied primarily from well-observed variables but not when denied primarily from poorly observed variables.