Finding plausible and diverse variants of a climate model. Part II: development and validation of methodology

The usefulness of a set of climate change projections largely depends on how well it spans a range of outcomes consistent with known uncertainties. Here, we present exploratory work towards developing a strategy to select variants of a state-of-the-art but expensive climate model suitable for climate projection studies. The strategy combines information from a set of relatively cheap, idealized perturbed parameter ensemble (PPE) and CMIP5 multi-model ensemble (MME) experiments, and uses two criteria as the basis to select model variants for a PPE suitable for future projections: (a) acceptable model performance at two different timescales, and (b) maintaining diversity in model response to climate change. This second part of a pair of papers builds upon Part I in which we established a strong relationship between model errors at weather and climate timescales across a PPE for a variety of key variables. This relationship is used to filter out parts of parameter space that do not give credible simulations of present day climate, while minimizing the impact on ranges in forcings and feedbacks that drive model responses to climate change. We use statistical emulation to explore the parameter space thoroughly, and demonstrate that about 90% can be filtered out without affecting diversity in global-scale climate change responses. This leads to the identification of plausible parts of parameter space from which model variants can be selected for projection studies. We selected and ran 50 variants from the plausible parameter combinations and validated the emulator predictions. Comparisons with the CMIP5 MME demonstrate that our approach can produce a set of plausible model variants that span a relatively wide range in model response to climate change. We also highlight how the prior expert-specified ranges for uncertain model parameters are constrained as a result of our methodology, and discuss recommendations for future work.