A model validation framework for climate change projection and impact assessment

Abstract

Models used for projection of climate change and its impacts are usually not validated for simulation of future climate conditions. This is a serious deficiency that introduces an unknown level of uncertainty in the projections. A framework and guiding principles are presented for testing models using proxies of future conditions. In general, a model that has been setup for solving a specific problem at a particular site should be tested in order to document its predictive capability and credibility. In a climate change context such tests, often referred to as model validations tests, are particularly challenging since the model is used for an unknown future with a climate that is significantly different from current conditions. Most model studies reported on projections of climate change and its impacts have not included formal model validation tests that address this issue. A model validation framework and guiding principles for testing the capabilities of models for projection of climate change and its impacts have been proposed by Refsgaard et al. (2014). This framework is based on the hierarchical test scheme for model validation developed by Klemes (1986), which distinguishes between model predictions performed under stationary (split‐sample tests) or non‐ stationary conditions (differential split‐ sample test), and if the model is applied at the site where it was calibrated or at a different site (proxy site tests). This model validation scheme has been assessed in relation to use of different methods for projection of climate change (single and ensemble model projections and space‐timesubstitution) and use of different data sources as proxy for future climate conditions (long historical records comprising non‐ stationarity, paleo data, and controlled experiments). The basic guiding principles state that: (i) before a model is used for climate change projections and impact assessments it must demonstrate its predictive capabilities using data that reflects the expected future climate, (ii) the validation test must be carried out using data that have not been used for model calibration, and (iii) the validation test must provide evidence on the expected accuracy of the model projections and impact assessments. The most commonly used validation test, the split‐sample test, is not sufficient in a climate change context. The differential splitsample test should be applied by using adequate proxy data, reflecting future conditions. This test can be used with both single and ensemble model projections as well as with space‐time‐substitutions. It is generally expected to be more powerful when applied to a model ensemble than to a single model. Since space‐timesubstitutions include identification of locations with current climate similar to the expected future climate at the site in consideration, any test with this projection methodology involves elements of proxy site tests. For testing models under non‐ stationary conditions in a climate change context it is recommended to apply a differential split‐sample test using best available proxy data that reflect the expected future conditions at the site being considered. Such proxy data may be obtained from long historical records comprising nonstationarity, paleo data, or controlled experiments. The test can be applied with different projection methods, including single and ensemble model projections and space‐time‐ substitutions.