Climate change impact assessment as function of model inaccuracy

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Climate change impact assessment as function of model inaccuracy P. Droogers, A. Van Loon, W. Immerzeel

in The Netherlands an innovative procedure has been introduced, referred to as the Model-Scenario-Ratio (MSR), to express model inaccuracy on climate change impact assessment. MSR values close to 1, indicating that impact assessment is mainly a function of the scenario itself rather than of the quality of the model, were found for most indicators evaluated. More extreme climate change scenarios and indicators based on threshold values showed lower MSR values, indicating that model accuracy is an important component of the climate change impact assessment. It was concluded that the MSR approach can be applied easily and will lead to more robust impact assessment analyses. 15 Numerical simulation models have been used extensively in climate change research over the last decades. In general these models have been applied for two types of research: climate change projections and climate change impact and adaptation. Typical examples of the first are the so-called General Circulation Models (GCM) which are defined as "a numerical representation of the climate system based on the physi-tain and downscale climate change projections, (iv) run the calibrated model with the downscaled climate change projections, and (v) evaluate impact of climate change (= difference between current situation and expected future). These five steps might be followed by an evaluation of potential adaptation strategies.

Introduction
A typical example of such a study is given by Brouyère et al. (2004) where a de-10 tailed physical hydrological model was extensively calibrated to mimic reality. Feeding the model with several climate change scenarios it was concluded that groundwater levels would decline under climate change. On a smaller scale Roberto et al. (2006) started with calibrating the detailed crop growth model DSSAT for two crops. Based on the calibrated model they evaluated the impact of climate change factors precipita- 15 tion, radiation and temperature on crop production. Along the same lines, a detailed agro-hydrological model was applied to study the impact of climate change on crop production in a basin in Sri Lanka (Droogers, 2004). This research expanded the impact assessment to an analysis of potential adaptation strategies to overcome negative impacts of climate change. This study was part of a seven countries study, where sim-Introduction  Choi and Beven, 2007;Mantovan and Todini, 2006;Feyen et al., 2007), there is a common hypothesis that model errors are reflected in the reference situation as well as in the climate change situation so that relative accuracy (difference between reference and scenario) is higher than absolute accuracy of the model. This hypothesis is so-far not tested nor has any attempt being made to 5 develop a common framework to assess the error due to model inaccuracy on climate change impact assessment studies. Related to this is the question to which level of detail model calibration and validation should be undertaken to ensure a reliable impact assessment.
In summary, the objective of this research is to develop an approach to evaluate and 10 quantify the consequence of model inaccuracy on climate change impact assessment studies.

Study area
A polder in The Netherlands managed by the Waterboard of Rivierenland is selected to 15 evaluate the impact of model inaccuracy on climate change impact assessment (Fig. 1). The area is located between the rivers Meuse and Rhine, and is characterised by lowlying meadows and many drainage canals. Soils in the study area are loamy clays and are described by the Mualem -Van Genuchten (MVG) parameter set (Van Genuchten, 1980). Meteorological data are taken from the meteorological station Megen, about 20 7 km south-west of the study area. Potential economic returns on pastures in the area are 1350 per hectare (LEI, 2006). Details about the study area can be found elsewhere (Immerzeel et al., 2007). Introduction The Soil-Water-Atmosphere-Plant (SWAP) model was applied to simulate all the terms of the water balance and to estimate yields for a reference situation and two climate change scenarios. SWAP is an integrated physically based simulation model for water, solute and heat transport in the saturated-unsaturated zone in relation to crop growth.

5
A first version of the SWAP model was already developed in 1978 (Feddes et al., 1978) and from then on, a continuous development of the program started. The version used for this study is SWAP 3.03 and is described by Kroes and Van Dam (2003). The core part of the program is the vertical flow of water in the unsaturated-saturated zone, which can be described by the well-known Richards' equation: where, θ denotes the soil water content (cm 3 cm −3 ), t is time (d), h (cm) the soil matric head, z (cm) the vertical coordinate, taken positive upwards, K the hydraulic conductivity as a function of water content (cm d −1 ). S (d −1 ) represents the water uptake by plant roots (Feddes et al., 1978), defined in case of a uniform root distribution 15 as: where, T pot is potential transpiration (cm d −1 ), z r is rooting depth (cm), and α (-) is a reduction factor as function of h and accounts for water deficit and oxygen deficit. Total actual transpiration, T act , is calculated as the depth integral of the water uptake function 20

S.
Crop yields can be computed using a simple crop-growth algorithm based on Doorenbos and Kassam (1979) or by using a detailed crop-growth simulation module that partitions the carbohydrates produced between the different parts of the plant, Introduction EGU as a function of the different phenological stages of the plant (Van Diepen et al., 1989). For this specific case, the first method was used as detailed crop parameters were lacking.
The SWAP model has been applied and tested already for many different conditions and locations and has been proven to produce reliable and accurate results (e.g. Bas-5 tiaanssen et al., 2007;Heinen, 2006;Varado et al., 2006;Droogers et al., 2000). A more detailed description of the model and all its components are beyond the scope of this paper, but can be found in Kroes and Van Dam (2003).
A SWAP model was built for the study area in the Dutch polder using best data available. An automatic calibration procedure followed using the PEST software (Doherty,10 2000), with observed groundwater levels for a six years period (1997 to 2003) as references. Details of the entire procedure for this study area are presented elsewhere (Van Loon et al., 2007 2 ).

Climate change scenarios
The Fourth Assessment Report (AR4) of the Intergovernmental Panel on Climate

15
Change (IPCC) was published recently (IPCC, 2007) and is a condense result from thousands of scientific publications into a general assessment of the current knowledge about the climate system and the man-induced changes to it. Despite this wealth of information, regional and local climate change predictions are still hard to make due to the complexity of the climate system. A regional manifestation of climate change 20 is subject to many interacting processes affecting atmospheric circulation and regionspecific responses of physical processes.

EGU
A total of four scenarios have been developed based on two sets of two assumptions: global increase of temperatures by 1 or 2 • C in 2050 and whether dominant wind directions will change to more eastern directions. For this study two scenarios with a global temperature increase of 2 • C in 2050 have been selected: W (= warm) and W+ (= warm and changes in wind directions). A summary of these scenarios is provided 5 in Table 1. Details of the entire procedure in which way these scenarios are developed can be found in Hurk et al. (2006).

Model inaccuracy
To evaluate the impact of model inaccuracy on impact assessment of climate change the calibrated model has been altered to reflect the most common model uncertainties.
For the SWAP model these are: (i) soil characteristics, (ii) bottom boundary condition and (iii) crop characteristics (Van Dam, 2000). For each of these three cases one subcase with 10% lower than calibrated and one with 10% higher than calibrated values have been used, resulting in a total of six cases. These six cases and how this was implemented in terms of parameter changes in 15 the calibrated model are: -ErrorSoils10% less clayey: all soil parameters of the MVG set have been altered by 10% from the optimal value. This was implemented by changing values for top and sub soils by the following percentages: n +10%; alfa -10%; log(Ksat) +10% -ErrorSoils10% more clayey: all soil parameters of the MVG set have been altered 20 by 10% from the optimal value. This was implemented by changing values for top and sub soils by the following percentages: n -10%; alfa +10%; log(Ksat) -10% -ErrorBottom10% more dynamics in seepage: bottom boundary condition altered by 10% from the calibrated value. In SWAP this was implemented by increasing two values determining the bottom boundary condition by 10%: SINAVE = aver- -ErrorCrop10% more drought resistance. This was implemented by increasing the 5 threshold value were reduction in root water uptake occurs by +10%. In SWAP this is defined by the two parameters: HLIM3 and HLIM4.
-ErrorCrop10% less drought resistance. This was implemented by decreasing the threshold value were reduction in root water uptake occurs by -10%. In SWAP this is defined by the two parameters: HLIM3 and HLIM4.

10
To compare the reference situation to these six cases a set of indicators was defined that describe key characteristics of the entire system in one number. Values for the following seven indicators have been extracted from the daily SWAP runs over 30 years (1976-2005): In summary, the entire approach is based on applying a well-calibrated model that was altered for six cases to reflect model inaccuracy. These seven models have been run for a period of 30 years (1976 to 2005) and daily model output was summarized by 5 seven indicators.

Results
Simulated and observed groundwater depths have been compared, indicating that the model as developed is able to mimic reality (Fig. 2). Statistical analysis reveals also that simulated groundwater depths are close to observed ones with: mean error is 10 0.2 cm, average mean error is 8.2 cm, root mean square error is 11.5 cm, and R 2 of 0.86. More details of the complete calibration and validation procedure can be found in Van Loon et al. (2007) 2 . A straight forward climate change impact assessment, based on indicator comparison as defined before, is shown in Table 2. The overall trend is that climate change will 15 increase water shortage, more wet days as well as more dry days, a small reduction in economic returns and a higher chance of crop failure. Especially the W+ scenario will have a substantial impact on water and agriculture with water shortages increasing by 74 mm per year (+143%), an increase of dry days by 263% and an increase of years with complete crop failure from 2 to 8 out of 30 years.

20
These results are based on the assumption that the model mimics reality. It is however impossible by definition that models are entirely correct and the question to be asked is what the impact would be if errors exist in models. Table 3 shows an impact assessment similar as presented in Table 2  EGU time the number of dry days increased substantially. This apparent contradiction can be explained by soil water dynamics where the less clayey soil increases capillary rise which reduces water shortages but lowers groundwater tables at the same time.
Comparing impact assessment based on the accurate and the inaccurate model, comparable trends can be observed for the W scenario (column 4 in Table 2 and Ta-5 ble 3). In other words, impact assessment based on this less accurate model will result in similar conclusions as based on the accurate model. For the W+ scenario, however, impact assessment based on the inaccurate model is somewhat deviating from the one based on the accurate model (column 5 in Table 2 and Table 3). So evaluating this W+ climate change scenario using an inaccurate model will yield to less reliable 10 conclusions.
Differences between impact assessment based on the accurate and the inaccurate model (Table 2 and Table 3) are shown in Table 4. First two columns in the Table reflect the error in actual values by using the inaccurate model, e.g. the impact of water shortage is underestimated by 19 and 21% (W and W+). However, if one is 15 not concentrating on actual changes due to climate change but on relative differences, model inaccuracy is less pronounced (last two columns in Table 4).
To for 50% by the model and for 50% by the scenario itself. MSR values lower than zero indicate that results are dominated by model inaccuracy rather than by the scenario evaluated. EGU (less clayey). For the W scenario MSR values are all above 0.90 indicating that even with these model inaccuracies more than 90% of the scenario impact assessment is due to the scenario itself rather than to model inaccuracies. The W+ scenario shows a different picture where the impact of the scenario is for most indicators still dominated by the scenario itself, except for the number of dry days (GWLdry) and the years with 5 complete crop failure. For these indicators model inaccuracy is the dominant factor leading to an erroneous impact assessment. Previous sections were focused on one case of model inaccuracy: errors in soil characteristics (less clayey soil). Other cases of model inaccuracy have been evaluated in same manner resulting in MSR values for a total of six cases of model inaccuracy, 10 two scenarios and seven indicators (total of 84 MSR values). A graphical display of these 84 MSR values is shown in Fig. 3. It is clear from the figure that in general MSR values for the W scenario are higher than for the more extreme W+ scenario. For the W scenario only 5, out of the 42 combinations of model cases and indicators, have a MSR value lower than 0.95. For the W+ scenario this is 15 out of 42. 15 These indicators have been combined as well to evaluate the overall impact of model inaccuracy. Most indicators are relatively insensitive to model inaccuracy and in general more than 90% of the impact assessment analysis is a result of the scenario considered (Table 6). However, two indicators are very sensitive to model inaccuracy and if these incorrect models are used for impact assessment huge erroneous results might be 20 produced. These two indicators are number of dry days (GWLdry) and number of years with crop failure (Crop Fail). The latter is biased by the small number of year based on a threshold value where a large change in results (like from 2 to 3 times in 30 years) can be triggered by a relative small model errors. The GWLdry indicator is also based on a threshold value and is also very sensitive to small model errors. 25 Finally, the average MSR for the indicators for each model inaccuracy case can be seen in Table 7. This has been done for the full set of seven indicators as well of a reduced set of five by leaving out the most erroneous ones (GWLdry and Crop Fail). The overall result, for the five indicators, is that of the evaluated climate change impact EGU assessment more than 90% can be contributed to the scenario itself rather than to model inaccuracy. In cases where an indicator is required that is sensitive to model inaccuracy more emphasis should be put on model calibration and validation.

Conclusions and recommendations
The research was based on applying existing simulation modelling for an area in The 5 Netherlands to assess the impact of model inaccuracy on climate change impact assessment. Based on this research main conclusions are: -The calibrated model can be considered as state-of-the-art and has been applied successfully over a wide range of applications. In this case the model is performing very well in simulating groundwater levels.
-In terms of climate change impact assessment for the two scenarios (W and W+) an increase in water shortage, more extremes in wet and dry periods, and a small reduction in agriculture production can be expected.
-The derived Model-Scenario-Ratio shows that for the less extreme scenario (W) model inaccuracy is for most indicators not relevant.

15
-The derived Model-Scenario-Ratio shows that for the more extreme scenario (W+) model inaccuracy plays a role for some indicators. However, still more than 90% of the assessed impact can still be contributed to the scenario itself and not to model inaccuracy.
The overall recommendation from this study is that the climate change impact assess-20 ment study as presented is quite robust and model inaccuracy is for most cases a less relevant issue. However, for some indicators and for the more extreme scenarios model inaccuracy can play an important role. It would therefore be recommended for future climate change impact assessment studies to explore the relevance of model inaccuracy using the Model-Scenario-Ratio as defined. MSR values close to 1 indicate that EGU the model used for the impact assessment is robust and results are mainly a function of the scenario itself rather than of the quality of the model. Low MSR values indicate that the quality of the model is an important factor in determining the indicator value and it is therefore essential that the model should be calibrated and validated at a very detailed level.

5
Finally, this research should be considered as a starting point to pay more attention to the importance of model accuracy in climate change impact assessment studies. More research is required to answer a range of questions such as: (i)