Hydrology and Earth System Sciences www.hydrol-earth-syst-sci.net 1027-5606 1607-7938 13 10 2009 10.5194/hess-13-1757-2009 http://www.hydrol-earth-syst-sci.net/13/1757/2009/ http://www.hydrol-earth-syst-sci.net/13/1757/2009/hess-13-1757-2009.html http://www.hydrol-earth-syst-sci.net/13/1757/2009/hess-13-1757-2009.pdf 1757 1764 2009-10-01 <i>HESS Opinions</i> "Crash tests for a standardized evaluation of hydrological models" V. Andréassian vazken.andreassian@cemagref.fr C. Perrin L. Berthet N. Le Moine J. Lerat C. Loumagne L. Oudin T. Mathevet M.-H. Ramos A. Valéry Cemagref, Hydrosystems and Bioprocesses Research Unit, Antony, France EDF-LNHE, Chatou, France Cemagref, G-EAU Research Unit, Montpellier, France Université Pierre et Marie Curie, UMR Sisyphe, Paris, France EDF-DTG, Grenoble, France AgroParisTech, Paris, France now at: CSIRO, Canberra, Australia As all hydrological models are intrinsically limited hypotheses on the behaviour of catchments, models – which attempt to represent real-world behaviour – will always remain imperfect. To make progress on the long road towards improved models, we need demanding tests, i.e. true <i>crash tests</i>. Efficient testing requires large and varied data sets to develop and assess hydrological models, to ensure their generality, to diagnose their failures, and ultimately, help improving them. Andersson, L.: Improvement of runoff models. What way to go?, Nord. Hydrol., 23, 315–332, 1992. Andréassian, V., Loumagne, C., Mathevet, T., Michel, C., Oudin, L., and Perrin, C.: What is really undermining hydrologic science today? Hydrol. Process., 21, 2819–2822, 2007. Bergström, S.: Principles and confidence in hydrological modelling, Nord. Hydrol., 22, 123–136, 1991. Beven, K.: On explanatory depth and predictive power, Hydrol. Process., 15, 3069–3072, 2001. Beven, K.: Towards integrated environmental models of everywhere: uncertainty, data and modelling as a learning process, Hydrol. Earth Syst. Sci., 11, 460–467, 2007. Boughton, W. and Chiew, F.: Estimating runoff in ungauged catchments from rainfall, PET and the AWBM model, Environ. Modell. Softw., 22, 476–487, 2007. Clarke, R. T.: A critique of present procedures used to compare performance of rainfall-runoff models, J. Hydrol., 352, 379–387, 2008a. Clarke, R. T.: Issues of experimental design for comparing the performance of hydrologic models, Water Resour. Res., 44(1), W01409, doi:10.1029/2007WR005927, 2008b. de Marsily, G., Combes, P., and Goblet, P.: Comment on "Ground-water models cannot be validated", by Konikow, L. F. and Bredehoeft, J. D., Adv. Water Resour., 15, 367–369, 1992. Donelly-Makowecki, L. M. and Moore, R. D.: Hierarchical testing of three rainfall-runoff models in small forested catchments, J. Hydrol., 219, 136–152, 1999. Ewen, J. and Parkin, G.: Validation of catchment models for predicting land-use and climate change impacts, 1, Method. J. Hydrol., 175, 583–594, 1996. Fenicia, F., Savenije, H. H. G., Matgen, P., and Pfister, L.: Understanding catchment behavior through stepwise model concept improvement, Water Resour. Res., 44, W01402, doi:10.1029/2006WR005563, 2008a. Fenicia, F., McDonnell, J. J., and Savenije, H. H. G.: Learning from model improvement: On the contribution of complementary data to process understanding, Water Resour. Res., 44(6), W06419, doi:10.1029/2007WR006386, 2008b. Kay, A. L., Jones, D. A., Crooks, S. M., Calver, A., and Reynard, N. S.: A comparison of three approaches to spatial generalization of rainfall-runoff models, Hydrol. Process., 20, 3953–3973, 2006. Kirchner, J.: Getting the right answers for the right reasons: linking measurements, analyses, and models to advance the science of hydrology, Water Resour. Res., 42, W03S04, doi:10.1029/2005WR004362, 2006. Klemeš, V.: Operational testing of hydrologic simulation models, Hydrolog. Sci. J., 31, 13–24, 1986. Konikow, L. F., and Bredehoeft, J. D.: Ground-water models cannot be validated, Adv. Water Resour. 15, 75–83, 1992. Koutsoyiannis, D., Makropoulos, C., Langousis, A., Baki, S., Efstratiadis, A., Christofides, A., Karavokiros, G., and Mamassis, N.: \textitHESS Opinions: "Climate, hydrology, energy, water: recognizing uncertainty and seeking sustainability", Hydrol. Earth Syst. Sci., 13, 247–257, 2009. Le Moine, N., Andréassian, V., Perrin, C., and Michel, C.: How can rainfall-runoff models handle intercatchment groundwater flows? Theoretical study over 1040 French catchments, Water Resour. Res., 43, W06428, doi:10.1029/2006WR005608, 2007. Le Moine, N.: Le bassin versant de surface vu par le souterrain: une voie d'amélioration des performances et du réalisme des modèles pluie-débit? Université Pierre et Marie Curie, Paris, PhD~thesis, 322~pp., 2008. Linsley, R. K.: Rainfall-runoff models-an overview, in: Proceedings of the international symposium on rainfall-runoff modelling, edited by: Singh, V. P., Water Resources Publications, Littleton, CO, 3–22, 1982. Martin, P. H.: Physics of stamp-collecting? Thoughts on ecosystem model design, Sci. Total Environ., 183, 7–15, 1996. Mathevet, T., Michel, C., Andréassian, V., and Perrin, C.: A bounded version of the Nash-Sutcliffe criterion for better model assessment on large sets of basins, IAHS Red Books Series, 307, 211–219, 2006. McDonnell, J. J., Sivapalan, M., Vache, K., Dunn, S., Grant, G., Haggerty, R., Hinz, C., Hooper, R., Kirchner, J., Roderick, M. L., Selker, J., and Weiler, M.: Moving beyond heterogeneity and process complexity: a new vision for watershed hydrology, Water Resour. Res., 43(7), W07301, doi:10.1029/2006WR005467, 2007. McIntyre, N., Lee, H., Wheater, H., Young, A., and Wagener, T.: Ensemble predictions of runoff in ungauged catchments, Water Resour. Res., 41, W12434, doi:10.1029/2005WR004289, 2005. Merz, R. and Blöschl, G.: Regionalisation of catchment model parameters, J. Hydrol., 287, 95–123, 2004. Michel, C., Perrin, C., Andréassian, V., Oudin, L., and Mathevet, T.: Has basin-scale modelling advanced beyond empiricism?, in: Large sample basin experiments for hydrological model parameterization, results of the Model Parameter Experiment – MOPEX, IAHS Publication 307, edited by: Andréassian, V., Hall, A., Chahinian, N., and Schaake, J., IAHS, Wallingford, 2006. Moore, I. D. and Mein, R. G.: An evaluation of three rainfall-runoff models, Hydrological Symposium, Sydney, 122–126, 1975. Mouelhi, S., Michel, C., Perrin, C., and Andréassian, V.: Linking stream flow to rainfall at the annual time step: the Manabe bucket model revisited, J. Hydrol., 328, 283–296, 2006. Nathan, R. J. and McMahon, T. A.: The SFB model Part~I – validation of fixed model parameters, Civ. Eng. Trans., Inst. Engs. Australia, CE32, 157–161, 1990. Oreskes, N., Shrader-Frechette, K., and Belitz, K.: Verification, validation, and confirmation of numerical models in the earth sciences, Science, 263, 641–646, 1994. Oreskes, N.: Evaluation (not validation) of quantitative models, Environ. Health Persp., 106, 1453–1460, 1998. Oudin, L., Perrin, C., Mathevet, T., Andréassian, V., and Michel, C.: Impact of biased and randomly corrupted inputs on the efficiency and the parameters of watershed models, J. Hydrol., 320, 62–83, 2006. Oudin, L., Andréassian, V., Lerat, J., and Michel, C.: Has land cover a significant impact on mean annual streamflow? An international assessment using 1508 catchments, J. Hydrol., 357, 303–316, 2008. Perrin, C., Michel, C., and Andréassian, V.: Does a large number of parameters enhance model performance? Comparative assessment of common catchment model structures on 429 catchments, J. Hydrol., 242, 275–301, 2001. Perrin, C., Oudin, L., Andréassian, V., Rojas-Serna, C., Michel, C., and Mathevet, T.: Impact of limited streamflow knowledge on the efficiency and the parameters of rainfall-runoff models, Hydrolog. Sci. J., 52, 131–151, 2007. Popper, K.: The logic of scientific discovery, Routldege, London, 513~pp., 1959. Refsgaard, J. C. and Knudsen, J.: Operational validation and intercomparison of different types of hydrological models, Water Resour. Res., 32, 2189–2202, 1996. Refsgaard, J. C. and Henriksen, H. J.: Modelling guidelines – terminology and guiding principles, Adv. Water Res., 27, 71–82, 2004. Roche, M.: Les divers types de modèles déterministes, La Houille Blanche, 111–129, 1971. Savenije, H. H. G.: \textitHESS Opinions "The art of hydrology", Hydrol. Earth Syst. Sci., 13, 157–161, 2009. Schaake, J., Duan, Q., Andréassian, V., Franks, S., Hall, A., and Leavesley, G.: The model parameter estimation experiment – MOPEX, J. Hydrol., 320, 1–2, 2006. Seibert, J.: Reliability of model predictions outside calibration conditions, Nord. Hydrol., 34, 477–492, 2003. Silberstein, R. P.: Hydrological models are so good, do we still need data? Environ. Modell. Softw., 21, 1340–1352, 2005. Smith, M. B., Seo, D. J., Koren, V. I., Reed, S. M., Zhang, Z., Duan, Q., Moreda, F., and Cong, S.: The distributed model intercomparison project (DMIP): motivation and experiment design, J. Hydrol., 298, 4–26, 2004. Vandewiele, G. L., Xu, C. Y., and Win, N. L.: Methodology and comparative study of monthly models in Belgium, China and Burma, J. Hydrol., 134, 315–347, 1992. WMO: Intercomparison of conceptual models used in operational hydrological forecasting, Operational Hydrology Report no 7 and WMO no 429, World Meteorological Organization, Geneva, 1975. WMO: Intercomparison of models of snowmelt runoff, Operational Hydrology Report no 23 and WMO no 646, World Meteorological Organization, Geneva, 1986. WMO: Simulated real-time intercomparison of hydrological models, Operational Hydrology Report no 38 and WMO no 779, World Meteorological Organization, Geneva, 1992. Xu, C. Y.: Operational testing of a water balance model for predicting climate change impacts, Agr. Forest Meteorol., 98(9), 295–304, 1999. Young, A. R.: Stream flow simulation within UK ungauged catchments using a daily rainfall-runoff model, J. Hydrol., 320, 155–172, 2006.