Hydrology and Earth System Sciences www.hydrol-earth-syst-sci.net 1027-5606 1607-7938 13 7 2009 10.5194/hess-13-999-2009 http://www.hydrol-earth-syst-sci.net/13/999/2009/ http://www.hydrol-earth-syst-sci.net/13/999/2009/hess-13-999-2009.html http://www.hydrol-earth-syst-sci.net/13/999/2009/hess-13-999-2009.pdf 999 1018 2009-07-07 Analysing the temporal dynamics of model performance for hydrological models D. E. Reusser dreusser@uni-potsdam.de T. Blume B. Schaefli E. Zehe University of Potsdam, Institute for Geoecology, Potsdam, Germany Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences, Potsdam, Germany Delft University of Technology, Faculty of Civil Engineering and Geosciences, Water Resources Section, Delft, The Netherlands TU München, Institute of Water and Environment, München, Germany The temporal dynamics of hydrological model performance gives insights into errors that cannot be obtained from global performance measures assigning a single number to the fit of a simulated time series to an observed reference series. These errors can include errors in data, model parameters, or model structure. Dealing with a set of performance measures evaluated at a high temporal resolution implies analyzing and interpreting a high dimensional data set. This paper presents a method for such a hydrological model performance assessment with a high temporal resolution and illustrates its application for two very different rainfall-runoff modeling case studies. The first is the Wilde Weisseritz case study, a headwater catchment in the eastern Ore Mountains, simulated with the conceptual model WaSiM-ETH. The second is the Malalcahuello case study, a headwater catchment in the Chilean Andes, simulated with the physics-based model Catflow. The proposed time-resolved performance assessment starts with the computation of a large set of classically used performance measures for a moving window. The key of the developed approach is a data-reduction method based on self-organizing maps (SOMs) and cluster analysis to classify the high-dimensional performance matrix. Synthetic peak errors are used to interpret the resulting error classes. The final outcome of the proposed method is a time series of the occurrence of dominant error types. For the two case studies analyzed here, 6 such error types have been identified. They show clear temporal patterns, which can lead to the identification of model structural errors. Abramowitz, G., Leuning, R., Clark, M., and Pitman, A.: Evaluating the Performance of Land Surface Models, J. Climate, 21, 5468–5481, 2008. Beven, K. and Kirby, M.: A physically based variable contributing area model of basin hydrology, Hydrological Sciences Bulletin, 24, 43–69, 1979. Bezdek, J.: Pettern Recognition with Fuzzy Objective Function Algorithms, Plenum, New York, 1981. Blume, T.: Hydrological processes in volcanic ash soils - Measuring, modelling and understanding runoff generation in an undisturbed catchment, Ph.D. thesis, University of Potsdam, 2008. Blume, T., Zehe, E., and Bronstert, A.: Rainfall runoff response, event-based runoff coefficients and hydrograph separation, Hydrolog. Sci. J., 52(5), 843–862, 2007. Blume, T., Zehe, E., Reusser, D E., Iroume, A., and Bronstert, A.: Investigation of runoff generation in a pristine, poorly gauged catchment in the Chilean Andes I: A multi-method experimental study, Hydrol. Process., 22, 3661–3675, 2008. Boyle, D P., Gupta, H V., and Sorooshian, S.: Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods, Water Resour. Res., 36, 3663–3674, 2000. Buytaert, W., Reusser, D., Krause, S., and Renaud, J.-P.: Why can't we do better than Topmodel?, Hydrol. Process., 22, 4175–4179, 2008. Choi, H T. and Beven, K.: Multi-period and multi-criteria model conditioning to reduce prediction uncertainty in an application of TOPMODEL within the GLUE framework, J. Hydrol., 332, 316–336, 2007. Clark, M P., Slater, A G., Rupp, D E., Woods, R A., Vrugt, J A., Gupta, H V., Wagener, T., and Hay, L E.: Framework for Understanding Structural Errors (FUSE): A modular framework to diagnose differences between hydrological models, Water Resour. Res, 44, W00B02, \doi10.1029/2007WR006735, 2008. Cloke, H. and Pappenberger, F.: Evaluating forecasts of extreme events for hydrological applications: an approach for screening unfamiliar performance measures, Meteorol. Appl., 15, 181–197, 2008. Cottrell, M. and de~Bodt, E.: A Kohonen map representation to avoid misleading interpretations, in: 4th European Symposium on Artificial Neural Networks, http://www.dice.ucl.ac.be/esann/proceedings/papers.php?ann=1996, 1996. Dawson, C W., Abrahart, R J., and See, L M.: HydroTest: A web-based toolbox of evaluation metrics for the standardised assessment of hydrological forecasts, Environ. Modell. Softw., 22, 1034–1052, 2007. Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., and Weingessel, A.: e1071: Misc Functions of the Department of Statistics (e1071), TU Wien, r~package version 1.5-18, 2008. DWD: Deutscher Wetter Dienst (German Weather Service) Climatological data for 11~cllimate stations around the Weisseritz catchment, data, 2007. Fenicia, F., Savenije, H. H G., Matgen, P., and Pfister, L.: Understanding catchment behavior through stepwise model concept improvement, Water Resour. Res., 44, W01402, doi:10.1029/2006WR005563, 2008. Graeff, T., Zehe, E., Reusser, D., Lück, E., Schräder, B., Bronstert, A., Wenk, G., and John, H.: Process identification through rejection of model structures in a mid-mountainous rural catchment: observations of rainfall-runoff response, geophysical conditions and model inter-comparison, Hydrol. Process., 23(5), 702–718, 2009. Gupta, H., Beven, K., and Wagener, T.: Encyclopedia of Hydrological Sciences, Model Calibration and Uncertainty Estimation, John Wiley & Sons, chap 131, 1–17, 2005. Gupta, H V., Sorooshian, S., and Yapo, P O.: Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information, Water Resour. Res., 34, 751–763, 1998. Gupta, H V., Wagener, T., and Liu, Y Q.: Reconciling theory with observations: elements of a diagnostic approach to model evaluation, Hydrol. Process., 22, 3802–3813, 2008. Haykin, S.: Neural networks – A comprehensive foundation, Self-organizing maps, Prentice-Hall, 2nd edn., chap 9, 425–474, 1999. Herbst, M. and Casper, M. C.: Towards model evaluation and identification using Self-Organizing Maps, Hydrol. Earth Syst. Sci., 12, 657–667, 2008. Iroumé, A.: Transporte de sedimentos en una cuenca de montańa en la Cordillera de los Andes de la Novena Reg\'ion de Chile, Bosque, 24, 125–135, 2003. Jachner, S., van~den Boogaart, K G., and Petzoldt, T.: Statistical Methods for the Qualitative Assessment of Dynamic Models with Time Delay (R Package qualV), J. Stat. Softw., 22, 1–30, 2007. Kohonen, T.: Self-Organizing Maps, in: Series in Information Sciences, vol 30, Springer, Heidelberg, 2nd edn., 1995. Krause, P., Boyle, D P., and Bäse, F.: Comparison of different efficiency criteria for hydrological model assessment, Adv. Geosci., 5, 89-97, 2005. Lee, H., Zehe, E., and Sivapalan, M.: Predictions of rainfall-runoff response and soil moisture dynamics in a microscale catchment using the CREW model, Hydrol. Earth Syst. Sci., 11, 819–849, 2007. LfUG: Landesamt für Umwelt und Geologie Sachsen (State office for environment and geology), Data about land use, soils, discharge, and the digital elevation model, data, 2007. Lindenmaier, F., Zehe, E., Dittfurth, A., and Ihringer, J.: Process identification at a slow-moving landslide in the Vorarlberg Alps, Hydrol. Process., 19, 1635–1651, 2005. Nash, J. and Sutcliffe, J.: River flow forecasting through conceptual models part I – A discussion of principles, J. Hydrol., 10, 282–290, http://www.sciencedirect.com/science/article/B6V6C-487FF7C-1XH/1/75ac51a8910cad95dda46f4756e7a800, 1970. Niehoff, D., Fritsch, U., and Bronstert, A.: Land-use impacts on storm-runoff generation: scenarios of land-use change and simulation of hydrological response in a meso-scale catchment in SW-Germany, J. Hydrol., 267, 80–93, http://www.sciencedirect.com/science/article/B6V6C-46HBKF8-2/2/e7d43db548caa8d7c0ee195052aa4e98, 2002. Pebesma, E J., Switzer, P., and Loague, K.: Error analysis for the evaluation of model performance: rainfall-runoff event time series data, Hydrol. Process., 19, 1529–1548, 2005. R Development Core Team: R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, available at: http://www.R-project.org, ISBN~3-900051-07-0, 2008. Rango, A. and Martinec, J.: Revisiting The Degree-Day Method For Snowmelt Computations, Water Resour. Bull., 31, 657–669, 1995. Reusser, D.: tiger: Analysing TIme series of Grouped ERrors, r~package version~0.1, 2009. Schaefli, B. and Gupta, H V.: Do Nash values have value?, Hydrol. Process., 21, 2075–2080, 2007. Schulla, J. and Jasper, K.: Model Description WaSiM-ETH, 2001. Shamir, E., Imam, B., Gupta, H V., and Sorooshian, S.: Application of temporal streamflow descriptors in hydrologic model parameter estimation, Water Resour. Res., 41, W06021, doi:10.1029/2004WR003409, 2005. SRTM: Shuttle Radar Topography Mission (SRTM) Elevation Data Set, dataset, 2002. van~den Boogaart, K., Jachner, S., and Petzoldt, T.: qualV: Qualitative Validation Methods, r~package version 0.2-3. Vrugt, J A., Gupta, H V., Bastidas, L A., Bouten, W., and Sorooshian, S.: Effective and efficient algorithm for multiobjective optimization of hydrologic models, Water Resour. Res., 39, 1214, doi:10.1029/2002WR001746, 2003. Wagener, T., McIntyre, N., Lees, M J., Wheater, H S., and Gupta, H V.: Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis, Hydrol. Process., 17, 455–476, http://dx.doi.org/10.1002/hyp.1135, 2003. WASY: Schätzung dominanter Abflussprozesse mit WBS FLAB (Assessment of dominant runoff processes with WBS FLAB), Tech. rep., WASY Gesellschaft für wasserwirtschaftliche Planung und Systemforschung mbH and Internationales Hochschulinstitut Zittau, 2006. Weihs, C., Ligges, U., Luebke, K., and Raabe, N.: klaR Analyzing German Business Cycles, in: Data Analysis and Decision Support, edited by: Baier, D., Decker, R., and Schmidt-Thieme, L., Springer-Verlag, Berlin, 335–343, 2005. Xie, X. and Beni, G.: A validity measure for fuzzy clustering, IEEE T. Pattern Anal., 13, 841–847, 1991. Yan, J.: som: Self-Organizing Map, r package version 0.3-4, 2004. Yapo, P O., Gupta, H V., and Sorooshian, S.: Multi-objective global optimization for hydrologic models, J. Hydrology, 204, 83–97, 1998. Zehe, E. and Blöschl, G N.: Predictability of hydrologic response at the plot and catchment scales: Role of initial conditions, Water Resour. Res., 40, W10202, doi:10.1029/2003WR002869, 2004. Zehe, E. and Fluhler, H.: Preferential transport of isoproturon at a plot scale and a field scale tile-drained site, J. Hydrol., 247, 100–115, 2001. Zehe, E., Maurer, T., Ihringer, J., and Plate, E.: Modeling water flow and mass transport in a loess catchment, Phys. Chem. Earth Pt B, 26, 487–507, 2001. Zehe, E., Becker, R., Bardossy, A., and Plate, E.: Uncertainty of simulated catchment runoff response in the presence of threshold processes: Role of initial soil moisture and precipitation, J. Hydrol., 315, 183–202, 2005. Zehe, E., Lee, H., and Sivapalan, M.: Dynamical process upscaling for deriving catchment scale state variables and constitutive relations for meso-scale process models, Hydrol. Earth Syst. Sci., 10, 981–996, 2006.