Hydrology and Earth System Sciences www.hydrol-earth-syst-sci.net 1027-5606 1607-7938 13 3 2009 10.5194/hess-13-395-2009 http://www.hydrol-earth-syst-sci.net/13/395/2009/ http://www.hydrol-earth-syst-sci.net/13/395/2009/hess-13-395-2009.html http://www.hydrol-earth-syst-sci.net/13/395/2009/hess-13-395-2009.pdf 395 409 2009-03-18 Mapping model behaviour using Self-Organizing Maps M. Herbst herbstm@uni-trier.de H. V. Gupta M. C. Casper Department of Physical Geography, University of Trier, Germany Department of Hydrology & Water Resources, University of Arizona, Tucson, USA Hydrological model evaluation and identification essentially involves extracting and processing information from model time series. However, the type of information extracted by statistical measures has only very limited meaning because it does not relate to the hydrological context of the data. To overcome this inadequacy we exploit the diagnostic evaluation concept of Signature Indices, in which model performance is measured using theoretically relevant characteristics of system behaviour. In our study, a Self-Organizing Map (SOM) is used to process the Signatures extracted from Monte-Carlo simulations generated by the distributed conceptual watershed model NASIM. The SOM creates a hydrologically interpretable mapping of overall model behaviour, which immediately reveals deficits and trade-offs in the ability of the model to represent the different functional behaviours of the watershed. Further, it facilitates interpretation of the hydrological functions of the model parameters and provides preliminary information regarding their sensitivities. Most notably, we use this mapping to identify the set of model realizations (among the Monte-Carlo data) that most closely approximate the observed discharge time series in terms of the hydrologically relevant characteristics, and to confine the parameter space accordingly. Our results suggest that Signature Index based SOMs could potentially serve as tools for decision makers inasmuch as model realizations with specific Signature properties can be selected according to the purpose of the model application. Moreover, given that the approach helps to represent and analyze multi-dimensional distributions, it could be used to form the basis of an optimization framework that uses SOMs to characterize the model performance response surface. As such it provides a powerful and useful way to conduct model identification and model uncertainty analyses. Abramowitz, G., Gupta, H. V., Pitman, A., Wang, Y., Leuning, R., Cleugh, H., and Hsu, K.-l.: Neural Error Regression Diagnosis (NERD): A Tool for Model Bias Identification and Prognostic Data Assimilation, J. Hydrometeorol., 7, 160–177, doi:10.1175/JHM479.1, 2006. Abramowitz, G., Pitman, A., Gupta, H. V., Kowalczyk, E., and Wang, Y.: Systematic Bias in Land Surface Models, J. Hydrometeorol., 8, 989–1001, doi:10.1175/JHM628.1, 2007. Abramowitz, G. and Gupta, H. V.: Toward a model space and model independence metric, Geophys. Res. Lett., 35, L05705, doi:10.1029/2007GL032834, 2008. Abramowitz, G., Leuning, R., Clark, M., and Pitman, A.: Evaluating the Performance of Land Surface Models, J. Climate, 21, 5468–5481, doi:10.1175/2008JCLI2378.1, 2008. Alhoniemi, E., Hollmén, J., Simula, O., and Vesanto, J.: Process Monitoring and Modeling using the Self-Organizing Map, Integr. Comput. Aid. E., 6, 3–14, 1999. Ambroise, B., Perrin, J. L., and Reutenauer, D.: Multicriterion validation of a semidistributed conceptual model of the water cycle in the Fecht Catchment (Vosges Massif, France), Water Resour. Res., 31, 1467–1482, 1995. Boogaard, H. F. P. v. d., Mynett, A. E., and Ali, M. S.: Self organizing feature maps for the analysis of hydrological and ecological data sets, in: Hydroinformatics '98, edited by: Babovic, V. M. and Larsen, L. C., Balkema, Rotterdam, The Netherlands, 733–740, 1998. Boyle, D. P., Gupta, H. V., and Sorooshian, S.: Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods, Water Resour. Res., 36, 3663–3674, 2000. Chang, D.-H.: Analysis and modeling of space-time organization of remotely sensed soil moisture, Department of Civil and Environmental Engineering, University of Cincinnati, Cincinnati, Ohio, USA, 169 pp., Ph. D. thesis, 2001. Duan, Q., Sorooshian, S., and Gupta, V. K.: Effective and efficient global optimization for conceptual rainfall-runoff models, Water Resour. Res., 28, 1015–1031, doi:10.1029/91WR02985, 1992. Franks, S. W., Gineste, P., Beven, K. J., and Merot, P.: On constraining the predictions of a distributed model: the incorporation of fuzzy estimates of saturated areas into the calibration process, Water Resour. Res., 34, 787–797, doi:10.1029/97WR03041, 1998. Gallart, F., Latron, J., Llorens, P., and Beven, K.: Using internal catchment information to reduce the uncertainty of discharge and baseflow predictions, Adv. Water Resour., 30, 808–823, doi:10.1016/j.advwatres.2006.06.005, 2007. Gupta, H. V., Sorooshian, S., and Yapo, P. O.: Toward improved calibration of hydrologic models: Multiple and noncommensurable measures of information, Water Resour. Res., 34, 751–764, 1998. Gupta, H. V., Sorooshian, S., Hogue, T. S., and Boyle, D. P.: Advances in Automatic Calibration of Watershed Models, in Calibration of Watershed Models, edited by Duan, Q., Gupta, H. V., Sorooshian, S., Rousseau, A. N., and Turcotte, R., Water Science and Application Series Vol. 6, AGU, Washington DC, USA, 9–28, 2003. Gupta, H. V., Wagener, T., and Liu, Y.: Reconciling theory with observations: elements of a diagnostic approach to model evaluation, Hydrol. Process., 22, 3802–3813, doi:10.1002/hyp.6989, 2008. Hall, M. J.: How well does your model fit the data? J. Hydroinform., 3, 49–55, 2001. Haykin, S.: Neural networks - a comprehensive foundation, 2nd ed., New Jersey, USA, 842 pp., 1999. Herbst, M. and Casper, M. C.: Towards model evaluation and identification using Self-Organizing Maps, Hydrol. Earth Syst. Sci., 12, 657–667, 2008. Hsu, K.-L., Gupta, H. V., Gao, X., Sorooshian, S., and Imam, B.: Self-organizing linear output map (SOLO): An artificial neural network suitable for hydrologic modeling and analysis, Water Resour. Res., 38, 1302, doi:10.1029/2001WR000795, 2002. Huang, M., Liang, X., and Liang, Y.: A transferability study of model parameters for the variable infiltration capacity land surface scheme, J. Geophys. Res., 108(D22), 8864, doi:10.1029/2003JD003676, 2003. Hydrotec: Rainfall-Runoff-Model NASIM – program documentation (in German), Hydrotec GmbH, Aachen, Germany, 579 pp., 2005. Kalteh, A. M. and Berndtsson, R.: Interpolating monthly precipitation by self-organizing map (SOM) and multilayer perceptron (MLP), Hydrol. Sci. J., 52, 305–317, 2007. Kaski, S.: Data Exploration Using Self Organizing Maps, Dr. thesis, Department of Computer Science and Engineering, Helsinki University of Technology, Helsinki, Finland, 57 pp., 1997. Kohonen, T.: Self-Organizing Maps, 3rd ed., Information Sciences, Springer, Berlin, Heidelberg, New York, 501 pp., 2001. Lamb, R., Beven, K., and Myrabø, S.: Use of spatially distributed water table observations to constrain uncertainty in a rainfall-runoff model, Adv. Water Resour., 22, 305–317, 1998. Lane, S. N.: Assessment of rainfall-runoff models based upon wavelet analysis, Hydrol. Process., 21, 586–607, 2007. Legates, D. R. and McCabe Jr., G. J.: Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation, Water Resour. Res., 35, 233–241, doi:10.10291998WR900018, 1999. Lin, G.-F. and Chen, L.-H.: Identification of homogeneous regions for regional frequency analysis using the self-organizing map, J. Hydrol., 324, 1–9, 2006. Lin, Z. and Beck, M. B.: On the identification of model structure in hydrological and environmental systems, Water Resour. Res., 43, W02402, doi:10.1029/2005WR004796, 2007. Lischeid, G.: A decision support system for mountain basin management using sparse data, EGU General Assembly 2006, Vienna, Geophys. Res. Abstr., 8, EGU06-A-04223, 2006. Lourenço, F. C.: Exploratory Geospatial Data Analysis Using Self-Organizing Maps – Case Study of Portuguese Mainland Regions, M.Sc. thesis, Instituto Superior de Estatística e Gestão de Informação, Universidade Nova de Lisboa, Lissabon, Portugal, 143 pp., 2005. Madsen, H.: Parameter estimation in distributed hydrological catchment modelling using automatic calibration with multiple objectives, Adv. Water Resour., 26, 205–216, 2003. Mele, P. M. and Crowley, D. E.: Application of self-organizing maps for assessing soil biological quality, Agr. Ecosyst. Environ., 126, 139–152, 2008. Nash, J. E. and Sutcliffe, J. V.: River flow forecasting through conceptual models – Part I – A discussion of principles, J. Hydrol., 10, 282–290, 1970. Peeters, L., Bação, F., Lobo, V., and Dassargues, A.: Exploratory data analysis and clustering of multivariate spatial hydrogeological data by means of GEO3DSOM, a variant of Kohonen's Self-Organizing Map, Hydrol. Earth Syst. Sci., 11, 1309–1321, 2007. Rajanayaka, C., Kulasiri, D., and Samarasinghe, S.: A Comparative Study of Parameter Estimation in Hydrology Modelling: Artificial Neural Networks and Curve Fitting Approaches, International Conference on Modelling and Simulation, MODSIM 2003 Congress, Townsville, Australia, 2003. Reusser, D. E., Blume, T., Schaefli, B., and Zehe, E.: Analysing the temporal dynamics of model performance for hydrological models, Hydrol. Earth Syst. Sci. Discuss., 5, 3169–3211, 2008. Schaefli, B. and Gupta, H. V.: Do Nash values have value? Hydrol. Proc., 21, 2075–2080, 2007. Schütze, N., Schmitz, G. H., and Petersohn, U.: Self-organizing maps with multiple input-output option for modeling the Richards equation and its inverse solution, Water Resour. Res., 41, W03022, doi:10.1029/2004WR003630, 2005. Seibert, J.: Multi-criteria calibration of a conceptual runoff model using a genetic algorithm, Hydrol. Earth Syst. Sc., 4, 215–224, 2000. Simon, G., Lendasse, A., Cottrell, M., Fort, J. C., and Verleysen, M.: Time series forecasting: Obtaining long term trends with self-organizing maps, Pattern Recogn. Lett., 26, 12, 1795–1808, doi:10.1016/j.patrec.2005.03.002, 2005. Simula, O., Vesanto, J., Alhoniemi, E., and Hollmén, J.: Analysis and Modeling of Complex Systems Using the Self-Organizing Map, in: Neuro-Fuzzy Techniques for Intelligent Information Systems, edited by: Kasabov, N. and Kozma, R., Physica Verlag (Springer), 3–22, 1999. Vesanto, J.: Using the SOM and Local Models in Time-Series Prediction, Workshop on Self-Organizing Maps, WSOM'97, Espoo, Finland, 1997. Vesanto, J.: Using SOM in Data Mining, Licentiate's thesis, Helsinki University of Technology, 57 pp., 2000a. Vesanto, J.: Neural network tool for data mining: SOM toolbox, Symposium on Tool Environments and Development Methods for Intelligent Systems (TOOLMET2000), Oulu, Finnland, 184–196, 2000b. Vesanto, J., Himberg, J., Alhoniemi, E., and Parhankangas, J.: SOM Toolbox for Matlab 5, Helsinki University of Technology, Report A57, Espoo, Finland, 60 pp., 2000. Vrugt, J. A., Weerts, A. H., and Bouten, W.: Information content of data foridentifying soil hydraulic parameters from outflow experiments, Soil Sci. Soc. Am. J., 65, 19–27, 2001. Vrugt, J. A., Bouten, W., Gupta, H. V., and Sorooshian, S.: Toward improved identifiability of hydrologic model parameters: The information content of experimental data, Water Resour. Res., 38, doi:10.1029/2001WR001118, 2002. Wagener, T., McIntyre, N., Lees, M. J., Wheater, H. S., and Gupta, H. V.: Towards reduced uncertainty in conceptual rainfall-runoff modelling: dynamic identifiability analysis, Hydrol. Process., 17, 455–476, 2003a. Wagener, T., Wheater, H. S., and Gupta, H. V.: Identification and Evaluation of Watershed Models, in Calibration of Watershed Models, edited by Duan, Q., Gupta, H. V., Sorooshian, S., Rousseau, A. N., and Turcotte, R., Water Science and Application Series Vol. 6, AGU, Washington DC, 29–47, 2003b. Warren Liao, T.: Clustering of time series data – a survey, Pattern Recogn., 38, 1857–1874, doi:10.1016/j.patcog.2005.01.025, 2005. Willmott, C. J.: On the validation of models, Phys. Geogr., 2, 184–194, 1981. Yilmaz, K. K., Gupta, H. V., and Wagener, T.: A process-based diagnostic approach to model evaluation: Application to the NWS distributed hydrologic model, Water Resour. Res., 44, W09417, doi:10.1029/2007WR006716, 2008. Zadeh, L.: Optimality and non-scalar-valued performance criteria, IEEE T. Automat. Contr., 8, 59–60, 1963.