Hydrology and Earth System Sciences www.hydrol-earth-syst-sci.net 1027-5606 1607-7938 13 3 2009 10.5194/hess-13-411-2009 http://www.hydrol-earth-syst-sci.net/13/411/2009/ http://www.hydrol-earth-syst-sci.net/13/411/2009/hess-13-411-2009.html http://www.hydrol-earth-syst-sci.net/13/411/2009/hess-13-411-2009.pdf 411 421 2009-03-19 Multi-criteria validation of artificial neural network rainfall-runoff modeling R. Modarres r_m5005@yahoo.com Faculty of Natural Resources, Isfahan University of Technology, Isfahan, Iran In this study we propose a comprehensive multi-criteria validation test for rainfall-runoff modeling by artificial neural networks. This study applies 17 global statistics and 3 additional non-parametric tests to evaluate the ANNs. The weakness of global statistics for validation of ANN is demonstrated by rainfall-runoff modeling of the Plasjan Basin in the western region of the Zayandehrud watershed, Iran. Although the global statistics showed that the multi layer perceptron with 4 hidden layers (MLP4) is the best ANN for the basin comparing with other MLP networks and empirical regression model, the non-parametric tests illustrate that neither the ANNs nor the regression model are able to reproduce the probability distribution of observed runoff in validation phase. However, the MLP4 network is the best network to reproduce the mean and variance of the observed runoff based on non-parametric tests. The performance of ANNs and empirical model was also demonstrated for low, medium and high flows. Although the MLP4 network gives the best performance among ANNs for low, medium and high flows based on different statistics, the empirical model shows better results. However, none of the models is able to simulate the frequency distribution of low, medium and high flows according to non-parametric tests. This study illustrates that the modelers should select appropriate and relevant evaluation measures from the set of existing metrics based on the particular requirements of each individual applications. Aksoy, H. and Dahamsheh, A.: Artificial neural network models for forecasting monthly precipitation in Jordan, Stoch. Environ. Res. Risk Assess., doi:10.1007/s00477-008-0267-x (in press and online available), 2009. Antar, M. A., Elassiouti, I., and Allam, M. N.: Rainfall-runoff modelling using artificial neural networks technique: a Blue Nile catchment case study, Hydrol. Proc., 20, 1201–1216, 2006. Blom, G.: Statistical estimates and transformed beta variables, John Wiley and Sons, NY, USA, 174 pp., 1958. Conover, W. J.: Practical Nonparametric Statistics, 3rd, Wiley, NY, USA, 592 pp., 1999 Coulibaly, P. and Evora, N. D.: Comparison of neural network methods for infilling missing daily weather records, J. Hydrol., 341, 27–41, 2007. Dawson, C. W., Abrahart, R. J., and See, L. M.: Hydrotest: A web-based toolbox of evaluation metrics for the standardized assessment of hydrological forecasts, Environ. Model. Softw., 22, 1034–1052, 2007. Dawson, C. W., Wilby, R. L., and Chen, Y.: Evaluation of artificial neural network techniques for flow forecasting in River Yangtze, China, Hydrol. Earth Syst. Sci., 6, 619–626, 2002. Dawson, C. W. and Wilby, R.: An artificial neural network approach for rainfall-runoff modeling, Hydrol. Sci. J., 43, 47–66, 1998. Jain, A. and Srinivasulu, S.: Development of effective and efficient rainfall-runoff models using integration of deterministic, real coded genetic algorithms and artificial neural network techniques, Water Resour. Res., 40, W04302, doi:10.1029/2003WR002355, 2004. Harmel, R. D. and Smith, P. K.: Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling, J. Hydrol., 337, 326–336, 2007. Harpham, C. and Dawson, C. W.: The effect of different basis functions on a radial basis function network for time series prediction: A comparative study, Neurocomputing, 69, 2161–2170, 2005. Hsu, K., Gupta, H. V., and Sorooshian, S.: Artificial neural network modeling of the rainfall–runoff process, Water Resour. Res., 31(4), 2517–1530, 1995. Khan, M. S, Coulibaly, P., and Dibike, P.: Uncertainty analysis of statistical downscaling methods, J. Hydro., 319, 357-382, 2006 Kumar, A. R. S., Sudheer, K. P., Jain, S. K., and Agarwal, P. K.: Rainfall-runoff modeling using artificial neural networks: comparison of network types, Hydrol. Proc., 19, 1277–1291. 2005. Legates, D. R. and McCabe Jr., G. J.: Evaluating the use of "goodness-of-fit" measures in hydrologic and hydroclimatic model validation, Water Resour. Res. 35(1), 233–241, 1999. Liu, J., Savenije, H. H. G., and Xu, J.: Forecast of water demand in Weinan City in China using artificial neural networks, Phys. Chem. Earth, 28(4–5), 219–224, 2003. Modarres, R.: Streamflow drought time series forecasting, Stoch. Env. Res. Risk A., 21, 223–233. 2007. Ramirez-Beltran, N. and Montes, J. A.: Neural networks to model dynamic systems with time delays, IIE T., 34, 313–327, 2002. Tokar, A. S., and Markus, M.: Precipitation-runoff modeling using artificial neural networks and conceptual models. J. Hydrol. Eng., 5, 156-161, 2000. Tokar, A. S. and Johnson, P. A.: Rainfall-runoff modeling using artificial neural networks, J. Hydrol. Eng., 4, 232–239, 1999. Smith, J. and Eli, R. B.: neural Network models of rainfall-runoff process, J. Water Res., 4, 232–239, 1995. Shahin, M. A, Maier, H. R., and Jaksa, M. B.: Evolutionary data division methods for developing artificial neural network models in geotechnical engineering. Research Report No. R 171., Department of Civil and Environmental Engineering, The University of Adelaide, Australia, 2000. Unal, N, E., Aksoy, H., and Akar, T.: Annual and monthly rainfall data generation schemes, Stoch. Environ. Res. Risk Assess., 18, 245–257, 2004. Zhang, B. and Govindaraju, R. S.: Geomorphology-based artificial neural networks for estimation of direct runoff over watersheds, J. Hydrol., 273, 18–34, 2003.