Abstract. This paper investigates the robustness of rainfall–runoff models when their parameters are transferred in time. More specifically, we propose an approach to diagnose their ability to simulate water balance on periods with different hydroclimatic characteristics. The testing procedure consists in a series of parameter calibrations over 10 yr periods and the systematic analysis of mean flow volume errors on long records. This procedure was applied to three conceptual models of increasing structural complexity over 20 mountainous catchments in southern France. The results showed that robustness problems are common. Errors on 10 yr mean flow volume were significant for all calibration periods and model structures. Various graphical and numerical tools were used to investigate these errors and unexpectedly strong similarities were found in the temporal evolutions of these volume errors. We indeed showed that relative changes in simulated mean flow between 10 yr periods can remain similar, regardless of the calibration period or the conceptual model used. Surprisingly, using longer records for parameters optimisation or using a semi-distributed 19-parameter daily model instead of a simple 1-parameter annual formula did not provide significant improvements regarding these simulation errors on flow volumes. While the actual causes for these robustness problems can be manifold and are difficult to identify in each case, this work highlights that the transferability of water balance adjustments made during calibration can be poor, with potentially huge impacts in the case of studies in non-stationary conditions.