Introduction
In many areas of the world the absence of past observational streamflow time
series to calibrate rainfall–runoff models limits the ability to apply such
models reliably to predict streamflow and inform effective water resources
management. Whilst a large and increasing number of regions across the world
are insufficiently gauged , there are also many highly
monitored catchments . Transferring the knowledge gained in
data-rich areas to ungauged catchments – a process known as regionalization
– offers a possible way of overcoming the absence of streamflow observations
in data-scarce regions. Several techniques for transferring information are
reported in the literature (for an overview of different methods used in
continuous streamflow regionalization, see , ,
and ; and for a recent comparative assessment of some of
the most commonly used methods, see ).
A commonly applied approach is to use response signatures (e.g., the runoff
ratio and the base flow index), which can provide insight into the
hydrological functional behavior of a catchment . Response
signatures are calculated from available system output or input–output time
series for numerous gauged catchments with known catchment attributes, i.e.,
physiographic and/or meteorological attributes (drainage area, latitude and
longitude, average annual temperature, average monthly precipitation, etc.).
Subsequently, statistical models relating each response signature to a set of
catchment attributes can be identified. Given the attributes of an ungauged
catchment, the signatures for the ungauged location can then be estimated
using the derived statistical models. Numerous regional models of this type
can be found in the literature e.g.,. These
regionalized signatures can be used to constrain the prior range of
streamflow simulations generated using a preselected rainfall–runoff model
structure and hence restrict the model parameter space
. Advantages of this
approach include (1) the flexibility in the selection of the response
signatures allowing it to be based on the specific parts of the hydrograph
that are of greatest importance for a given application and, if known, on the
dominant hydrological processes of the catchment, and (2) access to readily
available regional models for different signatures in the literature (such as
base flow index from the Hydrology of Soil Types system
and curve number from the United States Department of Agriculture's Soil
Conservation Service soil and land use classification; )
hence eliminating the need to build new regional regression models; in
addition, (3) the relationships between response signatures and catchment and
climatic characteristics are not specific to any rainfall–runoff model nor
to a particular calibration method used in the gauged catchments and are
therefore not obscured by model structural error and can be used to condition
any model.
Different ways of incorporating the regionalized information into a catchment
model have been suggested in the literature. This includes set-theoretic
approaches e.g., and more formal Bayesian
data assimilation frameworks e.g.,. Where probability distributions
characterizing regionalization quality have been estimated, a Bayesian
conditioning procedure is one of the possibilities
. This provides a framework for combining
prior knowledge with the regionalized data and/or other sources of
information e.g., small-scale physics-based knowledge and hydrological
measurements as in, which has the potential to formally
encompass the nature of the errors arising from the regionalization.
Conditioning a rainfall–runoff model on multiple independent signatures would
reflect a spectrum of processes and, in principle, lead to an accurate
prediction of flow time series . However, regionalized
signatures have correlated errors, for example, if the signatures have been
estimated using a common data set of catchment attributes or using the same
hydroclimatic data; in general, the correlations are expected to be
stronger for pairs of signatures that represent similar functional behaviors
of the catchment. This raises the questions of not only how many and which
signatures should be used but also how to avoid double counting of the
information in signatures with correlated error distributions. Previous
applications have tended to use a small number of signatures
e.g., and/or have tended to select
signatures that are considered to be independent e.g.,.
When multiple signatures are used, the correlations between the errors in the
different sources of information are commonly disregarded
e.g.,. To make better use of information in available
sets of signatures, a formal way of combining them, so that information is
neither double-counted nor neglected, is required. Using formal methods to
include autocorrelated data errors in model calibration is well researched
e.g.,; an application of comparable methods in the
regionalization context will allow making more formal and rigorous
assessments of the value of correlated information sources.
Formally, in a Bayesian context, it is necessary to distinguish between
correlated signatures and correlated signature errors. It is the
correlation between the errors that should be accounted for in the likelihood
function to avoid double counting of information. It is possible to have two
highly correlated signatures that are derived from independent information
sources and therefore have uncorrelated errors. In that case, it would be
valid to include both signatures in the likelihood function without
accounting for correlation. This principle is well established when
considering Bayesian calibration to a time series of flow observations, where
flow values are typically strongly autocorrelated – but it is the
observation error autocorrelation that is relevant to the likelihood function
derivation e.g.,. The same principle applies to
adopting signatures as the observations. In the case study below, the
signatures are derived from a common data set using a common approach, so in
practice the signature correlations are comparable to the signature error
correlations; nevertheless, for the sake of formality, we use the term
“signature error correlations” (or “covariance”).
In this paper, we introduce and test a method that considers multiple
regionalized signatures, explicitly accounting for the signature error
correlations. By formally accounting for the error covariance, we hypothesize
that accuracy of flow predictions will generally improve and a greater number
of signatures can usefully be included without introducing avoidable bias
related to the duplication of information. This should allow the modeler to
use all signatures available without having to select, on a more or less
subjective basis, the most relevant (independent) signatures. The objective
is thus to explore how to get fuller value out of a set of regionalized
information than has been achieved in past applications. The method is
applied to a set of 84 United States catchments with a broad range of
hydrometeorological characteristics, obtained from the Model Parameter
Estimation Experiment (MOPEX) data set . The
impact of signature error covariance is assessed using pairs of signatures to
condition a rainfall–runoff model. Along with the real data, synthetic
streamflow data are used to isolate the effect of model structural error.
Further, the model is conditioned on a variable number of regionalized
signatures to evaluate whether an increasing number of signatures is
justifiable when formally accounting for the error covariance.
Method
Bayesian method for signature assimilation
Using a simple least-squares regression, observed signatures of catchments'
functional responses are related to physical and climatic attributes of the
catchments. Assuming that the same catchment attributes are available for an
ungauged location, it is possible to obtain an estimate of the set of
signatures for the location. Further, the parametric distribution of
regression errors can be directly translated to a response signature(s)
likelihood function. The likelihood function can then be used to update the
prior available knowledge about model parameters via Bayes' law, which is
expressed as
p(Θ|s∗,I,M)=L(s(Θ)|s∗,I,M)×p(Θ|I,M)p(s∗|I,M)
where, for one catchment, s∗ represents the regionalized response
signature(s); p(Θ|I,M) is the prior distribution
of parameters Θ for a model structure M and inputs
I; L(s(Θ)|s∗,I,M) is the
likelihood function of the modeled response signature(s)
s(Θ) given s∗, I and M;
p(s∗|I,M) is the marginal density of s∗; and
p(Θ|s∗,I,M) is the posterior distribution
of Θ given s∗, I and M. For the
purpose of this paper, M is selected in advance and considered to be fixed
as it is the common practice in regionalization
studies;, as is I for any one catchment, and so
both these terms are dropped from (Eq. ) for convenience, resulting
in
p(Θ|s∗)=L(s(Θ)|s∗)×p(Θ)p(s∗).
Parameter sets are then sampled from the parameter posterior to allow an
ensemble of rainfall–runoff simulations and a posterior distribution of flow
at each time step to be estimated and evaluated against observed flow. This
can be repeated using different sets of signatures and different assumptions
about their error correlations.
Prior distribution and likelihood function
Prior distribution
To apply Bayes' law (Eq. ), it is necessary to specify the likelihood
function (L(s(Θ)|s∗) in Eq. ) and the
prior distribution (p(Θ) in Eq. ). The prior is
defined so that it reflects our initial lack of knowledge. We follow
and sample sets of signature values from uniform
distributions representing the feasible ranges of signatures. This approach
allows the signatures to be sampled uniformly using a simple amendment to the
commonly applied approach of sampling from uniform parameter priors, which
avoids highly skewed signature priors that have undue influence on the
posterior likelihood. More specifically, N parameter sets (N is equal to
10 000 in our study) are sampled from a uniform distribution using Latin
hypercube sampling, so that probability of each parameter set is 1/N
(10-4 in our study). Subsequently, to provide parameter samples that
correspond to a uniform in signatures' prior distribution, the parameter
probabilities are reweighted see, and used in the
further posterior distribution approximation. This allows accounting for
correlation among the parameters imposed by the uniform in signatures' prior
distribution.
Likelihood function approximation
The likelihood functions are defined using joint distributions of respective
signature errors obtained from the regionalization model. Errors introduced
by the regionalization procedure may come from at least five sources. First,
errors are introduced by the fact that the regression model is estimated
using a specific sample of catchments rather than the entire population;
second, differences may exist between the observed and the true value of the
response signature due, for example, to factors such as the discharge record
length and time period of record used in the computation ;
third, errors are present due to errors in the catchment properties data;
fourth, errors exist due to the incomplete set of catchment properties used
as explanatory variables in the regression equations; and, fifth, they exist
due to the assumed linear regression structure. It is assumed that the total
error model for the regionalized signature(s) s∗ can be estimated
using the following procedure:
Considering all available gauged catchments, stepwise regression is applied
to each signature independently to determine which predictors to include. The predictors are then fixed for the remaining steps.
Considering all available gauged catchments, one catchment is left out and
the remaining are used in the fitting of the regression models for each signature.
The regression models obtained in step 2 are used to estimate the signature values for the omitted catchment.
The error for each signature is calculated for the omitted catchment by comparing the regionalized and observed signature values.
The process is repeated for all catchments.
A parametric joint probability distribution is fitted to all the computed errors.
Furthermore, the errors are tested for independence that allows (approximately) factorizing a joint distribution into a product of marginal distributions.
The resultant error distribution defines the likelihood function L in
Eq. (). The main assumption here is that the potentially complex
nature of errors in the set of signature values can be usefully represented
by the fitted error distributions.
Synthetic case and likelihood functions
To avoid masking the potential value of the regionalized signatures with
model structure and observational errors, a “perfect model” is first
employed. This involves using the preselected rainfall–runoff model and the
observed forcing data to generate the “observed” catchment signatures. The
Nash–Sutcliffe criteria (NSE) optimal parameter set is taken
to generate a “perfect model” streamflow time series for each catchment. To
produce regionalized signature analogues in this case, two types of imposed
errors are introduced to these “observed” signatures. The first error type
is characterized by a range of standard deviations (1, 5, 10 and 20 % of
the signature value range observed over all catchments used in this study)
and a range of inter-signature error correlations (Pearson correlation
coefficients equal to 0, 0.25, 0.50, 0.75, and 0.90). This allows the
sensitivity of the results to the regionalization quality and the
regionalization errors' correlations to be evaluated. The second error type
is set to be equal to the observation-based likelihood function
(Sect. ). These error structures are the likelihoods used in
Eq. () for the synthetic case when flows are generated by a
“perfect model”.
Case study and rainfall–runoff model
Study catchments
A set of 84 medium-sized United States catchments (242 to 8657 km2) from
the MOPEX database , for which a variety of
regional response signature models have been determined in
(namely runoff ratio, base flow index, streamflow
elasticity, slope of slow duration curve, and high pulse count), are used to
test the method proposed in this paper. It has proven difficult to derive
regionalization equations of acceptable prediction quality for all catchments
in the MOPEX data set . This is due to the lack of
descriptive power in the set of available catchment attributes, e.g., the
attributes do not provide satisfactory information about catchment geology.
To isolate the effect of variable geology on the regression equations, the
selected 84 catchments are grouped based on the underlying geology, namely,
middle Paleozoic sedimentary rocks. Use of more catchments from the MOPEX
database would require different regionalization equations due to changing
process controls and would be unnecessary given that the focus of the study
is on signature error correlations in regionalization models. For more
details on the motivation for choosing these specific 84 catchments, see
and .
The 84 catchments are hydrologically varied with a selection of properties
summarized in Table . Daily time series for the period from
1 October 1949 to 30 September 1959 are employed. As highlighted in
, these 10 years of data, representing only a subset of
all the data available, are assumed to be of sufficient length to capture
climatic variability but short enough to avoid effects of long-term climatic
trends .
Summary of general catchment properties and response signatures of
the 84 MOPEX catchments.
Catchment property
Units
Range
Average annual streamflow
(mm yr-1)
208–896
Average annual precipitation
(mm yr-1)
758–1495
Average annual maximum temperature
(∘C)
12–23
Average annual minimum temperature
(∘C)
0–10
Average annual potential evaporation
(mm yr-1)
679–1112
Aridity index*
(-)
0.5–1.2
Average elevation
(m)
176–1056
Runoff ratio
(-)
0.16–0.76
Base flow index
(-)
0.36–0.90
Streamflow elasticity
(-)
0.02–4.34
Slope of flow duration curve
(-)
0.01–0.08
High pulse count
(yr-1)
2.10–120.80
* Long-term ratio of potential
evaporation over precipitation.
Response signatures
Five response signatures are considered: runoff ratio (RR), base flow index
(BFI), streamflow elasticity (SE), slope of flow duration curve (SFDC), and
high pulse count (HPC) (Table ). This specific subset of
signatures is selected to cover a wide range of different qualities of
regionalized information and also to ensure that some signature errors are
largely uncorrelated, whilst others are strongly correlated (see also
Sect. ).
RR reflects the amount of precipitation that becomes streamflow over a
certain area and time. It is determined as the ratio of catchment's outlet
streamflow and catchment average precipitation over the 10 years used in this
study. BFI gives the proportion of streamflow that is considered to be base
flow. A simple one-parameter single-pass digital filter method is used to
derive BFI . SE provides a measure of the sensitivity of
streamflow to changes in precipitation . It is
calculated as a median of the inter-annual variation in total annual
streamflow to the inter-annual variation in total annual precipitation ratios
normalized by the long-term runoff ratio
. SFDC gives an indication of the
streamflow variability and is calculated as the slope of the flow duration
curve between the 33 and 66 % flow exceedance values in a semi-log scale
. HPC reflects aspects of the high flow regime and
catchment flashiness and is calculated as the average number of events per
year that exceed 3 times the median daily flow
.
Rainfall–runoff model choice
The probability distributed moisture (PDM) model , together
with two parallel linear routing stores and a simple snow model
, is selected with two major motivations (a detailed
description of the model is given in Appendix ). First, this
type of model has been shown to have a suitable complexity for modeling
daily rainfall–runoff over a large sample of the MOPEX catchments
. Second, the model has been successfully applied in other
regionalization studies across a wide range of climate and physiographic
conditions, for example, , ,
, , and . Even
though other model structures may be better suited for some specific
catchments, it is prohibitively difficult to vary model structure between
catchments and no single model structure will ever be best for all catchments
. Consequently, the selected
model structure is believed to be a sufficient choice for the purposes of
this paper. Most importantly, the general framework is independent of the
rainfall–runoff model choice.
Posterior distribution and performance assessment
Employing Bayes' law (Eq. ), the rainfall–runoff model is
conditioned on different combinations of signatures: (1) assuming
independence between the signature regionalization errors (setting the
correlation values to zero in the joint probability function); and
(2) accounting for the inter-signature error correlations (using the
estimated covariance in the joint probability function).
Two metrics are used to assess the effectiveness of the parameter
conditioning procedure: (1) the Bayes factor to assess
convergence of the parameter posteriors to known parameter values; (2) the
probabilistic Nash–Sutcliffe efficiency to assess
convergence of the flow ensembles to the observed flows.
The Bayes factor (BF) is defined as the ratio between two marginal
distributions of the data y (e.g., observed streamflow time series)
for two competing hypotheses (H1 and H2) (more detail
is given in Appendix ):
BF=p(y|H1)p(y|H2).
Thus, to test the impact of representing the error correlations, the
hypothesis H1 corresponds to the inter-signature errors being treated as
correlated, while the hypothesis H2 corresponds to the inter-signature
errors assumed to be independent. If the resulting Bayes factor is greater
than 1, there is more support for hypothesis H1, and the inter-signature
error correlation is worth considering.
When using synthetic streamflow data (“perfect model” approach), with the
streamflow time series generated by a preselected parameter set,
p(y|H) in Eq. () can be seen as either the posterior
probability of the known observed streamflow time series under hypothesis H
or the probability of the known parameter set that generated that particular
flow time series under hypothesis H. As in a “perfect model” approach
there is no observational error, p(y|H) is the probability estimated
for the known value of the parameter set that generated the observed
streamflow under each of the hypotheses H1 and H2. Since there is no
known parameter value corresponding to the real data, the application of the
Bayes factor is less useful in this situation. In this case, defining
y as an NSE-optimal parameter set allows an indication of the
relative degree of convergence around the chosen point.
The probabilistic Nash–Sutcliffe efficiency NSEprob is a
probabilistic analogue of the traditional Nash–Sutcliffe efficiency
coefficient , and allows both prediction accuracy and
precision to be summarized by a single statistic (Eq. ),
NSEprob=1-∑t=1T(E[q^t]-qt)2∑t=1T(qt-E[q])2-∑t=1TVar[q^t]∑t=1T(qt-E[q])2,
where qt denotes a set of streamflow observations for time t=1,…,T, E[q]
is the average value for the qt time series, q^t is the
simulated time series of streamflow for time t=1,…,T,
Var[q^t] is the prediction variance at time t,
E[q^t] is the mathematical expectation of the predictions at time
t, and T is the total number of time steps in the sequence. The first
part of Eq. () corresponds to the traditional Nash–Sutcliffe
efficiency coefficient in which expected streamflow values
are considered as predictors. The latter part of the equation represents the
variance, whereby higher predictor variance corresponds to less precise
predictions . An NSEprob of 1 indicates a perfect fit,
i.e., the results are both accurate and precise. The incremental improvement
in the NSEprob can be used to measure the value of adding signatures into the
conditioning or otherwise changing the likelihood function.
For model validation, we use a jack-knife approach (or leave-one-out
strategy), commonly employed in regionalization studies
e.g.,. One catchment at a time is removed as a
test “ungauged” catchment and the remaining gauged catchments are used to
support the regionalization process, including steps 2–6 listed in
Sect. The procedure is repeated for each of the available
catchments.
Results and discussion
Regionalized signature errors and likelihood functions
The regionalization error probability distributions (that define the
likelihoods) are generated following steps 2–6 in Sect. 2.2.2 and are
shown in Fig. . The marginal error distributions, shown on the
Fig. diagonal, are approximated using histograms, and parameters
of normal distributions are fitted using the method of moments. The
univariate Kolmogorov–Smirnov test shows that the marginal distribution
normality cannot be rejected at the 95 % confidence level for each of the
five signatures. The off-diagonal shows the regionalization errors for
different signature pairs (lower off-diagonal), the corresponding correlation
coefficient values and their statistical significance (upper off-diagonal).
The joint error distributions are approximated using multivariate normal
distributions that are fitted using estimates of the marginal normal
distribution parameters and the inter-signature error correlations. These
marginal and joint distributions define the likelihood functions in
Eq. (). Note that Fig. represents the regionalization
errors based on all 84 catchments. Meanwhile, the jack-knife procedure (see
Sect. ) utilized in the performance assessment employs only 83
catchments at a time.
Distribution of individual signature residuals (res) are
approximated as histograms and normal distributions. The scatterplots and
correlation coefficients (ρ) show correlation between the signature
residuals.
The impact of inter-signature error correlations (pairs of signatures)
This section considers the role of inter-signature error correlation on model
parameter estimation when pairs of signatures are used. First, different
imposed error variances and correlations together with synthetic streamflow
data are employed to test the impact of inter-signature error correlation
without the impact of model structural error. Then, the results obtained
using the observation-based error structure, for both synthetic and observed
data streamflow, are analyzed.
Synthetic streamflow data (imposed likelihoods)
Synthetic streamflow data are generated as described in
Sect. and the imposed likelihood functions are defined as
described in Sect. . The imposed likelihoods are considered to
have standard deviations equal to 1, 5, 10, and 20 % of the signature value
range observed over all catchments. A comparison of the imposed error
structures under the different levels of variance and the observed error
structure is given in Table . Furthermore, different
inter-signature error correlations are also tested, namely 0 (linear
independence), 0.25, 0.50, 0.75, and 0.90.
Tested variance values for the data-based and imposed error
structures.
1 % observed
5 % observed
10 % observed
20 % observed
Observed error
signature
signature
signature
signature
structure
ranges
ranges
ranges
ranges
RR residuals
0.0542
0.0052
0.0272
0.0552
0.1092
BFI residuals
0.0442
0.0062
0.0302
0.0602
0.1212
SE residuals
0.6352
0.0232
0.1162
0.2322
0.4642
SFDC residuals
0.0062
0.00052
0.0022
0.0052
0.0102
HPC residuals
10.6872
0.9772
4.8832
9.7672
19.5332
Ten possible pairs of the five response signatures are used in parameter
conditioning, and the median Bayes factor, calculated over the 84 MOPEX
catchments, is calculated for each pair. The Bayes factor (Eq. )
compares the two following hypotheses: H1, the inter-signature error
correlation is to be taken into account, and H2, the errors between the
different sources of information can be assumed independent. The Bayes factor
is found to be relatively insensitive to the selection of response signature
pairs (Kruskal–Wallis test). Table summarizes the 95 % pooled
confidence intervals for the median Bayes factor across all catchments and
across all 10 signature pairs, for each choice of the likelihood (i.e., 20
likelihoods). This provides reference values indicative of the error
interdependency importance in model regionalization depending on the
signature pair correlations and marginal distribution variances. As it would
be expected, the median Bayes factor is equal to 1 when signatures errors are
not correlated (i.e., ρ=0). However, as correlations between signatures
errors increase the median Bayes factor increases noticeably. This suggests
that considering error correlations allocates higher likelihoods to parameter
sets that capture a considered signature pair. Furthermore, the results shown
in Table also imply that the median Bayes factor is relatively
insensitive to the precision with which the signatures are regionalized.
Reference table showing the 95 % confidence interval for the
median Bayes factor. The correlation coefficient ρ and the standard
deviation of the marginal distributions σ are shown.
σ
1 %
5 %
10 %
20 %
0
1
1
1
1
0.25
1.01–1.03
1.03–1.04
1.02–1.04
1.04–1.05
ρ
0.50
1.09–1.15
1.16–1.19
1.14–1.17
1.14–1.18
0.75
1.41–1.51
1.50–1.57
1.45–1.53
1.40–1.49
0.90
1.94–2.11
2.11–2.32
2.12–2.26
2.20–2.34
Synthetic and observed streamflow data (observation-based likelihoods)
Figure shows the distribution of the Bayes factor values obtained
across the 84 catchments for each of the 10 possible different pairs of
signatures, when the observation-based error structure is used for each
catchment. Figure a shows the results for the observed streamflow
data with regionalized signatures calculated from the derived regressions;
Fig. b shows the results for the synthetic streamflow data with
regionalized signatures calculated by adding noise to the exact signature
values. The Tukey boxplots in red correspond to pairs of signatures whose
errors are statistically significantly correlated (see Fig. ). The
upper whisker represents the upper quartile plus 1.5 times the
interquartile range and the lower whisker represents the lower quartile
minus 1.5 times the interquartile range. The matrix below
Fig. b shows the pairs of signatures used.
The Bayes factor for the 10 pairs of signatures over the 84
catchments when the observation-based error structure is used with
(a) observed streamflow data, (b) synthetic streamflow
data. The upper whisker represents the upper quartile plus 1.5
times the interquartile range and the lower whisker represents the lower
quartile minus 1.5 times the interquartile range. The dashed line
represents BF = 1.
The signature pair [SFDC, HPC] shows the strongest correlation between errors
(ρ=0.65, Fig. ). A likelihood function with a standard
deviation equal to 10 % of the observed signature ranges and ρ=0.75 in
Table is comparable to the observation-based likelihood of the
pair [SFDC, HPC] (Table ), with Table indicating
[1.45,1.53] as a 95 % confidence interval for the median Bayes factor.
However, a median Bayes factor of 2.17 is obtained for the observed
streamflow data (Fig. a). Similar differences are found for the
other pairs of signatures, although the comparison with the reference table
(Table ) becomes challenging, as the individual signatures have
not been regionalized necessarily with similar quality. On the other hand,
Fig. b shows that the Bayes factors for the synthetic study (when
there is no model structural error) are consistent with the values provided
in the look-up Table . The difference between the median Bayes
factor for the two cases is likely to be caused by the model structure error,
or may be related to the location of the NSE-optimal in the parameter space.
Nevertheless, it is clear from Fig. that those pairs of signatures
whose errors are significantly correlated (i.e., [SFDC, HPC], [BFI, HPC],
[BFI, SFDC] and [BFI, SE]) have wider interquartile ranges. Furthermore, the
pair of signatures with the strongest correlation between errors [SFDC, HPC]
presents the greatest interquartile range. Therefore the inclusion of
significant correlations in the likelihood function matters, but whether or
not it is beneficial to conditioning the parameters seems to depend on the
interplay between model structure error, parameter space and likelihood
function. Only strong correlations (as in the [SFDC, HPC] case) can be
expected to result in a median Bayes factor clearly above 1.
The impact of inter-signature error correlations (multiple signatures)
Multiple signatures are used for parameter constraining and flow prediction.
The information value of multiple signatures and its dependence on
inter-signature error correlations is explored in this section.
Synthetic streamflow data (observation-based likelihood)
Figure shows Bayes factors derived for the synthetic streamflow
data (generated using the NSE-optimal parameter set) when the
observation-based likelihood is used. The Bayes factor considers p(.|H2)
to be the prior parameter distribution and p(.|H1) to be one of the
parameter posteriors that includes or ignores the inter-signature error
correlations. Figure summarizes the variability in the Bayes
factor for the different combinations of signatures for all 84 catchments.
Boxplots are color coded by the total number of signatures combined, when the
inter-signatures error correlation is considered in the likelihood function
definition. The grey dashed boxplots correspond to the results obtained
assuming that the inter-signature errors are independent when defining the
likelihood function. Although the colored boxplots visually seem to have
higher values than the grey dashed boxplots, these differences are not
statistically significant at a 95 % confidence level (Kolmogorov–Smirnov
two-sided tests).
Boxplots representing the distribution of the Bayes factor for each
combination of signatures for synthetic streamflow data. The colored boxplots
correspond to the results obtained when inter-signature error correlations
are considered in the likelihood function, whereas the grey dashed boxplots
correspond to the results obtained assuming that the inter-signature errors
are independent.
To better evaluate whether the incorporation of additional sources of
information improves parameter identification, one-sided Kolmogorov–Smirnov
tests are applied between any combination of certain signatures (e.g., [SE,
SFDC]) and any other combination that contains the same signatures and a new
one (e.g., [SE, SFDC, HPC]). It is found that adding more signatures improves
parameter identification in 82.5 % of the cases (66 out of 80 cases) at a
95 % confidence level.
Figure summarizes the variability in the analog Nash–Sutcliffe
efficiency measure NSEprob for different combinations of signatures for all
84 catchments. The colored boxplots correspond to the results obtained when
the inter-signature error correlations are considered in the likelihood
definition and the grey dashed boxplots correspond to the results when the
inter-signature errors are assumed to be independent. There is no visual or
statistical (two-sided Kolmogorov–Smirnov tests) difference between the
colored boxplots and the grey dashed boxplots in Fig. . Moreover,
visually, adding more response signatures seems to improve streamflow
predictions in terms of accuracy and precision when no model structure error
exists. However, only in 59 % of the cases (47 out of 80 cases) more
signatures contribute to improved streamflow predictions at a 95 %
confidence level (one-sided Kolmogorov–Smirnov test). The other 33 cases
always involve the inclusion of the most poorly regionalized signatures (with
the highest variance from the five regionalized signatures) – SE, SFDC, or HPC
– as additional sources of information (see Table ).
Boxplots representing the distribution of NSEprob values for each
combination of signatures for synthetic streamflow data. The colored boxplots
correspond to the results obtained when inter-signature error correlations
are considered in the likelihood function, whereas the grey dashed boxplots
correspond to the results obtained assuming that the inter-signature errors
are independent.
It is worth noting that very similar results (not shown here) are obtained
when instead of regionalized signatures, “observed” signatures are used but
with the same error derived from regionalization. This suggests that the
uncertainty around the regionalized signatures values, as well as signature
information content, are the key factors leading to the results shown in
Fig. .
Observed streamflow data (observation-based likelihood)
Figure shows the results when the same methodology as in the
Sect. is applied using the observed streamflow data. As in
the synthetic streamflow case, the differences between the Bayes factor
distributions when inter-signature error correlations are considered and when
inter-signature errors are assumed to be independent are not statistically
significant at a 95 % confidence level (Kolmogorov–Smirnov two-sided
tests).
Boxplots representing the distribution of the Bayes factor for each
combination of signatures for observed streamflow data. The colored boxplots
correspond to the results obtained when inter-signature error correlations
are considered in the likelihood function, whereas the grey dashed boxplots
correspond to the results obtained assuming that the inter-signature errors
are independent.
Further, by comparing Fig. with Fig. , it becomes clear
that the signatures contribute less information and there is a smaller
increase in performance as more signatures are added. It is found that adding
more signatures tends to improve parameter identification only in half of the
cases when compared to the synthetic streamflow case at a 95 % confidence
level (42.5 vs. 82.5 % in the synthetic streamflow case).
Furthermore, and contrastingly to the case where no structural error exists,
in five situations, adding more signatures contributes to a decrease in
performance. These five cases always involve adding either SFDC or HPC as an
additional source of information. This performance deterioration can be
attributed to model structure and observational error. Overall, a
statistically significant drop in performance with regard to the Bayes factor
is observed most of the time when model structural error is present.
Figure presents the results in terms of NSEprob using the observed
streamflow data. As in the synthetic study in Sect. , there is
no statistically significant difference at a 95 % confidence difference
between the NSEprob distributions when the inter-signature error correlations
are considered and when the errors are treated independently
(Kolmogorov–Smirnov two-sided tests).
Boxplots representing the distribution of NSEprob values for each
combination of signatures for observed streamflow data. The colored boxplots
correspond to the results obtained when inter-signature error correlations
are considered in the likelihood function, whereas the grey dashed boxplots
correspond to the results obtained assuming that the inter-signature errors
are independent.
Figure shows that better results in terms of NSEprob are not
necessarily achieved when all five signatures are used simultaneously. It is
found that adding more signatures tends to improve parameter identification
only in 36 % of the cases at a 95 % confidence level (compared to 59 %
when there is no model structure error). Furthermore, and contrasting the
case where no model structure error exists, in two situations, adding more
signatures may contribute to a decrease in performance (when we start with
[RR, BFI] and add HPC, and when we start with [RR, BFI] and add SFDC). This
might be due to regionalization biases in SFDC and HPC and/or due to the
inability of the PDM model to maintain a satisfactory overall performance
when conditioned on high peak flow and medium flow information. This negative
impact is not observed when synthetic streamflow data are used
(Fig. ), indicating that the decrease in performance may be due to
model structural deficiencies. Moreover, a statistically significant drop in
performance with regard to NSEprob is observed most of the time when there is
model structural error.
In summary, unless there is no model structural error, an all-round
performance improvement is not guaranteed by adding more signatures.
Furthermore, model structure uncertainty seems to have a much bigger effect
on the performance than the explicit inclusion of the inter-signature error
correlations.
Limitations and applicability
The main feature of the method suggested in this paper lies in the
possibility of allowing a large number of signatures to be added to the
conditioning process, without worrying about double counting of information
or degree of uncertainty in signature estimates and avoiding subjective
decisions about removal of possibly non-independent information. Although the
proposed framework can be applied to any number of signatures, the limited
sample size (i.e., number of gauged catchments available) can have an impact
on the definition of the likelihood distribution. For this specific study 83
samples were available to define that distribution. When a single response
signature is used to condition the hydrological model, this sample size is
likely to be sufficient to confidently judge whether the normal distribution
assumption is sufficient. However, when moving to multidimensional problems,
in which various signatures may be used simultaneously to condition the
hydrological model, it is increasingly difficult to judge the adequacy of any
multivariate parametric distribution and to judge which catchments are
outliers. This implies that as more signatures are used simultaneously in the
conditioning of the hydrological model, the more gauged catchments should be
used to define the likelihood function. As stressed by ,
large samples are of great importance to support statistical regionalization
of uncertainty estimates and this is particularly the case if dependencies
between information sources are to be specified.
While the work presented in this paper addresses a number of issues
associated with model regionalization, it is important to highlight some
additional areas for future research. An important source of uncertainty
comes from model structure error . The
conditioning framework suggested here is independent of the selected model
and, in principle, Figs. and could be created by using
the model structure that is considered suitable for each catchment rather
than using a model structure that we consider good for generalizing. Further
research is needed to diagnose the relative importance of different model
structures in various climate regimes and for different catchment
characteristics . This is crucial to both
identifying the most appropriate model structure for an ungauged location and
quantifying the uncertainty in the model structure that should be integrated
into the likelihood, thus allowing virtually any model choice. Similarly,
other sources of uncertainty, namely observational error (e.g., rainfall
error), should ideally be evaluated and integrated into the likelihood
function. By accounting for all the important sources of uncertainty, further
insight should be achieved into the information value of sets of signatures
and the value of including their dependencies in the likelihood function.
Some of the results presented may be sensitive to the response signatures
used. The relationship between value of signatures and catchment type remains
ambiguous and an interesting aspect for posterior evaluation would be how the
value of signatures depends on catchment type. Other aspects that are worth
further research include whether a similar framework could be applied to
different types of information source, e.g., can some discharge measurements
be added into the model conditioning process? While
suggests a framework capable of combining multiple sources of knowledge,
namely physically based information, regionalized signatures and spot
observations to identify parameters for models of ungauged catchments, the
errors between them were assumed to be independent in their case study. A
combination of the framework suggested by and the method
proposed in this paper may be the way forward to maximizing the value of the
available information within a framework of uncertainty reduction.
Conclusions
Uncertainty in streamflow estimation in ungauged catchments
originates not only from the traditional sources of error generally
identified in rainfall–runoff modeling (i.e., model structural, parameter, and
data errors) but also by errors introduced by the transposition of
information from data-rich areas and use of this information to condition
model simulations. To identify which and how many types of signatures can
usefully be included in model conditioning, it is critical to understand the
effects of all these uncertainties. Moreover, when multiple signatures are
used simultaneously to condition model simulations, inter-signature error
dependencies may also introduce uncertainty and affect decisions about the
value of information. While error and uncertainty analyses are quite common
in regionalization studies, the question of how much information can be taken
from a set of uncertain signatures and determining how many and which
signatures should be used given their error dependencies has not been
extensively studied.
The method suggested in this paper allows the specification of a signature
error structure. A common reason for not including large numbers of
signatures in regionalization studies is the potential for underestimation
of uncertainty due to duplication of information. This study helps to justify
the inclusion of larger sets of signatures in the regionalization procedure
if their error correlations are formally accounted for and thus enables a
more complete use of all available information. The results show that adding
response signatures to constrain the hydrological model, while accounting for
inter-signature error correlations, can contribute to a stronger
identification of the optimum parameter set when the error correlations
between different sources of information are strong. Furthermore, the results
show that assuming independency of errors does not result in significant
deterioration in model performance, unless the error correlation is very
strong. The results also show that the effect of error correlations is likely
to be overwhelmed by model structure and observation errors. The method
suggested here can therefore become more relevant if observational and
structural errors are reduced. In addition, it is illustrated that using more
signatures, with and without considering their error correlations, may lead
to deterioration in performance. In our case, there were particular problems
when adding the slope of the flow duration curve and/or the high pulse count.
As this is likely to be specific to the rainfall–runoff model used, the
selected performance criteria and the set of catchments, it is recommended
that the misinformative information sources are identified as part of any
regionalization study, in a similar manner as has been done here.