The importance of parameterization when simulating the hydrologic response of vegetative land-cover change

Computer models of hydrologic systems are frequently used to investigate the hydrologic response of landcover change. If the modeling results are used to inform resource-management decisions, then providing robust estimates of uncertainty in the simulated response is an important consideration. Here we examine the importance of parameterization, a necessarily subjective process, on uncertainty estimates of the simulated hydrologic response of landcover change. Specifically, we applied the soil water assessment tool (SWAT) model to a 1.4 km2 watershed in southern Texas to investigate the simulated hydrologic response of brush management (the mechanical removal of woody plants), a discrete land-cover change. The watershed was instrumented before and after brush-management activities were undertaken, and estimates of precipitation, streamflow, and evapotranspiration (ET) are available; these data were used to condition and verify the model. The role of parameterization in brush-management simulation was evaluated by constructing two models, one with 12 adjustable parameters (reduced parameterization) and one with 1305 adjustable parameters (full parameterization). Both models were subjected to global sensitivity analysis as well as Monte Carlo and generalized likelihood uncertainty estimation (GLUE) conditioning to identify important model inputs and to estimate uncertainty in several quantities of interest related to brush management. Many realizations from both parameterizations were identified as “behavioral” in that they reproduce daily mean streamflow acceptably well according to Nash– Sutcliffe model efficiency coefficient, percent bias, and coefficient of determination. However, the total volumetric ET difference resulting from simulated brush management remains highly uncertain after conditioning to daily mean streamflow, indicating that streamflow data alone are not sufficient to inform the model inputs that influence the simulated outcomes of brush management the most. Additionally, the reduced-parameterization model grossly underestimates uncertainty in the total volumetric ET difference compared to the full-parameterization model; total volumetric ET difference is a primary metric for evaluating the outcomes of brush management. The failure of the reduced-parameterization model to provide robust uncertainty estimates demonstrates the importance of parameterization when attempting to quantify uncertainty in land-cover change simulations.


Introduction
An important use for computer models of hydrologic systems is simulation of the hydrologic response of land-cover change (Fohrer et al., 2001;DeFries and Eshleman, 2004); many modeling analyses have been undertaken in attempts to better understand how changes in land cover may change the timing and quantity of runoff, recharge, and evapotranspiration (e.g., Schilling et al., 2014;Ahn and Merwade, 2017;Chu et al., 2010). Given the uncertainties that exist in nearly every hydrologic model input dataset, the potential exists for the simulated outcomes to be highly uncertain, even after conditioning to streamflow data. Given this potential uncertainty in model outcomes, quantifying uncertainty in the simulated results of land-cover change is an important consideration, especially if simulation results are to be used in resource-management decision making.
Previous research has shown that the subjective process of selecting which model inputs to treat as uncertain (e.g., parameterization) may affect uncertainty estimates in model outcomes (White et al., 2014). Herein, parameterization refers to the subjective and necessary process of selecting uncertain model inputs to treat as adjustable in the conditioning process. We investigate how parameterization may affect the uncertainty quantification process when simulating a discrete, vegetative land-cover change, the mechanical removal of woody plants.
Woody-plant encroachment into grasslands has been a worldwide phenomena in the past 150 years (Archer et al., 2011). This encroachment has several ramifications for the ecosystem, including changes to the hydrologic function and the response of the surface-water basins (Archer et al., 2011). Woody species are commonly thought to consume a larger quantity of water (by transpiration) in comparison to native grasses (Tennesen, 2008). By removing the woody species and allowing native grasses to reestablish in the area (commonly referred to as "brush management"), changes in the hydrology in the watershed might occur (US Department of Agriculture, 2009).
Many hydrologic modeling analyses have been completed to evaluate the feasibility of applying brush management in order to decrease the quantity of water transpired within a given watershed. (Ben Wu et al., 2001;Lemberg et al., 2002;Brown and Raines, 2002;Afinowicz et al., 2005;Bumgarner and Thompson, 2012;Harwell et al., 2016). However, to date (2017), very few, if any, of the modeling-based brushmanagement feasibility studies have included uncertainty estimation in the simulated hydrologic response of brush management, even though substantial uncertainty in other applications of the soil water assessment tool (SWAT) model have been reported .
To demonstrate the utility of including uncertainty estimation and to investigate how parameterization may affect the reliability of a model to resolve the hydrologic outcomes of simulated land-cover changes, such as brush management, the SWAT (Arnold et al., 1998) was applied to a 1.4 km 2 watershed in southern Texas. The same watershed assessed in this study was subject of a previous investigation in which multiple types of data (precipitation, streamflow, and evapotranspiration -ET) were collected (Banta and Slattery, 2011). The objectives of our study are to (1) assess the reliability of a computer model to simulate pre-and posttreatment water-budget components in the context of uncertainty and (2) evaluate the role of model parameterization in the uncertainty estimation process by investigating the number of model inputs that influence the important model outputs.

Hydrologic setting
The brush-management simulation described herein is applied to a 1.4 km 2 watershed in the Honeycreek State Natural Area in southern Texas (Fig. 1). For a complete description of the study area, see Banta and Slattery (2011). Note the watershed analyzed in this study is referred to as the "treatment watershed" in Banta and Slattery (2011).
According to Banta and Slattery (2011), long-term average precipitation near the watershed is 863.6 mm year −1 and is equally distributed throughout the calendar year. The watershed generally has gentle slopes (less than 5 %), with steeper slopes in the stream channel ravines. Clay and clay loam soils overlie the Trinity aquifer outcrop in the watershed; the Trinity aquifer is a regional karst aquifer system (Banta and Slattery, 2011). Before brush management was implemented, the watershed was largely dominated by Juniperus ashei (ashe juniper). Approximately 40 % of the ashe juniper land cover was mechanically cleared from the watershed during calendar year 2004 (Homer et al., 2007). The watershed configuration before removal of 40 % of the ashe juniper is referred to as the "pretreatment" configuration. Following ashe juniper removal, the land returned to a native rangeland land-cover type (referred to hereinafter as the "posttreatment" configuration).

Model construction
The SWAT model was used to simulate the hydrologic response of the watershed, including the effects of brush management. Specifically, a SWAT2012 (Arnold et al., 2012b, a) model of the watershed was built using the ArcSWAT tool (Winchell et al., 2007). The resulting model files were incorporated into the model-independent framework of PEST++ V3 (Welter et al., 2015) to facilitate programmatic interaction with the model so that any model input quantity could be treated as a parameter and a variety of model outputs, including derived and processed quantities, can be included in the modeling analysis.
These three datasets were used within the ArcSWAT tool to find unique land slope, soil, and land-cover combinations across the watershed. These unique combinations became HRUs in the SWAT model. The NED digital elevation model for the watershed was smoothed with a 4-pixel-width averaging kernel to remove apparent artifacts.
As part of the previous study that evaluated the effects of brush management at the Honey Creek State Natural Area (Banta and Slattery, 2011), daily total precipitation, ET, and  (Winchell et al., 2007 daily mean streamflow were measured during 2001-2010 (Fig. 2). The methods used to collect the input datasets are described in Banta and Slattery (2011). The precipitation data were used as inputs to the SWAT model, whereas the ET and streamflow data were used for conditioning and model evaluation (described below). Because the SWAT model is sensitive to precipitation intensity, the original 5 min measurements from four precipitation measurement stations in the study area were combined via arithmetic averaging to develop the precipitation input dataset -the averaging was needed to account for missing data caused by instrument issues in order to form a complete precipitation dataset. The National Centers for Environmental Prediction (NCEP) Climate Forecast System Reanalysis (CFSR; Saha et al., 2014) Global Weather Database was used in the SWAT simulation as the input for weather data when on-site precipitation data were not available (Banta and Slattery, 2011). To account for errors induced by averaging precipitation data and the use of lower-resolution NCEP precipitation data, we treat precipitation as uncertain; the treatment of model inputs as uncertain is discussed in detail in Sect. 2.4.

ArcSWAT
The ArcSWAT tool (Winchell et al., 2007) was used with the previously described datasets to construct a SWAT2012 model of the watershed. Surface runoff is simulated with SWAT using the Green-Ampt excess-rainfall method (Mein and Larson, 1973;Jeong et al., 2010). The NLCD 2001 (Homer et al., 2007) land-cover data were modified so that areas of mixed brush-rangeland within the watershed were reclassified as rangeland, which is consistent with site-specific knowledge (Banta and Slattery, 2011).
The application of the ArcSWAT tool with the previously described datasets resulted in a model with a single subbasin covering the 1.4 km 2 watershed study area with 47 distinct HRUs (Fig. 1). A summary of the HRU characteristics is included in Table S1 of the Supplement; the detailed HRU characteristics obtained by applying the ArcSWAT tool are included in the associated data release (White et al., 2017).

Model configurations
The modeling analysis described herein includes two specific simulation periods that correspond to the pretreatment and posttreatment configurations: The conditioning period and forecast period models simulate the years 2001 and 2004, respectively; the initial year of simulation for each model is used as a model warm-up period to remove any transient artifacts from initial conditions.
In a typical modeling feasibility study, the model is constructed and conditioned to pretreatment (conditioning period) system states, then forecasts are made using the model related to how simulated brush management will affect the hydrology within the watershed.
Here, two distinct SWAT models were constructed. The first SWAT model simulated the pretreatment configuration and is hereinafter referred to as the "pretreatment" model. The second SWAT model simulated the posttreatment configuration and is hereinafter referred to as the "posttreatment" model. The only difference between the two SWAT models are specific inputs to HRUs 18, 20, 22, and 32, which represented the area of watershed that was converted from evergreen forest (e.g., ashe juniper) to rangeland. Modifications to the input files for the listed HRUs were as follows (herein, references to specific SWAT input variables are shown in all caps): maximum canopy interception -the CANMX variable in the .HRU input files plant growth cycle -the PLANT_ID and HEAT_UNITS variables in the .MGT input files.
In this study, brush management is simulated by modifying the maximum canopy storage and inputs that control the simulated growth cycle for a representative area of the subbasin from evergreen forest to rangeland, because this required few assumptions and allowed injection of the desired uncertainty into the simulation workflow. We modified the maximum canopy storage and the plant growth aspects of HRUs 18, 20, 22, and 32 because these inputs directly affect the available precipitation for partitioning and simulated ET processes, respectively, whereas plant-growth variables affect the timing and intensity of simulated ET processes related to the annual plant-growth cycle. In the pretreatment model, these model inputs were specified to represent ashe juniper land cover for HRUs 18, 20, 22, and 32, Table 1. Summary of parameters used in the reduced parameterization. These 12 inputs were selected from Table 1 in Arnold et al. (2012b) and are adjusted on the subbasin scale. whereas in the posttreatment model, these inputs for HRUs 18, 20, 22, and 32 were specified to represent rangeland land cover, effectively capturing the change in the simulated inputs that corresponds to the brush-management operations that occurred during 2004. See the SWAT theory (Neitsch et al., 2011) and input-output documentation (Arnold et al., 2012a) for more information on the model inputs listed in the .HRU and .MGT files.

Parameterization
Parameterization is a critical part of any modeling analysis and has received considerable attention in the literature (Abbaspour et al., 2004;Romanowicz et al., 2005;Sexton et al., 2011;Zhenyao et al., 2013;Migliaccio and Chaubey, 2008;Cibin et al., 2010;Gitau and Chaubey, 2010;Du et al., 2013;Malone et al., 2015;Zhang et al., 2016). In this analysis, we investigated two parameterization designs: reduced parameterization uses the 12 model inputs listed on Table 1 of Arnold et al. (2012b) to represent model input uncertainty. These 12 model inputs are the most cited SWAT model inputs treated as parameters when simulating surface-water runoff and base-flow processes (Arnold et al., 2012b). The reduced parameterization was, therefore, representative of many SWAT modeling analyses in the literature. For the reduced parameterization model, inputs were adjusted on the watershed scale -that is, all 47 HRUs receive the same value for each of these 12 model inputs (Table 1).
full parameterization used 1305 model inputs. It builds on the 12 parameters of the reduced parameterization by adding unique multiplier parameters on the HRU scale for each of the 12 parameters in Table 1, and also includes many other model inputs that are not typically adjusted, although they are still uncertain, such as soil properties, and inputs that govern the simulation of plant growth, such as leaf area index (LAI) variables. The full parameterization also includes annual quartile precipitation multipliers to account for uncertainty and potential bias in precipitation estimates (Leta et al., 2015;Renard et al., 2011;Kavetski et al., 2006;Kuczera et al., 2006). See Table S1 of the Supplement for a summary of the full parameterization and the associated data release (White et al., 2017) for a complete description of the full parameterization.
These two parameterizations represent different approaches to hydrologic modeling. From a computational standpoint, the reduced parameterization is more desirable, whereas the full parameterization offers the opportunity for a more complete expression of model input uncertainty.
The SWAT input CANMX is of particular importance in simulating brush management because it controls how much precipitation is available for partitioning, and it is directly affected by land-cover changes. Therefore, CANMX potentially exhibits a strong control of the simulated outcomes of brush management. CANMX is not treated as uncertain in the reduced parameterization as it is not commonly treated as adjustable (Arnold et al., 2012b). However, CANMX is included in the full parameterization and is parameterized as follows (herein, references to specific parameters are shown in italics): the parameter canmx_v represents the maximum canopy storage for evergreen forest land-cover-type HRUs; the parameter canmxfac_07 represents the portion of canmx_v that is applied to deciduous forest land-covertype HRUs; and the parameter canmxfac_15 represents the portion of canmx_v that is applied to rangeland land-cover-type HRUs.
In this way, we can incorporate uncertainty in the values of CANMX for all three land-cover types while also enforcing the relations we expect for the maximum canopy storage between the land-cover types. This treatment for CANMX allows both the pre-and posttreatment models to receive the same parameter values for the same land-cover types. Because HRUs 18, 20, 22, and 32 switch from evergreen land cover to rangeland land cover, the CANMX values assigned to these HRUs is in harmony with the CANMX values assigned to other HRUs. The HRU-scale multipliers, named canmx_XX, where XX is the HRU number, still account for HRU-scale variability in CANMX for HRUs of the same land-cover type. In the reduced parameterization, the parameters canmx_v, canmxfac_07, and canmxfac_15 are specified values of 13.0 mm, 0.625×13.0 mm (8.13 mm) and 0.25 × 13.0 mm (3.25 mm), respectively, which corresponds to the midpoint of the respective parameter ranges.
The upper and lower bounds of each parameter were defined using a combination of literature values (Abbaspour, 2015;Douglas-Mankin et al., 2010) and expert knowledge. Collectively, the upper and lower bounds of each parameter form a multivariate uniform distribution (hereinafter referred to as the "Prior"). Conceptually, the Prior is the distribution of "acceptable" parameter values based on hydrologic system knowledge. The upper and lower bounds of each parameter are summarized in Table S1 of the Supplement; the upper and lower bounds of the reduced parameterization are distilled on Table 1.

Model interface
Both the pre-and posttreatment SWAT models must be evaluated repeatedly to simulate hydrologic outcomes of brush management and evaluate the importance of parameterization in said outcomes. To accomplish this repeated evaluation, a model-independent interface to SWAT was constructed. This interface facilitated the translation of parameter values into SWAT model input files, the execution of both the pre-and posttreatment SWAT models, and the postprocessing of SWAT model output into quantities of interest.
To translate parameter values to SWAT model input files, parameters were assigned two characteristics: 1. Scale: a given parameter is either subbasin-scale or HRU-scale. Subbasin-scale parameters are applied to all 47 HRUs, whereas an HRU-scale parameter applies only to a specific HRU.
2. Type: a given parameter is either a multiplier-type parameter or a value-type parameter. Multiplier-type parameters are treated as scaling factors against the original SWAT model input variable(s), whereas value-type parameters replace the original SWAT model input variables(s).
The following steps represent a single model evaluation in the model interface: 1. Construct two"base" tables of HRU-scale inputs where the columns are the SWAT model input names and the rows are the 47 HRUs (one table for the pretreatment  model and one table for the posttreatment model). Populate these tables with the base input values from the ArcSWAT tool.
2. For each value-type, subbasin-scale parameter, replace the values in the base tables for each corresponding column with the specified parameter value, assigning all HRUs the same value.
3. For each multiplier-type, subbasin-scale parameter, multiply the corresponding column of the base tables by the specified parameter value, scaling all HRUs by the same value.
4. Apply canmx_v, canmxfac_07 and canmxfac_15 parameters to the CANMX column of both base tables according to the land-cover type of each HRU using the previously described relation between these parameters.
5. For each multiplier-type, HRU-scale parameter, multiply the corresponding row-column location in the base tables by the specified parameter value, scaling only a single entry in the table.
6. Translate the base tables into the appropriate SWAT input files for both the pre-and posttreatment models. 11. Postprocess both model runs to formulate brushmanagement quantities of interest and conditioning measures (described in Sect. 2.6).
The forward run process was completed many times as part of both the global sensitivity analysis and the uncertainty analysis (described in Sect. 2.6). For the reduced parameterization, the HRU-scale parameters, precipitation parameters, and plant growth parameters were each assigned a value of 1.0, effectively removing the influence of these parameters on the model outputs.

Evaluation of brush-management simulations
We used uncertainty quantification techniques to investigate how well the previously described SWAT models simulate the effects of brush management on long-term water-budget components. Specifically, after applying the global sensitivity analysis (GSA) method of Morris (Morris, 1991) (hereinafter referred to as the "method of Morris"), we used Monte Carlo (MC) analysis in conjunction with Generalized Likelihood Uncertainty Estimation (GLUE; Beven and Binley, 1992) to construct prior and behavioral distributions for several model outputs that are important to simulating the outcomes of brush management, which we term quantities of interest (QOIs).
The work of Banta and Slattery (2011) includes daily mean streamflow and daily total ET for the watershed during the forecast (posttreatment) period, which means measured values for QOI-1 through QOI-4 are available. Posttreatment streamflow measurements as well as pre-and posttreatment ET measurements are not available in most real-world applications of modeling to support brush-management activities. Therefore, we treat QOI-1 through QOI-4 as verification measures to check how well the model reproduces long-term water-budget components, measures that are related to simulating the feasibility of brush management. QOI-5 is the primary quantity we use to evaluate the effectiveness of brush management: how does the simulated long-term volumetric ET change as a result of brush management? QOI-5 is simulated by running the pre-and posttreatment models for 2004 to 2010 and summing the differences in simulated ET between the two simulations.

Monte Carlo and GLUE
Monte Carlo analysis (Tarantola, 2005) was used to investigate the effects of SWAT model input uncertainty on brush-management QOIs. MC was chosen because it employs few assumptions and because the forward model run time is relatively short.
To perform the MC analysis, a 1 million parameter set ensemble was drawn from the Prior for each of 1305 elements of the full parameterization using the python module pyEMU (White et al., 2016). Note the upper and lower bounds of each parameter are provided in the data release (White et al., 2017) and are summarized in Table S1 in the Supplement. Once the prior parameter ensemble was constructed, the SWEEP utility of the PEST++ software suite (Welter et al., 2015) was used to run the pre-and posttreatment SWAT models for each of the 1 million realized parameter sets in a distributed, parallel environment using the steps described in Sect. 2.5. The result of this process yielded 1 million values for each of the conditioning measures and brush-management QOIs.
The reduced parameterization was evaluated in a similar fashion. The full-parameterization prior ensemble was modified so that the value of each parameter that was not included in reduced parameterization was fixed at the value representing the midpoint of the parameter's range. In this way, parameters not included in the full parameterization were treated as if they were not in the analysis and are instead "fixed" or "known" model inputs -just as they would be treated in a modeling analysis that only adjusted the 12 inputs of the reduced parameterization. Whereas the midpoint values of the fixed parameters may not be "best" in the sense that they reduce model-to-measurement misfit, they are nonetheless centered within the range of plausibility as described by the Prior.
The reduced-parameterization prior ensemble was also evaluated using the SWEEP utility in a distributed parallel environment, yielding 1 million values for each of the conditioning measures and brush-management QOIs.
Once the prior ensembles of both the reduced and full parameterizations were evaluated, the GLUE method of Beven and Binley (1992) was used to condition the prior ensembles. The GLUE method was selected because it accommodates a subjective likelihood function, which allows the conditioning process to be flexible and can simultaneously accommodate several criteria (Beven and Binley, 1992). In this study, the behavioral parameter ensembles are a subset of prior parameter ensembles which meet three criteria (herein referred to as conditioning measures). Following Moriasi et al. (2007), we selected the following conditioning measures, which are based on daily mean streamflow, to form the behavioral ensemble: CM-1 conditioning-period (pretreatment) Nash-Sutcliffe model efficiency coefficient (NSE) > 0.75 CM-2 conditioning-period (pretreatment) percent bias < 5 % CM-3 conditioning-period (pretreatment) coefficient of determination (R 2 ) > 0.85.
These conditioning measures are widely used to judge a hydrologic model's ability to reproduce observed daily mean streamflow (Moriasi et al., 2007). Briefly, NSE is a statistic that determines the relative magnitude of simulated residual variance to the observed variance (Nash and Sutcliffe, 1970). Percent bias measures the tendency of the model to systematically over-or undersimulate the observed data, whereas the coefficient of determination measures the colinearity between simulated and observed pairs. By using all three of these conditioning measures simultaneously, the parameter realizations that "best" reproduce different facets of the observed streamflow data are identified.
Realizations in each of the prior ensembles that satisfied all three of conditioning measures are designated as "behavioral" and, taken together, comprise the reduced and full parameterization behavioral ensembles, respectively. These behavioral ensembles represent parameter realizations that respect the Prior but that also reproduce daily mean streamflow acceptably well according to the three conditioning measures. That is, each parameter realization in the full-and reduced-parameterization behavioral ensembles can be considered "calibrated" in that each of these parameter realizations results in simulated daily mean streamflow that acceptably matches the observed data according to the three conditioning measures.

Global sensitivity analysis
Given the large difference in the number of parameters between the reduced (12) and full (1305) parameterizations, the interested reader may be wondering how many members of the reduced and full parameterizations influence either the conditioning measures or the QOIs or both. In an effort to address this question, we employed the method of Morris (Morris, 1991) which is a "one-at-a-time" GSA method; each parameter is varied, in turn, across the specified range, effectively sampling the sensitivity of QOIs and conditioning measures across parameter space. We used the modelindependent implementation of the method of Morris encoded in GSA utility of the PEST++ software suite (Morris, 1991;Welter et al., 2015) with 20 discretization points across the range of each parameter.

Results
The application of the method of Morris (1991) reveals a considerable number of model inputs that influence the conditioning measures as well as the designated brushmanagement QOIs. Furthermore, the combined Monte Carlo and associated GLUE-based conditioning process (MC-GLUE) analysis reveals a relatively large difference in the estimated range of QOI-5 between the reduced and full parameterization models.

Global sensitivity analysis
Of the 1305 model inputs treated as parameters, the method of Morris analysis indicates that only 194 parameters are noninfluential to the three conditioning measures and five brush-management QOIs (see the Supplement for a complete summary of the GSA results, including a table of the five most influential parameters for each QOI and conditioning measure, Tables S2 and S3). Note that many of the most influential parameters, specifically precipitation multipliers, plant growth parameters, and HRU-scale parameters, are not in the reduced parameterization and are not included in typical hydrologic modeling analyses (Arnold et al., 2012b).

Monte Carlo
The MC-GLUE analysis yielded 7155 and 6846 realizations (out of the 1 million member prior ensembles) that compose the behavioral ensembles for the reduced and full parameterizations, respectively. These behavioral ensembles reproduce the pretreatment daily mean streamflow data acceptably well according to the three conditioning measures. The relation of prior and behavioral ensembles to the three conditioning measures for the reduced and full parameterizations can be seen graphically in Fig. 3. The diagonal panes of Fig. 3 show the histograms of each of the three conditioning measures, whereas the off-diagonal panes show the relation between conditioning measures. Parameter realizations within the hatched boxes in Fig. 3 collectively form the behavioral ensembles for both the reduced and full parameterization.

Verification QOIs
In general, for both the reduced and full parameterizations, the behavioral distributions for ET-based QOIs (QOI-1 and QOI-3) are similar to prior distributions; conditioning has slightly shifted the distributions towards larger precipitation-ET ratios but has not substantially decreased the width of the distributions. The similarity between prior and behavioral distributions indicates the conditioning process has not changed the uncertainty that exists in model-simulated ET. The prior and behavioral distributions of reduced and full parameterizations bracket the measured value for QOI-1, QOI-2, and QOI-3 at the 95 % confidence level (Figs. 4,5,and 6).
QOIs related to streamflow (QOI-2 and QOI-4) have markedly different behavioral distributions compared to prior distributions, indicating considerable conditioning of streamflow-sensitive parameters. The measured value for QOI-4 (volumetric forecast-period -posttreatmentstreamflow-precipitation ratio) was not bracketed at the 95 % confidence level by either behavioral distribution or the prior distribution of the reduced parameterization (Fig. 7).

Forecast QOI
The prior uncertainty in the QOI-5 (the simulated difference between the total forecast-period ET in the pre-and posttreatment models) was substantially larger for the full parameterization compared to the reduced parameterization (Fig. 8): the reduced parameterization prior uncertainty ranged from approximately −4.1 to −2.1 %, whereas the full parameterization model yielded a prior uncertainty that ranged from approximately −7.5 to +0.5 %. Note a negative ET difference indicates a decrease in ET as a result of simulated brush management. The larger range yielded by the full parameterization is a direct outcome of specifying more uncertain parameters that influence QOI-5. QOI-5 behavioral uncertainty from the reduced parameterization is substantially different than the prior uncertainty; the 95 % confidence interval of the reduced parameterization behavioral distribution ranges from −2.5 to −2.0 %. The behavioral distribution of QOI-5 yielded by the full parameterization is similar to the prior distribution, but shifted slightly towards positive values; the 95 % confidence interval of the full parameterization behavioral distribution ranges from −6.2 to +0.5 % (Fig. 8a). Only slight differences between the prior and behavioral distributions for the full parameterization, again, indicate the selected conditioning process did not substantially change the reliability in simulated long-term changes in ET as a result of brush management. We attribute the differences in QOI-5 distributions between  Conditioning-period discharge-precipitation ratio (percent) Figure 5. Quantity of interest QOI-2: simulated conditioning-period (pretreatment) streamflow as a percentage of precipitation. The effects of the conditioning process can be seen as large reduction in the range of the behavioral distribution compared to the prior distribution. The prior and behavioral distributions for model parameterizations bracket the measured value.
the reduced and full parameterizations to the model error generated by using a reduced set of parameters to represent SWAT model input uncertainty. Note the prior distribution for the reduced parameterization was also nonparametric compared to the full parameterization counterpart, a numerical artifact we also attribute to the model error induced by the reduced parameterization.

Discussion
The full-parameterization behavioral distribution of QOI-5 included a range of possible outcomes from a net decrease to a slight net increase in the ET component of the long-term water budget (Fig. 8). This range of possible outcomes stems from the number of model inputs that were identified as uncertain and treated as parameters in the MC-GLUE analysis.
The possibility of a net increase in ET following brush management is not unprecedented. Harwell et al. (2016) showed a net decrease in surface-water yield following simulated brush-management activities for one of their simulated subbasins. Furthermore, we have demonstrated that conditioning of a hydrologic model to daily mean streamflow does not necessarily increase the reliability of forecasts made with the model. This study demonstrates the importance of robust uncertainty quantification to support simulations of brush management, and, more generally, simulation of the hydrologic outcomes of land-cover change. Without uncertainty quantification, the simulated outcomes of simulating brush management are simply single points on the behavioral QOI distributions, which conveys no information related to the reliability of the model results. The failure of the reduced-parameterization model to provide robust uncer- Fforecast-period ET-precipitation ratio (percent) Figure 6. Quantity of interest QOI-3: simulated forecast period (posttreatment) ET as a percentage of precipitation. All 95 % confidence intervals bracket the measured value. However, the conditioning process has done little to decrease uncertainty, as the behavioral distributions are similar to the prior distributions for both model parameterizations. Forecast-period discharge-precipitation ratio (percent) Figure 7. Quantity of interest QOI-4: simulated forecast period (posttreatment) streamflow as a percentage of precipitation. Both the parameterizations appear to have been "overfit" with respect to this QOI, as neither behavioral distributions bracket the measured value at the 95 % confidence level.
tainty estimates demonstrates the importance of parameterization when attempting to quantify uncertainty in land-cover change simulations. The results of our analysis should not be directly extrapolated to other hydrologic settings that are different from the one described herein. The MC-GLUE analysis showed that using a reduced parameterization to represent model input uncertainty leads to a misrepresentation and critical underestimation of the uncertainty in QOI-5, leading to artificially high confidence that brush-management activities will decrease the ET component of the water budget by approximately 2.0 to 2.5 %. By including a more representative and complete set of parameters to represent model input uncertainty, the resulting QOI-5 uncertainty estimate more appropriately conveys the reliability in the modeled outcome of brush management.
A clear link between level of parameterization and uncertainty estimates for the simulated results of brush management has been demonstrated, and issues such as underestimation of uncertainty and numerical artifacts are shown to be associated with a reduced parameterization. Furthermore, the results of applying the method of Morris revealed more than 1100 model inputs that were identified as uncertain and that also influence conditioning measures, QOIs or both. Following Sexton et al. (2011), parameters that influence the QOIs must be included in the uncertainty analysis, even if said parameters do not influence the likelihood function (e.g., they are not "identified" by the conditioning data). The demonstrated issues with the level of parameterization raise questions related to the concept of "overparameterization" (Jakeman and Hornberger, 1993) in the context of simulating the hydrologic outcomes of land-cover change. Each of the inputs that were selected for adjustment in the fullparameterization model were deemed uncertain at the start of the modeling analysis; whereas other practitioners may choose different prior distributions and/or ranges for these parameters, we doubt any practitioners would state these model inputs are known with certainty. There are two avenues to reduce QOI-5 uncertainty: (1) collect information directly about the model input variables that most influence QOI-5 -that is, reduce the prior uncertainty of the parameters that represent these inputs -or (2) collect additional hydrologic observations that, through conditioning, reduce the uncertainty of parameters that influence QOI-5. We recognize that the ET observation data used to formulate QOI-1 could in fact be used as a condition measure. Given the similarity between QOI-1 and QOI-5, it is possible that the conditioning-period ET data could be used to further condition several parameters that influence QOI-5, thereby reducing the behavioral uncertainty of QOI-5. However, the conditioning-period ET data provide a valuable validation of the model's performance, and using these data as a conditioning measure would provide unique and atypical conditioning.

Conclusions
This study provided an analysis of the ability of a SWAT model to forecast how brush management affects the longterm water balance within a watershed. The analysis relies on measured streamflow and independently derived evapotranspiration estimates to condition the parameterized model inputs as well as provide a verification of the model's performance during the forecast period. The method of Morris was used to investigate model input influence on conditioning measures and brush-management quantities of interest. Following the method of Morris, Monte Carlo and GLUE analyses were used to estimate the uncertainty of brushmanagement QOIs for the reduced and full parameterization schemes.
Our analysis reveals the importance of robust uncertainty quantification when simulating the outcomes of brush management, especially as it relates to how the model is parameterized. Failure to specify a complete and encompassing parameterization is shown to lead to an underestimation of uncertainty in simulated brush-management outcomes, which may lead to suboptimal water-resource decision making.
Given the number of identified uncertain model inputs and the associated specified uncertainty in said inputs, the model-simulated change in the long-term ET in the watershed is largely uncertain and includes a range of possible outcomes from a net negative to a slightly net positive change in the long-term ET component of the water budget. The resulting uncertainty in one of the primary metrics of brushmanagement effectiveness underscores the importance of robust and conservative uncertainty quantification. Watersheds with different hydrologic response characteristics will obviously behave differently, but, if modeling is used to evaluate brush-management outcomes, robust uncertainty quantification is needed to place the model results in a representative context. Data availability. A data release that supports the analyses presented herein is available at https://doi.org/10.5066/F7WH2NGR (White et al., 2017). The data release includes files and data needed to reproduce our analyses, including the following: 1. an ESRI ArcMAP 10.2.2 project that includes the ArcSWAT version 2012.10.2.18 project used to create the base model 2. base SWAT2012 input files generated by the ArcSWAT tool 3. PEST++ interface files including python pre-and postprocessing scripts.
The comma-separated value files used in the reduced and fullparameterization Monte Carlo analysis can be generated from the files provided in the data release (White et al., 2017). The ET, precipitation, and streamflow data used for conditioning and verification are available for download as the appendices to Banta and Slattery (2011) at the US Geological Survey Publication Warehouse (http://pubs.usgs.gov/sir/2011/5226/).