Complex relationship between seasonal streamflow forecast skill and value in reservoir operations

Turner, Sean W. D.; Bennett, James C.; Robertson, David E.; Galelli, Stefano

doi:https://doi.org/10.5194/hess-21-4841-2017

Articles | Volume 21, issue 9

https://doi.org/10.5194/hess-21-4841-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Special issue:

Sub-seasonal to seasonal hydrological forecasting

https://doi.org/10.5194/hess-21-4841-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 21, issue 9

Research article

| Highlight paper

|

28 Sep 2017

Research article | Highlight paper |

| 28 Sep 2017

Complex relationship between seasonal streamflow forecast skill and value in reservoir operations

Sean W. D. Turner, James C. Bennett, David E. Robertson, and Stefano Galelli

Download

Final revised paper (published on 28 Sep 2017)
Preprint (discussion started on 04 Jan 2017)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review of HESS-2016-691, Value of Seasonal Streamflow Forecasts in Emergency Response Reservoir Managment', Chris Delaney, 02 Feb 2017
- AC1: 'Reply to Chris Delaney (reviewer #1)', Stefano Galelli, 20 Apr 2017
RC2: 'Review of the manuscript hess-2016-691 "Value of seasonal streamflow forecasts in emergency response reservoir management"', Anonymous Referee #2, 09 Feb 2017
- AC3: 'Reply to reviewer #2', Stefano Galelli, 20 Apr 2017
RC3: 'Interactive comment on “Value of seasonal streamflow forecasts in emergency response reservoir management” by Sean W. D. Turner et al.', Anonymous Referee #3, 10 Mar 2017
- AC2: 'Reply to reviewer #3', Stefano Galelli, 20 Apr 2017

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Publish subject to revisions (further review by Editor and Referees) (08 May 2017) by Andy Wood

AR by Stefano Galelli on behalf of the Authors (18 Jun 2017) Author's response Manuscript

ED: Referee Nomination & Report Request started (07 Jul 2017) by Andy Wood

RR by Anonymous Referee #2 (08 Aug 2017)

Suggestions for revision or reasons for rejection

The paper has generally improved by re-structuring the presentation of the results around the distinction between “supply objective” and “level objective” and by highlighting how the buffering role of the reservoir storage contributes to the forecast value.

Still, the paper needs a revision before the publication, in my opinion. In particular, the discussion of the results is, in some cases, biased and little effort has been made to generalize the results so to be exportable to other case studies (as detailed in the specific comments below). Also, it is important to remark in the text that the results are obtained using a deterministic optimization and to highlight the potential drawbacks of this approach. Finally, I suggest revising the section “Discussion and conclusions” of the paper to highlight the limitations of the approach (as detailed below).

Specific comments

Page 1, Lines 17-20. The fact that “good forecast accuracy does not necessarily translate into performance improvement” is not “surprisingly” (see among others: Goddard, L., Aitchellouche, Y., Baethgen, W., Dettinger, M., Graham, R., Hayman, P., Kadi, M., Martìnez, R., and Meinke, H. (2010). Providing seasonal-to-interannual climate information for risk management and decision-making. Procedia Environmental Sciences, 1, 81-101.).
The authors should revise the abstract accordingly and (at least briefly) mention in the introduction that the relationship between forecast skill and value has already been studied (including some relevant references).

Page 1, Lines 20-21. I would remove the sentence because it is not clear.

Page 2, Lines 19-20. I would remove the statement. Instead, I would enlarge the description by including more technical details on the approach (e.g., what “design parameters” are “adjusted”, what optimization scheme is used, how the forecast value is computed, which forecasts are used, etc). I would remove the last line, because the paper does not deal with the risk associated to forecast-informed decisions.

Page 4, Equation 2. The mass balance should include the spill.

Page 4, Line 15-16. Specify what you mean for “hydrological conditions”.

Page 4, Line 20. Include the description of the release strategy when the demand can not be met in full.

Page 6, Line 25. From this paragraph, it seems that a deterministic approach to reservoir optimization has only advantages, while it is well known that optimization approaches that explicitly account for the forecast uncertainty (included in a forecast ensembles, for instance) allow for better operating performances (e.g., Boucher, M., D. Tremblay, L. Delorme, L. Perreault, and F. Anctil (2012), Hydro-economic assessment of hydrological forecasting systems, Journal of Hydrology).
The authors should properly comment the drawbacks of the deterministic optimization approach.

Page 8, Lines 11-25. The whole paragraph is not fully convincing, because the authors comment only part of the results reported in Figure 5. For instance, in contrast to what it is currently commented in the text, there are also some cases in which the forecast skill is weak, but the decisions outperform the benchmark (as in Figure 5 e-f). Why does this happen?
The authors should comment all the results they obtain and should try to infer the causes, so that the entire description would result more convincing and exportable to other cases.

Page 8, Line 20. Which reservoirs are the “very large reservoirs” mentioned in the text? The results in Figures 5-7 should be commented also in relation to the storage ratio, which might clarify some of the results.
The storage ratio should correspond to the capacity to inflow ratio. I would include also this definition in the caption of Table 2, because it is more commonly used in the literature.

Page 8, Line 26. This comment is quite strange, because the Upper Yarra Reservoir is the reservoir with the lowest storage ratio, among the ones considered in the experiment 1 (i.e., 0.2 against 0.76, 0.35, and 2.84), which suggests that this reservoir has the lower potential for buffering inflows. Can the authors comment on this? If not, this comment should be removed.

Page 9, Line 13. I suggest using “difference” instead of “variance”.

Page 9, Lines 17-21. Can you cite relevant literature describing this behavior? Otherwise, it should be clear that you infer this comment from your results and not that your results confirm what other works have already observed (as it seems from the current text).

Page 9, Lines 22-23. The Eppalock reservoir is very big in almost all the configurations that you consider, as demonstrated by the storage ratio in Table 2. Can you comment on the reason why it is interesting or even realistic take into account such configurations?

Page 9, Lines 30-31. The Eppalock and Upper Yarra reservoirs are characterized by pretty different configurations in this experiment (see for example the storage ratio). Why do they show similar results? I believe that the authors should put more effort in interpreting the results, so to give insights that allow extending the results from the specific configurations to more general settings. If this is not possible, the discussion and conclusions should acknowledge that the results described in the paper are relevant to the specific case studies.

Page 9, Lines 32-33. A (similar?) drawdown is visible also for the draft ratios equal to 50% and 80% reported in Figure 7. Why is the 90% draft ratio not included?

Page 9, Lines 33-36. Can this be a valuable justification given that the performances of the Serpentine catchment (reported in Figure 3) are similar to the performances for the Burrinjuck reservoir and better than the ones for the Eppalock reservoir?

Page 9, Lines 37-39. I agree with the authors that the forecasts should be assessed on the longest period possible (see the author response to the previous review). Still, I believe it would be interesting to quantify the forecast value on informing reservoir operation in case of less extreme droughts that the Millennium drought. It might result that the value is not as pronounced or that errors in the forecasts are not as decisive for properly hedging.

Page 10, Line 25. What do the author mean exactly with “high sensitivity of the Serpentine reservoir”? If I am not mistaken, this is not commented in Section 3.2.

Page 10, Lines 33-43. What is the rationale behind computing the consistency of the forecast quality over blocks of more that 12 or 24 months? This piece of information would be relevant only for reservoirs that are able to buffer inflows on time scales that are comparable to those blocks (which does apply only to the configurations of the Eppalock reservoir, among the ones considered in this work).

Page 11, Section 5. I have the impression that the section “Discussion and conclusions” summarizes only part of the results (for instance the sentence reported in Lines 12-14 is not true for all the configurations tested, e.g., it does not hold for Figure 5d). It should be revised in order to be more balanced.
In addition, the authors should mention explicitly that the results are obtained using a deterministic approach and they should comment about the drawbacks of adopting such an approach.
Finally, Lines 26-40 include a discussion that is not supported by the results of the paper and therefore should be removed (or moved to the introduction). For the same reason, I would remove the last sentence on Page 12, Lines 13-14.

Page 12, Lines 4-5. Not always mediocre forecasts would imply “a false confidence in the forecast-informed decision scheme”. For instance, forecasts with medium skill may drive good reservoir operation performances when the system is insensitive to some forecast errors (this can happen, for example, because of the buffer capacity of the reservoirs).

Page 21, Figure 5. Why does the forecast value show such a high variance when the supply objective is considered (subplot a-d)? I would expect that the storage buffer effect would make the reservoir operation insensitive to some forecast errors.

Hide

ED: Publish subject to minor revisions (further review by Editor) (14 Aug 2017) by Andy Wood

AR by Stefano Galelli on behalf of the Authors (22 Aug 2017) Author's response Manuscript

ED: Publish as is (24 Aug 2017) by Andy Wood

Short summary

This study investigates the relationship between skill and value of ensemble seasonal streamflow forecasts. Using data from a modern forecasting system, we show that skilled forecasts are more likely to provide benefits for reservoirs operated to maintain a target water level rather than reservoirs operated to satisfy a target demand. We identify the primary causes for this behaviour and provide specific recommendations for assessing the value of forecasts for reservoirs with supply objectives.