Exploring the merging of the global land evaporation WACMOS-ET products based on local tower measurements

Jiménez, Carlos; Martens, Brecht; Miralles, Diego M.; Fisher, Joshua B.; Beck, Hylke E.; Fernández-Prieto, Diego

doi:https://doi.org/10.5194/hess-22-4513-2018

Articles | Volume 22, issue 8

https://doi.org/10.5194/hess-22-4513-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/hess-22-4513-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 22, issue 8

Research article

|

27 Aug 2018

Research article |

| 27 Aug 2018

Exploring the merging of the global land evaporation WACMOS-ET products based on local tower measurements

Carlos Jiménez, Brecht Martens, Diego M. Miralles, Joshua B. Fisher, Hylke E. Beck, and Diego Fernández-Prieto

Download

Final revised paper (published on 27 Aug 2018)
Preprint (discussion started on 06 Nov 2017)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'review comments', Anonymous Referee #1, 13 Dec 2017
- AC1: 'Commenst to Review 1', Carlos Jimenez, 21 Dec 2017
RC2: 'Reviewer comments hess-2017-573', Anonymous Referee #2, 13 Dec 2017
- AC2: 'Commenst to Review 2', Carlos Jimenez, 21 Dec 2017
RC3: 'Review of hess-2017-573', Anonymous Referee #3, 28 Dec 2017
- AC3: 'Commenst to Review 3', Carlos Jimenez, 08 Jan 2018
  - AC4: 'Further comments to response to Review 3', Carlos Jimenez, 26 Jan 2018

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reject (21 Jan 2018) by Harrie-Jan Hendricks Franssen

Dear Mr Jimenez,

Your manuscript “Local tower-based merging of two land evaporation products” has been subjected now to review by three reviewers. The three reviews are strongly dispersing with one reviewer recommending rejection and two reviewers recommending minor revision. Giving the very different recommendations, I studied the paper in more detail. The paper is very well written, but I agree with reviewer #3 that the motivation for this work is unclear. The main points of reviewer #3, which I agree with are:
- The merged ET-product is not independently validated. The towers ET-data also have considerable systematic and random errors. The fact that the merged ET-product fits the tower ET-data closer is by construction.
- Only estimates are provided at the tower locations, so the additional value of the product is also unclear. You argue in your response that this is a first step. I would suggest to also take the next step and to include both steps in one paper, instead of dividing it over multiple papers with not enough scientific contribution in this first paper.
- The energy balance closure error of the EC-data is an important issue not sufficiently addressed.
- Why are these two products merged and not a large number of ET-products? Reviewer #2 also addresses this point and suggests the use of “a more diverse palette of models”.

Although I think that the comments of reviewer #3 could be handled in a major revision (extending estimates beyond tower locations, including independent verification), in your response letter you indicate that you plan only to make very minor changes. Therefore, unfortunately I will have to reject the paper but would recommend resubmission along the lines indicated above. I am sorry not to be able to make a more favorable decision on this manuscript.

Best regards,

Harrie-Jan Hendricks Franssen – editor

Hide

ED: Reconsider after major revisions (further review by editor and referees) (27 Jan 2018) by Harrie-Jan Hendricks Franssen

Dear Mr Jimenez,

Your manuscript “Local tower-based merging of two land evaporation products” has been subjected now to review by three reviewers. The three reviews are strongly dispersing with one reviewer recommending rejection and two reviewers recommending minor revision. Giving the very different recommendations, I studied the paper in more detail. The paper is very well written, but I agree with reviewer #3 that the motivation for this work is unclear. The main points of reviewer #3, which I agree with are:
- The merged ET-product is not independently validated. The towers ET-data also have considerable systematic and random errors. The fact that the merged ET-product fits the tower ET-data closer is by construction.
- Only estimates are provided at the tower locations, so the additional value of the product is also unclear. You argue in your response that this is a first step. I would suggest to also take the next step and to include both steps in one paper, instead of dividing it over multiple papers with not enough scientific contribution on this first paper.
- The energy balance closure error of the EC-data is an important issue not sufficiently addressed.
- Why are these two products merged and not a large number of ET-products? Reviewer #2 also addresses this point and suggests the use of “a more diverse palette of models”.

You indicated in your response letter that you can handle the comments of reviewer #3 in a major revision (extending estimates beyond tower locations, including independent verification). We are therefore looking forward for a new version that addresses these concerns.

Best regards,

Harrie-Jan Hendricks Franssen – editor

Hide

AR by Carlos Jimenez on behalf of the Authors (23 Mar 2018) Author's response Manuscript

ED: Referee Nomination & Report Request started (26 Mar 2018) by Harrie-Jan Hendricks Franssen

RR by Anonymous Referee #2 (13 Apr 2018)

RR by Anonymous Referee #3 (25 Apr 2018)

Suggestions for revision or reasons for rejection

The authors have expanded on their original submission, by extrapolating the estimated weights using a neural network, and adding a third product to the merger. While the manuscript is certainly improved from the first submission, I still have major reservations about the motivation for this work, and the appropriateness/accuracy of the chosen methodology. These issues would need to be address prior to publication.

1.
The motivation for merging these particular products needs to be provided in the manuscript (not just in the reviewer response). Also, there are other products available that you could use, say from reanalyses.

Given that all three input ET products are forced wih the same data, it is not clear to me why using a statistical method (based on random errors) would be beneficial to account for systematic differences (driven by model structure / parameters) between the three products. This is a fundamental flaw in this work.

While this is not actually stated, it seems to me that the main point of this work is to determine whether these three products can be improved upon by merging them, and if so, can they be merged more effectively than using an (unweighted) average. There are two steps to this:

i. Are the weights (or more importantly, the final product) significantly different between the standard averaged, and with the eights calculated using a more sophisticated approach?
Currently this has not been adequately answered, due to the flawed method used to estimate the weights.

ii. Can the weights calculated at the limited number of tower locations be usefully extrapolated globally?
Currently this cannot be answered, as not enough information is given regarding the NN used to extrapolate the weights.

2.
As in the first review. The inverse-error variance weights are based on the assumption of unbiased and independent data sets. The data sets used here are strongly dependent and biased, and this cannot be ignored. At a minimum for publication, the weights (and consequently the averaging) needs to be based on anomalies (to remove the bias, and the more systematic aspects of the dependence), and the lack of independence needs to be acknowledged prominently, including qualifying all conclusions by noting that these data sets were not independent. If you don't have enough data to use anomalies, then you cannot apply this method by ignoring the inconvenient biases.

3.
The NN network is not adequately explained, and reads like it has been blindly applied rather than carefully investigated. As with the first review, I still have concerns about over-fitting. Since so little information is given, it is not clear whether the NN was not useful for extrapolation, because of the limited locations of the tower observations, or because the NN itself was inadequate.
For publication, the NN needs to be more adequately explained. This includes details of how the training data sets were split into training and evaluation data within the NN algorithm (and how this then relates to the experiments where a single station was removed), how the input data was selected / what else was tested, and how you ensure that the output weights equal 1.

Also, it isn't stated whether a separate NN is calculated for each day of the year, or if all data is thrown in to a single NN. I suspect it is the former, but would strongly recommend the latter.
How do you deal with hemispheric differences? It doesn't really make sense to predict SH weights for a day of year, based on NH data only (there is only one SH tower, in Australia).
By training a single NN using data for all days (and not including day of year in the training data) you could withold data for some days to test whether the NN can at least reproduce the sites that are sampled.

MINOR:

1A general grammar check would be useful, with attention paid to missing possessive apostrophes.

L49: please double check that the MTE product is a regression. I thought it was a machine learning approach, but could be wrong.

Section 2.1 (from first review also). Add the spatial resolutions in here.

Replace all instances of 'inverse-variance weighting' with 'inverse error variance weighting'

L210. "Only the systematic component of the error can potentially be captured by the NN, and there is not warranty that all the systematic errors are dependent on the model inputs".

This is incorrect/misleading. "systematic errors" usually refers to biases, and these methods can predict the mean square of the random errors (which is what your inverse error variance weights should reflect). Note that statistical methods like this do not predict individual errors, rather they predict the tendency towards larger/smaller errors. Regarding the second half of the sentence, this is why you need to do some work to show that you have selected appropriate input data sets for NN.

L273: quantify 'too close' . How is "clearly not representing the overall land cover' determined?

Explicitly note here that the towers do not provide comprehensive global coverage, and don't cover many biomes, climate regimes, or the Southern Hemisphere.

L280: representativity errors should be mentioned here (tower to 25 km).

Figure 2 : 100 Wm2 is a huge range for each colorbar. Please plot with a finer discretization.

Figure 4: It is stated repeatedly that the weights don't differ much from 1/3, and yet this plot shows a large deviation. Statistics are needed here to quantify the divergence from 1/3.

Figure 7: It doesn't really make sense to plot global fields for three month blocks.

Figrue 10: THis plot is difficult to understand. Why not plot Ih v. RMSE?

Ih has not been defined anywhere in the manuscript.

L507: what about systematic errors in the obs? (inc. representativity)?

L509: "if the difference with the observations were mostly random in nature, we should not expect the observations to provide much guidance to combined the products".
This is not correct. See comment on L210.

L521: No. You could have defined the bias over the data time period that you do have.

L565: the occurrence of negative weights is because you have dependent products.

L581: Qualify that this approach assumes that the surface water storage has not changed over this time period, and that this assumption will not necessarily hold over the limitted time period you are using.

Note that this is the same precip used in the ET products, and that this does not represent an independent evaluation.

L605: either explain why the slope is an important statistic here, or delete it.

Hide

ED: Reconsider after major revisions (further review by editor and referees) (04 May 2018) by Harrie-Jan Hendricks Franssen

Dear Mr Jimenez,

Your manuscript “Local tower-based merging of two land evaporation products” has been subjected now to re-review by two of the original reviewers. The two reviews are again strongly dispersing with one reviewer recommending major revision or rejection and another reviewer recommending technical corrections only. I think again that the most critical reviewer brings in important points to be considered. Basically, the reviewer argues that merging three correlated and biased datasets is very tricky, and that the assumptions behind the methods you used are unbiasedness and independence. This reviewer argues that there is therefore a fundamental problem with the current analysis.

Important points to be handled in a revised version of the manuscript are:
- Motivation why you merge these three particular ET-products.
- Address the issue of biasedness and dependence of the ET-products in the context of using NN.
- Additional explanation of the applied NN-methodology.
- Is the weighted average of the three ET-products and the global interpolation an advantage compared to a simple average?

Handling the comments of this reviewer requires a major revision. The policy of HESS is not to repeat major revision rounds. However, given the fact that in this case two reviewers recommend publication of the manuscript, and only one reviewer major revision or rejection (but in general I agree with the arguments of this reviewer), my recommendation is to allow for a second major revision round. The authors should evaluate whether such an additional revision round is feasible. I am looking forward to a new version that addresses the concerns.

Best regards,

Harrie-Jan Hendricks Franssen – editor-

Hide

AR by Carlos Jimenez on behalf of the Authors (15 Jun 2018) Author's response Manuscript

ED: Referee Nomination & Report Request started (18 Jun 2018) by Harrie-Jan Hendricks Franssen

RR by Anonymous Referee #3 (26 Jul 2018)

ED: Publish subject to technical corrections (26 Jul 2018) by Harrie-Jan Hendricks Franssen

AR by Carlos Jimenez on behalf of the Authors (07 Aug 2018) Author's response Manuscript

Short summary

Observing the amount of water evaporated in nature is not easy, and we need to combine accurate local measurements with estimates from satellites, more uncertain but covering larger areas. This is the main topic of our paper, in which local observations are compared with global land evaporation estimates, followed by a weighting of the global observations based on this comparison to attempt derive a more accurate evaporation product.