3.1 Frequency distribution of sub-daily rainfall

A point-to-grid analysis is undertaken to evaluate the sub-daily frequency distribution of BARRA precipitation. At each gauge location as shown in Fig. 1, corresponding BARRA estimates are obtained using the nearest-neighbour interpolation. The basis of comparison requires some thought as the fraction of zero values in sub-daily rainfall data is high (the 95 % quantiles at hourly temporal resolution are zero). This issue could be addressed by selecting suitable conditional thresholds, though different thresholds would need to be adopted for each temporal accumulation. For example, a threshold of 0.1 mm h⁻¹ could be adopted for hourly data and 1 mm d⁻¹ for daily data. However, the slight problem with this approach is that any trends in the performance metric with temporal accumulation will be confounded by the somewhat arbitrary choice of thresholds for intermediate temporal accumulations (3, 6, and 12 h). Accordingly, we derived quantiles for sub-daily rainfalls that occurred only on the rain days for each dataset individually, where a “rain day” is defined based on a threshold of 1 mm d⁻¹ (Ebert et al., 2007).

In investigating the frequency of rainfall, we compute the various quantiles (80, 90, 95, and 99 %) at different time accumulations (1, 3, 6, 12, and 24 h). The analysis of frequencies corresponding to different accumulations (up to 24 h) is selected to be of relevance for design rainfall applications. We then estimate percentage bias using $(m-o)/o\times \mathrm{100}$, where m and o denote the reanalysis and observed precipitation corresponding to the selected quantiles, respectively. This evaluation is further stratified across three broad climatic zones (arid, tropical, and temperate) as defined by the Köppen–Geiger classification (Peel et al., 2007; Fig. 1). It is worth noting that comparisons of point (gauged) and areal (gridded) rainfall are generally biased as the average precipitation over a grid cell is lower than rainfall recorded at a particular point. It would also be expected that sub-daily point rainfalls are more variable than those averaged over a grid cell area.

3.2 Neighbourhood-based sub-daily patterns at point locations

A direct comparison of precipitation, especially in higher-resolution datasets, is difficult as the conventional metrics are not able to penalise intensity and location errors in a desirable manner (Rossa et al., 2008). Performance metrics which compare point observations with model estimates averaged over a grid cell are heavily influenced by errors in the spatial pattern and location of rainfall events, even if the average rainfall depths over the local domain are the same. This is particularly the case with high-resolution precipitation datasets and with the analysis of sub-daily periods.

To mitigate this problem in a gauge-based evaluation, we adopt an approach that allows for possible timing and displacement differences in rainfall occurrence. The approach is similar to the neighbourhood (or “fuzzy”) approach used with single observation-neighbourhood forecasts (Ebert, 2008), in which gridded observations were evaluated against gridded forecasts. This method accounts for situations where an event defined on the basis of gauge measurements may miss the nearby grid cell, resulting in spatial error, and/or appear non-coincident at the nearby grid cell signifying a temporal error. For the former case, an evaluation that considers neighbouring grid cells can account for spatial errors. For cases involving a time displacement, a moving storm may appear at a neighbouring grid cell at the time step under consideration, approaching the nearby grid cell.

To account for these different types of errors, we employ an analysis that explicitly considers the occurrence and timing of rainfalls at neighbouring grid cells. This evaluation is undertaken by selecting large rainfall events (greater than 10 mm d⁻¹) in the gauge dataset. The cumulative fraction of rainfall over the day is computed for the gauge and the nearest neighbouring grid (Eq. 1). The squared difference in the fraction of cumulative rainfall occurring in each hour between the gauge and reanalysis rainfalls is then averaged to provide an error score (Eq. 2).

where f_i is the fraction of daily rainfall occurring at the ith hour.

Note that 23 is used in the denominator as $F_{\mathrm{24},\mathrm{model}}=F_{\mathrm{24},\mathrm{gauge}}=\mathrm{1}$ and is not included in the computation of the error score. The temporal error is considered by using cumulative precipitation at a daily scale, which penalises large temporal errors more than small ones. Similarly, to account for possible spatial displacement, we analyse the temporal distribution of precipitation by searching over neighbourhood space to find the best-performing grid cell. The error score is computed for both the nearest grid and the neighbourhood grid cells equidistant from the nearest grid. The minimum error score is selected and then averaged across all rainfall events at a location.

The average error score varies between 0 and 1. A score of 1 represents the worst-possible situation in which rainfall occurs in the first hour in one dataset and the last hour in the other. Conversely, a score of 0 indicates a perfect match between observations and model estimates. Scores between 0 and 1 indicate differing degrees of temporal error, where, for example, a score of 0.33 indicates that rainfall occurs in either the first or last hour in one dataset and is distributed uniformly throughout the day in the other.

3.3 Neighbourhood-based spatial evaluation

Finally, a spatial evaluation of reanalysis precipitation against blended radar estimates is undertaken. For each spatial domain, the largest 25 rainfall events to have occurred over the 3-year period are selected based on domain-averaged daily precipitation. Radar precipitation, which is available as 30 min accumulations, is aggregated to hourly values to match the temporal resolution of BARRA. Similarly, a common spatial resolution is adopted. The precipitation from BARRA (∼12 km) is re-gridded to the resolution of the radar grid (∼1.5 km) using area-weighted re-gridding, which means that single BARRA precipitation estimates are distributed over 8×8 radar grid cells.

Different metrics have been developed for undertaking spatially variable evaluations (Ebert, 2008, 2009). This study adopts the fractions skill score (FSS) from Roberts and Lean (2008), as it measures the variation of skill across increasing spatial scales and hence indicates the minimum spatial scale at which the model is skilful.

The FSS metric is based on the likelihood that rainfall over a given threshold occurs somewhere within the neighbouring window of grid cells. A common threshold rainfall rate is selected for both the observed and modelled grid cells. An increasing window of size n×n centred on a particular grid is selected (which yields N windows over the whole domain). For each window i, a fraction of grid points exceeding the threshold in observed rainfall (p_o,i) and modelled rainfall (p_m,i) are computed. Then, the FSS is calculated as

The FSS score varies between 0 and 1. A score of 0 represents a complete mismatch between two rainfall fields, and a score of 1 represents a perfect match. Usually, FSS is computed for varying sizes of spatial windows, and results are plotted as a function of window size. A random score (FSS_random) is the FSS that would be achieved, on average, by a random field with the same fraction of observed events (p_o) over the domain. A benchmark score, a target or “uniform” skill (FSS_uniform), is given to a uniform field with a probability of occurrence equal to p_o at each grid cell (Roberts and Lean, 2008). FSS_uniform is expressed as $\mathrm{0.5}+p_{\mathrm{o}}/\mathrm{2}$, which is halfway between a perfect score (1) and a random score (FSS_random=p_o). This FSS_uniform can be approximated to 0.5 when p_o is small, as in the case of larger precipitation thresholds.

The thresholds used to calculate FSS are based on rainfall quantiles in observed and reanalysis data which are derived for each time step. A rainfall intensity greater than 0.2 mm h⁻¹ is used as a threshold to define a rainy grid cell for which quantile-based thresholds are computed. This ensures that the model and observed rainfall fields have an identical fraction of rain events for each threshold value. The application of quantile-based thresholds aims to remove the impact of any bias in rainfall amount and focus solely on spatial accuracy of modelled precipitation (Mittermaier et al., 2013; Roberts and Lean, 2008; Skok and Roberts, 2018). It is worth noting that the FSS metric only provides information on variation of performance with increasing spatial scale. The timing errors at finer temporal scales can be indirectly discerned by analysing variation of FSS with time accumulations. Therefore, the FSS is evaluated at temporal aggregations of 3 and 6 h.

4.1 Frequency distribution of sub-daily rainfall

Figure 2 illustrates the spatial distribution of quantile bias and its summary across climatic zones. Overall, the spatial distribution of bias across all time accumulations and quantiles exhibits a similar pattern. The biases in the northern region are higher and spatially more variable than those in the southern region. The variation in bias across quantiles is the highest for hourly rainfall, especially in arid and tropical climatic zones. For the 80 % quantile of hourly rainfalls, all stations in the arid and tropical zones exhibit a higher positive bias than those in the temperate zone. This difference is partly due to the tendency of BARRA to overestimate light-rain events (Su et al., 2019). Despite considering wet days only in comparing quantiles, hourly rainfall at the 80 % quantile for the arid and tropical region is dominated by small precipitation amounts, which results in a high positive bias. At higher quantiles, the BARRA estimates are all lower than the gauged point rainfalls. This negative bias is largest in the tropical zone, followed by the arid zone. Although the nature of the high hourly bias varies with location, a step reduction in bias is observed when the accumulation time periods increase from 1 to 3 and 6 h. This is partly due to reduced inherent bias arising from the adoption of longer temporal accumulations, which reduces the potential for differences in timing between observations and model estimates.

Figure 2Percentage bias in precipitation at quantiles (rows) and accumulations (columns). The box plots represent the summary of percentage bias across climatic zones for respective rows and columns. Each box in the box plot represents 25th–75th-quantile values, the horizontal line in the box represents the median, and whiskers represent the 5th–95th quantiles.

The spatial distribution of biases associated with the different quantiles exhibits a similar pattern for all temporal accumulations considered. At shorter accumulations (up to 3 h), the biases change from positive to negative and gradually increase in magnitude with increasing quantile. At higher accumulations (6, 12, and 24 h), the bias is negative but decreases in magnitude with increasing quantiles. At higher quantiles (95 and 99 %), the bias is the least in the temperate zone. The BARRA estimates are under-predicted at all time accumulations, but this improves with increasing temporal accumulations.

4.2 Neighbourhood-based sub-daily patterns at point locations

Figure 3 shows how the minimum error score changes with varying neighbourhood grid size. It is seen that the error score decreases significantly when a neighbourhood of 3×3 grid cells (about 35×35 km) is considered instead of a single nearest neighbour only. The error score continues to decrease slightly as the size of the neighbourhood increases, though the adoption of a larger neighbourhood increases the likelihood that rainfall events unrelated to the gauge observations are being considered. The results show that the temporal distribution of the BARRA precipitation estimates within a wet day are representative yet displaced in terms of location. The error scores are all less than 0.33, which indicates that the temporal distribution of sub-daily BARRA precipitation is on average superior to a uniform distribution of estimates derived by simply disaggregating daily rainfalls over a day.

Figure 3Box plots showing the distribution of the minimum error score across stations in each neighbourhood grid size. The box represents 25–75th-quantile values, the horizontal line in the box represents the median, and whiskers represent 5–95th quantiles.

Download

4.3 Neighbourhood-based spatial evaluation

Figure 4 shows the hourly precipitation rates corresponding to different quantile thresholds in the Brisbane, Darwin, Melbourne, and Sydney regions, for the largest 25 events that occurred between 2014 and 2016. The Melbourne and Sydney regions are more similar than other domains in terms of the frequency distribution of hourly precipitation and the discrepancy between radar and BARRA precipitation.

Figure 4Hourly precipitation rates corresponding to quantiles for BARRA and blended radar data at four different locations.

Download

Overall, the rainfall magnitude corresponding to quantiles is higher for the radar estimates than for BARRA, and this can be attributed to the difference in spatial scale and the area-weighted re-gridding scheme. The difference between the two datasets is greater at higher quantiles. However, at around 99 % quantiles, the precipitation from BARRA is close to or greater than radar precipitation.

FSS is calculated for varying quantile thresholds (50, 75, 90, 95, and 99 %) at different accumulations across time (1, 3, and 6 h) and presented in Fig. 5. As expected, FSS increases with increasing neighbourhood size and decreases with increasing threshold. If there is no frequency bias, then the FSS curve is expected to asymptote to 1 as the neighbourhood size is increased. In Fig. 5, the maximum FSS of 1 is not achieved even at the neighbourhood size of 300 km, which signifies a frequency bias in BARRA.

Figure 5Mean FSS as a function of neighbourhood distance for rainfall above quantile thresholds (indicated by colours) at various locations (rows) and accumulations (columns). The dashed horizontal lines indicate the target or uniform (FSS_target or FSS_uniform) skill for each threshold as specified by Roberts and Lean (2008).

Download

The FSS scores vary with location, where the regions in increasing order of performance are Darwin, Brisbane, Melbourne, and Sydney. The results for hourly precipitation and 75 % quantile threshold suggest that the skilful spatial scale L ($\mathrm{FSS}>\mathrm{0.5}+p_{\mathrm{o}}/\mathrm{2}$) is less than 100 km for all the locations except Darwin. The skilful spatial scale for 90 and 95 % quantile thresholds increases to 150 km. For the 99 % quantile threshold, BARRA estimates only exhibit useful skill over spatial scales larger than around 250 km.

The FSS metric only provides information on how performance varies with increasing spatial scale. It does not account for timing errors associated with events that might be initiated at different times and/or evolve at different rates. An indication of such timing errors may be discerned by assessing how the FSS varies with increasing time accumulations. The results shown in Fig. 5 indicate that FSS improves with increasing time accumulations. For example, at Darwin, BARRA is able to provide skilful estimates of 50 % threshold of hourly rainfalls only at a scale of 200 km; however, at 3 and 6 h accumulations, the corresponding spatial scale reduces to 125 and 100 km, respectively. The accuracy of the BARRA estimates decreases with increasing rainfall severity, and even at the longest time accumulations the spatial scale at which rainfalls above the 99 % quantiles are skilful extends out to 300 km.

5.1 Performance based on wet-day quantiles

The unconditional evaluation of precipitation frequency in terms of wet-day quantiles examines the representativeness of sub-daily climatology (Fig. 2). The minimum bias across all time accumulations and quantiles in the temperate zone suggests that BARRA provides unbiased estimates of temperate rainfalls over the observed range of events. In addition, an improved performance is observed across all regions when temporal accumulations are considered. This includes the arid and tropical zones where BARRA performs poorly at an hourly scale. A slight negative bias is observed across all locations in most of the quantiles. This underestimation is however expected due to the mismatch in spatial scale between point observations and grid-average precipitation (Maraun, 2013). The point precipitation, in general, is expected to be higher than areal rainfall at a 12 km spatial scale.

5.2 Spatio-temporal representation of rainfall

A potential displacement of the precipitation field in space and time is expected when evaluating high-resolution precipitation datasets, especially when performance is assessed on an hourly timescale and at a point (or single grid cell) location (Rossa et al., 2008). The BARRA estimates exhibit such displacement errors, as evidenced by improved performance in neighbourhood analysis and FSS analysis. An assessment of hourly temporal patterns shows an improvement when neighbourhood grid cells are considered (Fig. 3). This suggests that, when using precipitation from BARRA, users could benefit from considering spatial and temporal displacement in the precipitation field, especially during large events. This further suggests an opportunity to utilise hourly distribution of rainfall for disaggregating daily totals.

At the catchment scale, the accuracy of the spatial distribution of rainfall is important for flood modelling. The evaluation of the spatial performance of BARRA against radar data for selected large events showed a mixed result between locations. BARRA rainfalls for the 90 % quantile at 3 h accumulation achieves the target FSS at a spatial scale of range 100–140 km for all domains except Darwin (250 km) (Fig. 5). Achieving useful skill only at a large spatial scale can be partly attributed to the spatial error in the precipitation fields. The higher quantile thresholds are related to small-scale features which are even more likely to be subject to spatial error. Therefore, it is difficult to achieve a skilful spatial scale for these more extreme rainfalls. This has also been pointed out by Roberts and Lean (2008) based on the skill of numerical weather prediction (NWP) model outputs at various quantile thresholds (75, 90, 95, and 99 %). The nature of such spatial displacement errors should be considered if BARRA estimates of precipitation are intended to be used for hydrological modelling.

This spatial evaluation is conducted at selected locations to take advantage of the availability of high-resolution blended radar datasets. Across the whole Australian continent there is a dearth of such high-resolution observations of precipitation, and BARRA could be relied upon to provide estimates of sub-daily precipitation at selected spatial scales in regions where there is a paucity of gauging data.

5.3 Performance dependence on spatial location

The overall performance of BARRA varies with location, and this similar trend was evident across all evaluation methods. Both the bias analysis of quantiles and the FSS evaluation show that the performance of BARRA gradually improves as we move from northern to southern regions (Figs. 2 and 5). This variation in performance can partly be attributed to the different rainfall climate in these regions. The convective precipitation during the summer season is dominant at northern (low) latitudes, whereas winter (or uniform) frontal rainfall is dominant at southern (high) latitudes. The poorer performance in the tropics, where convective precipitation is dominant, reflects the limitations of the parameterisation scheme to describe sub-grid-scale rainfall (Su et al., 2019). This is consistent with the finding of Ebert et al. (2007) that focuses on general performance of NWP models. The variation in performance of sub-daily BARRA precipitation across spatial location and different climatologies is consistent with the daily evaluation performed in Acharya et al. (2019).

5.4 Performance as a function of temporal resolution

Availability of hourly rainfall observations (based on both pluviometer and radar products) enables BARRA estimates of hourly rainfalls to be evaluated. The uneven and sparse distribution of pluviometer gauges across Australia and the limited availability of radar products is not sufficient for an overall assessment of sub-daily precipitation across the whole of Australia, yet, given the wide range of climatologies considered, the analyses provide a comprehensive evaluation of BARRA. Our unconditional (Fig. 2) and direct comparisons (Fig. 5) both illustrate a similar dependency of performance on temporal resolution. The performance shows a significant improvement when estimates are accumulated from 1 to 3 h. The increased performance for coarser temporal resolution can also be linked to the smoothing effect of rainfall at larger accumulation times and reduction of inherent bias.

The trade-off between performance and temporal resolution needs to be considered in combination with the objectives for which the precipitation estimates are used. However, in general terms the increased performance at 3 and 6 h accumulation suggests that it is better to use these accumulations from BARRA. Given that the performance of BARRA does vary with location, it is expected that shorter (3 h) accumulations may be appropriate for use in temperate locations, and longer accumulations (6 h) in the tropical and arid regions.