Percentile-based grain size distribution analysis tools (GSDtools) – estimating confidence limits and hypothesis tests for comparing two samples

Eaton, Brett C.; Moore, R. Dan; MacKenzie, Lucy G.

doi:https://doi.org/10.5194/esurf-7-789-2019

Articles | Volume 7, issue 3

https://doi.org/10.5194/esurf-7-789-2019

© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/esurf-7-789-2019

© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 7, issue 3

Research article

|

02 Sep 2019

Research article |

| 02 Sep 2019

Percentile-based grain size distribution analysis tools (GSDtools) – estimating confidence limits and hypothesis tests for comparing two samples

Brett C. Eaton, R. Dan Moore, and Lucy G. MacKenzie

Download

Final revised paper (published on 02 Sep 2019)
Supplement to the final revised paper
Preprint (discussion started on 13 Feb 2019)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review comments to Eaton et al. (2019): Estimating confidence intervals for gravel bed surface grain size distributions by K. Bunte.', Kristin Bunte, 16 Mar 2019
- AC1: 'Reply to comments from Reviewer 1', Brett Eaton, 24 Apr 2019
RC2: 'Review for Eaton et al. (2019)', Anonymous Referee #2, 21 Mar 2019
- AC2: 'Reply to comments from Reviewer 2', Brett Eaton, 24 Apr 2019
EC1: 'Editorial guiding remarks for revision', Jens Turowski, 21 Mar 2019

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Brett Eaton on behalf of the Authors (29 May 2019) Author's response Manuscript

ED: Referee Nomination & Report Request started (13 Jun 2019) by Jens Turowski

RR by Anonymous Referee #2 (04 Jul 2019)

Suggestions for revision or reasons for rejection

I have read both versions of the submitted manuscript and thank the authors for their detailed and thoughtful responses to the comment as well as for the substantial revisions that they have made to the paper. I thought that the changes made the paper much easier to understand and appreciate the effort entailed in making these revisions. I have a handful of remaining comments that should not take much work to address.

P2 L24 I would have to agree with Kristin Bunte’s original review about this line in the text. It is very difficult to say what is the largest source of uncertainty in measured grain size distributions and all of the other things that she lists could be just as important as sample size. For example, not choosing an appropriate representative sample location in a reach (with wide spatial variations in grain sizes) could lead to larger errors than not sampling enough grains in the right location. I appreciate the authors’ changes to the text but I think that this sentence is still misleading and ignores many other error sources. I would suggest simply changing the sentence to “…is ONE of the largest sources of uncertaintIES…” to address this.

Section 2.1.3 on approximate solution. I think it would be good to note here, beyond the title of this section, that this relies on assuming equal area tails. Doesn’t this mean that the more the grain size distribution deviates from the equal area tail assumption, the larger the errors in approximate solution? If so, a note of caution is really needed for potential users of this approximation.

Section 3.1 I think that in step 3 of the bootstrap approach, you need to clarify that you are calculating the desired percentile value from the bootstrap samples, rather than just the samples as stated here. The original data used in the bootstrapping approach were also called “samples” so this step may be confusing as to which exact “samples” are being used here (original or bootstrapped data). Also shouldn’t the subscripts in the equations for this step be “xk” rather than “x” and “yl” rather than “y”? I would recommend the same changes are made in Section 3.2, which has similar steps, equations, and use of the word “sample” in step 3.

Figure 5. I am somewhat confused as to why the 95% confidence interval bounds seem to be different between Figure 5a and 5b. If the confidence bounds are calculated from the true population of 3411 measurements, then shouldn’t they be identical between the two figures because they are independent of the sample size of the randomly chosen grains? If the confidence bounds are not based on the entire population of rocks, can you please explain this more in the text because as currently written you state “along the with 95% grain size confidence interval bracketing the true grain size population”. Also, why do the confidence intervals not extend though the entirety of the distribution (the tails of the distributions are missing confidence bounds)?

Figure 6. Please see my comment in Figure 5, why do the confidence bounds not extend to all grain size percentiles and seem to end at about 5 and 95 percentiles? I think there is also an error in labeling this figure (three of the symbols/lines are labeled with 100).

Figure 7. Although I generally agree with what is stated in the text about this figure, I am not clear how the comparisons between the three different distributions are being made here. The differences between pools, bars and riffles are not being individually discussed in terms of confidence intervals (paragraph on page 14) and you just state that the “samples” are or are not different. Are you somehow pooling all samples in this comparison, analogous to an ANOVA test and if so, please specify this? If not, please specify which of the distributions you mean are actually statistically different in this discussion rather than just saying “samples”. In Figure 7, it seems to me that most grain size percentiles could be different for the bars than for the pools or riffles, but this is not discussed until the paragraph on page 15. If bars are different from pools and riffles, then I do not understand what all of the various statistical comparisons of “samples” mean in the paragraph on page 14; that you state many of “samples” grain size percentiles are not different on page 14 but on page 15, you state that bars are different from the other two locations. I am also unclear as to why significance at 99% is now being used as the level for significance when throughout the rest of the paper, 95% was used. Why is a stricter significance level now being included?

Figures 8 and 9. Why is the Bonferroni correction only applied when comparing these data and not any of the other comparisons in the paper, and what is m in these cases (each percentile compared?)?

P20 L9-19 Thank you for answering some of my questions about these data but I have a few remaining concerns. For example, how was visual observation of tracer movement conducted? I assume this was not during the flow that actually moved the tracers and that you simply observed that tracers were in different locations before and after a hydrograph and that the assumed critical discharge was the peak discharge value of that hydrograph? If so, can you please specify this because “visual observations of tracer stone movement” are usually impossible during sediment transport events and is somewhat misleading.

Hide

RR by Rebecca Hodge (10 Jul 2019)

Suggestions for revision or reasons for rejection

This paper presents a new approach to estimating confidence intervals for grain size distributions. I agree with the authors’ overall message that we should consider confidence intervals when analysing grain size distributions, and think that this paper and the associated toolbox will be a useful prompt in getting researchers to do this. This is a revised paper, though I did not see the original version. Overall I found the paper to be fairly clear and easy to understand, though there were some areas (identified below) that might benefit from a bit more signposting.

One comment I had was about the assumptions made in the model. It is assumed that the probability of sampling grains larger than D50 is 0.5, with the statement that “half the surface grains are smaller” than D50. Both are not necessarily true. D50 is the median grain size sampled using whatever sampling technique is applied. However, if the grains are sampled on a grid, then more than half of the surface grains (by number) could be smaller than D50. The discrepancy is because larger grains are more likely to be sampled than smaller ones as they take up more space and so there is a higher probability of a grid node or foot landing on them. In the case of equal numbers of large and small grains, the larger grains would occupy more than half the surface area and be more likely to be sampled. I don’t think that this sampling bias affects your analysis, but it needs more careful wording in the section around page 5, line 5. (See the Bunte and Abt 2001 technical report, section 4.3, for converting between distributions collected using different sampling techniques, e.g. grid to area.)

From a quick look through the response to reviewers, it looks like the authors have worked on incorporating previous literature. There were some places where this could have been developed a bit further. For example, demonstrating the range of different recommendations that are currently in the literature. Could you have also compared the results of your bootstrapping with the findings of Rice and Church, rather than just saying that you used the same method?

Comments by page/line:
1/15: Make it clear that the spreadsheet is available with this paper?
2/19: The problem with image based analysis is that you don’t know whether you are seeing the b-axis, which could introduce a different bias into the data.
3/3: Can you demonstrate how different the results from the different empirical analyses are, e.g. in terms of % error in D50?
3/11: It might be helpful to summarise what Fripp and Diplas presented, e.g. the sample size suggested for a given level of precision.
3/17: Not clear who ‘they’ is referring to.
3/28: This issue of overlapping intervals that don’t include both estimates will not be unique to analysing grain size data. Why can’t we use methods that have been developed by other disciplines to address it?
4/13: Move bracket to before 1993.
5/16: It took me a couple of reads to get my head round this; it might be useful to add an additional statement (as you do later on) along the lines of ‘In the case that 60 stones are smaller than D50, then d60 = D50’.
7/3: After a clear overview, I wasn’t sure where these subsections (2.1.1 onwards) were going. Can you add a bit more signposting? In particular, I wasn’t sure what the aim of this paragraph was.
7/10: It seemed a bit odd to be referring to a confidence interval, when you hadn’t yet addressed how it was calculated.
7/13: I needed a bit more help with comparing the two different approaches in 2.1.2 and 2.1.3. The differences seem to be that 2.1.2 gives asymmetric intervals and is mapped to specific grain size measurements. 2.1.3 seems to be symmetric, and allows interpolation between grain size measurements. When would you use the different approaches? Could you have a version that was asymmetric, but allowed interpolation?
11/17: Change to ‘measurements’.
14/Fig.7: How have the polygons been calculated, i.e. which percentiles have confidence intervals been calculated for? Also, add something to the caption to explain why 22.6 mm is highlighted.
14/16: Some of this explanation was a bit confusing, because the earlier analysis only refers to comparisons between two GSDs, but there are three in Fig. 7. Are all the significance values for a three-way comparison? I was surprised that the text didn’t report more differences between the bars and the other two units as there seems to be almost no overlap in the figure.
18/9: In this line and the next, clarify that you are referring to the mean ϵ. It might be helpful to give the range as well.
18/12: Change to ‘wide range of’.

Hide

ED: Publish subject to minor revisions (review by editor) (15 Jul 2019) by Jens Turowski

AR by Brett Eaton on behalf of the Authors (24 Jul 2019) Author's response Manuscript

ED: Publish as is (25 Jul 2019) by Jens Turowski

ED: Publish as is (29 Jul 2019) by Tom Coulthard (Editor)

AR by Brett Eaton on behalf of the Authors (29 Jul 2019) Author's response Manuscript

Short summary

Researchers studying gravel bed rivers almost always require a sample of the bed surface grain size range. These samples are typically expressed as cumulative frequency distributions. We present statistical techniques for generating and plotting confidence intervals for the various size percentiles and for determining whether size percentiles from two samples are statistically different. The techniques are implemented in an R package; a simplified version is implemented in a spreadsheet.