Testing against “normal” with environmental data

Normal ranges are some fraction of a reference distribution deemed to represent an expected condition, typically 95%. They are frequently used as the basis for generic criteria for monitoring programs designed to test whether a sample is outside of "normal," as in reference-condition approach studies. Normal ranges are also the basis for criteria for more classic environmental effects monitoring programs designed to detect differences in mean responses between reference and exposure areas. Limits on normal ranges are estimated with error that varies depending largely on sample size. Direct comparison of a sample or a mean to estimated limits of a normal range will, with some frequency, lead to incorrect conclusions about whether a sample or a mean is inside or outside the normal range when the sample or the mean is near the limit. Those errors can have significant costs and risk implications. This article describes tests based on noncentral distributions that are appropriate for quantifying the likelihood that samples or means are outside a normal range. These noncentral tests reverse the burden of evidence (assuming that the sample or mean is at or outside normal), and thereby encourage proponents to collect more robust sample sizes that will demonstrate that the sample or mean is not at the limits or beyond the normal range. These noncentral equivalence and interval tests can be applied to uni- and multivariate responses, and to simple (e.g., upstream vs downstream) or more complex (e.g., before vs after, or upstream vs downstream) study designs. Statistical procedures for the various tests are illustrated with benthic invertebrate community data collected as part of the Regional Aquatics Monitoring Program (RAMP) in the vicinity of oil sands operations in northern Alberta, Canada. An Excel workbook with functions and calculations to carry out the various tests is provided in the online Supplemental Data. Integr Environ Assess Manag 2017;13:188-197. © 2016 SETAC.