The Effects of Publication Selection on Test Probabilities and Estimator Distributions

Editorial decisions, based in part on reported hypothesis test results, affect the probabilities associated with those results: the probabilities of Type I and Type II errors thus become different for readers than for authors. The distributions of published parameter estimates are similarly affected. A framework for studying the consequences of test‐based information filtering is developed and illustrative examples are provided. The examples indicate that filtering can markedly distort the power functions of hypothesis tests and can induce large estimator biases and increases in mean square error. It is argued that test‐based filtering is relevant not only to journal publication but to other forms of information dissemination as well.