Procedures that assess inconsistency in meta-analyses can assess the likelihood of response bias in multiwave surveys

BACKGROUND AND OBJECTIVE: Response bias may affect the result of surveys with <100% response rate. We applied methods commonly used in meta-analysis to ascertain the extent to which response bias affects multiwave survey results. METHODS: To test hypotheses of between-wave similarity, we used the Cochran-Armitage test for trends and the Q-test of heterogeneity across waves in a survey of 2,127 North American clinicians using six e-mail waves and one fax wave and achieving a response rate of 22%. We used the I2 statistic To quantify the extent of inconsistency in survey outcomes across waves not due to within-wave random error (i.e., inconsistency due to response bias). RESULTS: With this survey, tests of heterogeneity and trend were not significant and I2 equaled 0%. These results suggest that the underlying responses did not differ across waves and thus strengthened the inference that response bias was not affecting the interpretation of the survey. CONCLUSION: Researchers can use procedures that assess inconsistency in meta-analyses to evaluate the validity of a multiwave survey with a less than optimal response rate.