Should unpublished data be included in meta-analyses? Current convictions and controversies
Journal Articles
Overview
Research
Identity
Additional Document Info
View All
Overview
abstract
OBJECTIVE: To identify the extent to which meta-analyses currently include unpublished data and whether editors, meta-analysts, and methodologists believe unpublished material should be included. DESIGN: This article describes two related studies: a literature review and a cross-sectional survey. SAMPLE SELECTION: For the literature review, we identified all articles indexed by the key word meta-analysis from January 1989 to February 1991 and determined whether unpublished material had been searched for, obtained, and included in the meta-analyses. For the cross-sectional survey, we surveyed authors of these meta-analyses, authors of articles addressing methodological issues in meta-analysis published during the same period, and editors of journals in which both types of articles were published. INTERVENTION: We asked the respondents about their attitudes concerning inclusion of unpublished data and publication of articles from which data had previously been included in a scientific overview. MAIN OUTCOME MEASURES: Inclusion of unpublished data and opinions about whether unpublished material should be included in overviews and whether prior inclusion of data in an overview should bear on publication. RESULTS: Of 150 meta-analyses, 46 (30.7%) included unpublished data in their primary analysis. Of authors surveyed, 85% responded. Of the meta-analysts and methodologists, 77.7% felt that unpublished material should definitely or probably be included in scientific overviews; this was true of 46.9% of the editors. A total of 86.4% of the meta-analysts and methodologists and 53.2% of the editors felt that inclusion of data in a prior overview should have no bearing on full publication of original research. CONCLUSION: While inclusion of unpublished data in scientific overviews remains controversial, most investigators directly involved in meta-analysis believe that unpublished data should not be systematically excluded. The most valid synthesis of available information will result when meta-analysts subject published and unpublished material to the same rigorous methodological evaluation and present results with and without unpublished sources of data.
