Methodology and Reports of Systematic Reviews and Meta-analyses: A Comparison of Cochrane Reviews With Articles Published in Paper-Based Journals

CONTEXT: Review articles are important sources of information to help guide decisions by clinicians, patients, and other decision makers. Ideally, reviews should include strategies to minimize bias and to maximize precision and be reported so explicitly that any interested reader would be able to replicate them. OBJECTIVE: To compare the methodological and reporting aspects of systematic reviews and meta-analyses published by the Cochrane Collaboration with those published in paper-based journals indexed in MEDLINE. DATA SOURCES: The Cochrane Library, issue 2 of 1995, and a search of MEDLINE restricted to 1995. STUDY SELECTION: All 36 completed reviews published in the Cochrane Database of Systematic Reviews and a randomly selected sample of 39 meta-analyses or systematic reviews published in journals indexed by MEDLINE in 1995. DATA EXTRACTION: Number of authors, trials, and patients; trial sources; inclusion and exclusion criteria; language restrictions; primary outcome; trial quality assessment; heterogeneity testing; and effect estimates. Updating by 1997 was evaluated. RESULTS: Reviews found in MEDLINE included more authors (median, 3 vs 2; P<.001), more trials (median, 13.5 vs 5; P<.001), and more patients (median, 1280 vs 528; P<.001) than Cochrane reviews. More Cochrane reviews, however, included a description of the inclusion and exclusion criteria (35/36 vs 18/39; P<.001) and assessed trial quality (36/36 vs 12/39; P<.001). No Cochrane reviews had language restrictions (0/36 vs 7/39; P<.01). There were no differences in sources of trials, heterogeneity testing, or description of effect estimates. By June 1997, 18 of 36 Cochrane reviews had been updated vs 1 of 39 reviews listed in MEDLINE. CONCLUSIONS: Cochrane reviews appear to have greater methodological rigor and are more frequently updated than systematic reviews or meta-analyses published in paper-based journals.