An efficient strategy allowed English-speaking reviewers to identify foreign-language articles eligible for a systematic review Academic Article uri icon

  • Overview
  • Research
  • Identity
  • Additional Document Info
  • View All


  • OBJECTIVE: To assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review. STUDY DESIGN AND SETTINGS: Systematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non-English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility. RESULTS: Of 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity=0.89; specificity=0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word "random" appears) led to accurate classification of 51 of 53 articles (sensitivity=0.96; specificity=0.70). CONCLUSION: Our findings show promise for limiting the need for non-English-language review teams in systematic reviews with large numbers of potentially eligible non-English-language articles.


  • Busse, Jason Walter
  • Bruno, Paul
  • Malik, Keshena
  • Connell, Gaelan
  • Torrance, David
  • Ngo, Trung
  • Kirmayr, Karin
  • Avrahami, Daniel
  • Riva, John
  • Ebrahim, Shanil
  • Struijs, Peter AA
  • Brunarski, David
  • Burnie, Stephen J
  • LeBlanc, Frances
  • Coomes, Eric A
  • Steenstra, Ivan A
  • Slack, Tesha
  • Rodine, Robert
  • Jim, Janey
  • Montori, Victor M
  • Guyatt, Gordon

publication date

  • May 2014