Grading recommendations in clinical practice guidelines: randomised experimental evaluation of four different systems

OBJECTIVE: To evaluate the effect of presenting a recommendation in a clinical practice guideline using different grading systems to determine to what extent the system used changes the clinician's eventual response to a particular clinical question. DESIGN: Randomised experimental study. SETTING: Clinician offices and academic settings. PARTICIPANTS: Paediatricians and paediatric residents in private and public practice in Mexico. INTERVENTION: Case notes of a child with diarrhoea and a question about clinician preference for using racecadotril. The same evidence was provided in a clinical recommendation but with different presentations according to the following grading systems: NICE (National Institute for Health and Clinical Excellence), SIGN (Scottish Intercollegiate Guideline Network), GRADE (Grading of Recommendations Assessment, Development and Evaluation) and CEBM (Centre for Evidence-Based Medicine, Oxford). MAIN OUTCOME MEASURE: Mean change in direction from baseline response (measured on a 10 cm visual scale and a Likert scale) and among groups. RESULTS: 216 subjects agreed to participate. Most participants changed their decision after reading the clinical recommendations (mean difference 0.7 cm, 95% CI 0.29 to 1.0; p<0.001). By groups, mean change (95% CI) from baseline was 0.04 (-0.68 to 0.77) for NICE, 0.31 (-0.41 to 1.05) for SIGN, 2.18 (1.48 to 2.88) for GRADE and 0.08 (-0.52 to 0.69) for CEBM (p=0.007 between groups). In a final survey, a small difference was noted regarding the clarity of the results presented with the GRADE system. CONCLUSION: The clinician's decision to use a therapy was influenced most by the GRADE system. TRIAL REGISTRATION NUMBER: NCT00940290.