Answering medical questions at the point of care: a cross-sectional study comparing rapid decisions based on PubMed and Epistemonikos searches with evidence-based recommendations developed with the GRADE approach

INTRODUCTION: Using the best current evidence to inform clinical decisions remains a challenge for clinicians. Given the scarcity of trustworthy clinical practice guidelines providing recommendations to answer clinicians' daily questions, clinical decision support systems (ie, assistance in question identification and answering) emerge as an attractive alternative. The trustworthiness of the recommendations achieved by such systems is unknown. OBJECTIVE: To evaluate the trustworthiness of a question identification and answering system that delivers timely recommendations. DESIGN: Cross-sectional study. METHODS: We compared the responses to 100 clinical questions related to inpatient management provided by two rapid response methods with 'Gold Standard' recommendations. One of the rapid methods was based on PubMed and the other on Epistemonikos database. We defined our 'Gold Standard' as trustworthy published evidence-based recommendations or, when unavailable, recommendations developed locally by a panel of six clinicians following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Recommendations provided by the rapid strategies were classified as potentially misleading or reasonable. We also determined if the potentially misleading recommendations could have been avoided with the appropriate implementation of searching and evidence summary tools. RESULTS: We were able to answer all of the 100 questions with both rapid methods. Of the 200 recommendations obtained, 6.5% (95% CI 3% to 9.9%) were classified as potentially misleading and 93.5% (95% CI 90% to 96.9%) as reasonable. 6 of the 13 potentially misleading recommendations could have been avoided by the appropriate usage of the Epistemonikos matrix tool or by constructing summary of findings tables. No significant differences were observed between the evaluated rapid response methods. CONCLUSION: A question answering service based on the GRADE approach proved feasible to implement and provided appropriate guidance for most identified questions. Our approach could help stakeholders in charge of managing resources and defining policies for patient care to improve evidence-based decision-making in an efficient and feasible manner.