A guide and pragmatic considerations for applying GRADE to network meta-analysis

Assessing the certainty of evidence from network meta-analyses using the GRADE (grading of recommendations, assessment, development, and evaluations) approach requires not only a thorough understanding of the methods but also substantial workload for raters. This article describes how implementing practical strategies (including rule setting and automation) can facilitate efficient application of the GRADE approach to rating certainty of evidence in network meta-analyses while maintaining rigor. This article describes a stepwise strategy for proceeding through the process, including assessment of the certainty of direct, indirect, and NMA estimates; developing directions for each step in the process; implementation of rules to improve robustness of information appraisal to reduce workload; and alternatives for automation of some of the required steps. The presented approach includes a detailed description of every step of the process supported by figures, tables, and real life examples. To facilitate implementation of this process, a spreadsheet that incorporates automation of several of the steps described is also provided (https://www.covid19lnma.com/).