Using decision thresholds for ranking treatments in network meta-analysis results in more informative rankings

OBJECTIVES: To evaluate how the rank probabilities obtained from network meta-analysis (NMA) change with the use of increasingly stringent criteria for the relative effect comparing two treatments which ranks one treatment better than the other. STUDY DESIGN AND SETTING: Systematic survey and reanalysis of published data. We included all systematic reviews (SRs) with NMA from the field of cardiovascular medicine that had trial-level data available, published in Medline up to February 2015. We reran all the NMAs and determined the probabilities of each treatment being the best. For the best treatment, we examined the effect on these probabilities of varying, what we call the decision threshold, the relative effect required to declare two treatments different. RESULTS: We included 14 SRs, having a median of 20 randomized trials and 9 treatments. The best treatments had probabilities of being best that ranged from 38% to 85.3%. The effect of changing the decision thresholds on the probability of a treatment being best varied substantially across reviews, with relatively little decrease (∼20 percentage points) in some settings but a decline to near 0% in others. CONCLUSION: Rank probabilities can be fragile to increases in the decision threshold used to claim that one treatment is more effective than another. Including these thresholds into the calculation of rankings may aid their interpretation and use in clinical practice.