Network vs. Pairwise Meta-Analyses: A Case Study of the Impact of an Evidence-Synthesis Paradigm on Value of Information Outcomes

ObjectiveTo evaluate the impact of using two evidence-synthesis paradigms, pairwise meta-analysis (PMA) vs. network meta-analysis (NMA), on the expected value of information (EVI) outcomes, using pharmacotherapy of chronic obstructive pulmonary disease as a case study.MethodsBayesian random-effects PMAs were performed for each pharmacotherapy vs. placebo, and a Bayesian random-effects NMA was performed combining both placebo-controlled and head-to-head trials. Both provided comparative rate ratio (RR) estimates between each pharmacotherapy vs. placebo. A Markov model was developed to project costs and quality-adjusted life-years of five commonly used treatments for chronic obsructive pulmonary disorder. RRs for the treatment effect compared with placebo derived using PMA and NMA were used alongside values from the literature to populate the model. In addition to standard cost-effectiveness outputs, we calculated and compared the expected value of perfect information (EVPI) and the expected value of partial perfect information (EVPPI) for treatment effects, for comparisons that included all or a subset of treatments.ResultsThe network of evidence included five different treatments, compared in 19 randomized controlled trials (RCTs), which in total included 28,172 individuals. The cost-effectiveness outcomes were similar between the two evidence-synthesis paradigms. The individual EVPI for all treatments was Can$1,262 for PMA-based analyses and Can$572 for NMA-based analyses. For all comparisons involving two, three, or four treatments, the comparison with the highest EVPI was different between the two methods. Similarly, the choice of PMA or NMA had resulted in substantially different EVPPI rankings.ConclusionOur case study shows that the choice of PMA or NMA can have significant effects on the EVI results. Under comparable conditions, the incorporation of more evidence in the NMA most likely increases the precision of estimates and therefore is likely to result in lower EVI outcomes. As our study demonstrates, the difference in EVI outcomes can be substantial, potentially affecting the decision to conduct research and the design of future research.