Do trials reduce uncertainty? Assessing impact through cumulative meta-analysis of neonatal RCTs

Objective:To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews.Study Design:We selected meta-analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact.Results:For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ⩾20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference.Conclusion:Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.