Confidence intervals of the relative risk and odds ratio can predict when the optimal information size in a meta-analysis is not met

OBJECTIVES: We sought to empirically evaluate whether the width of confidence interval (CI) of the relative risk (RR) and odds ratio (OR) can obviate the need for calculating the optimal information size (OIS) when making Grading of Recommendations Assessment, Development and Evaluation imprecision judgments. STUDY DESIGN AND SETTING: We analyzed a convenience sample of meta-analyses extracted from the Cochrane Database of Systematic Reviews. From each meta-analysis, we calculated OIS based on relative risk reductions (RRRs) of 15%-50% and evaluated the ratio of upper to lower 95% CI boundaries of RR (RR CI ratio) and OR (OR CI ratio). We calculated the positive predictive value (PPV) as the probability that a small CI ratio would correctly identify an OIS that was met, and the negative predictive value as the probability that a large CI ratio would correctly identify an OIS that was not met. RESULTS: We analyzed 39,569 meta-analyses (266,877 studies). Analyses across various RRR's demonstrated high NPV's and low PPV's, suggesting that using the CI approach is helpful in identifying when OIS would not be met, but not as helpful in 'ruling in' that OIS would be met. Cutoffs previously proposed for RR CI ratio (>2.5) and OR CI ratio (>3) had very high NPV's (>90%) except when the RRR used to calculate OIS was high (30%-50%) and when the baseline risk or number of included studies increased. CONCLUSION: This empirical analysis supports using CI ratios of RR and OR to 'rule out' meeting OIS (ie, to rate down the certainty of evidence due to imprecision). The CI approach is not helpful to 'rule in' meeting OIS and is not helpful when the plausible effect size is >30% RRR. PLAIN LANGUAGE SUMMARY: Sample size calculation have been done on meta-analyses akin to individual trials producing the optimal information size (OIS), which helps in determining if a meta-analysis has sufficient power. If the meta-analysis total sample size was less than the OIS, certainty in the results would be lowered. In this study, the ratios of upper to lower 95% confidence interval boundaries of relative risk and odds ratio were shown to predict when OIS would not be met.