Performance of methods for analyzing continuous data from stratified cluster randomized trials – A simulation study

Background: The adoption of cluster randomized trials (CRTs) with the stratified design is currently gaining widespread interest. In the stratified design, clusters are first grouped into two or more strata and then randomized into treatment groups within each stratum. In this study, we evaluated the performance of several commonly used methods for analyzing continuous data from stratified CRTs. Methods: This is a simulation study where we compared four methods: mixed-effects, generalized estimating equation (GEE), cluster-level (CL) linear regression and meta-regression methods to analyze the continuous data from stratified CRTs using a simulation study with varying numbers of clusters, cluster sizes, intra-cluster correlation coefficients (ICCs) and effect sizes. This study was based on a stratified CRT with one stratification variable with two strata. The performance of the methods was evaluated in terms of the type I error rate, empirical power, root mean square error (RMSE), and width and coverage of the 95% confidence interval (CI). Results: GEE and meta-regression methods had high type I error rates, higher than 10%, for the small number of clusters. All methods had similar accuracy, measured through RMSE, except meta-regression. Similarly, all methods but meta-regression had similar widths of 95% CIs for the small number of clusters. For the same sample size, the empirical power for all methods decreased as the value of the ICC increased. Conclusion: In this study, we evaluated the performance of several methods for analyzing continuous data from stratified CRTs. Meta-regression was the least efficient method compared to other methods.