Using conventional and machine learning propensity score methods to examine the effectiveness of 12-step group involvement following inpatient addiction treatment

BACKGROUND: Continuing care following inpatient addiction treatment is an important component in the continuum of clinical services. Mutual help, including 12-step groups like Alcoholics Anonymous, is often recommended as a form of continuing care. However, the effectiveness of 12-step groups is difficult to establish using observational studies due to the risks of selection bias (or confounding). OBJECTIVE: To address this limitation, we used both conventional and machine learning-based propensity score (PS) methods to examine the effectiveness of 12-step group involvement following inpatient treatment on substance use over a 12-month period. METHODS: Using data from the Recovery Journey Project - a longitudinal, observational study - we followed an inpatient sample over 12-months post-treatment to assess the effect of 12-step involvement on substance use at 12-months (n = 254). Specifically, PS models were constructed based on 34 unbalanced confounders and four PS-based methods were applied: matching, inverse probability weighting (IPW), doubly robust (DR) with matching, and DR with IPW. RESULTS: Each PS-based method minimized the potential of confounding from unbalanced variables and demonstrated a significant effect (p <  0.001) between high 12-step involvement (i.e., defined as having a home group; having a sponsor; attending at least one meeting per week; and, being involved in service work) and a reduced likelihood of using substances over the 12-month period (odds ratios 0.11 to 0.32). CONCLUSIONS: PS-based methods effectively reduced potential confounding influences and provided robust evidence of a significant effect. Nonetheless, results should be considered in light of the relatively high attrition rate, potentially limiting their generalizability.