The multi-level impact of clinical decision support system: A framework and a call for mixed methods evaluation

Clinical decision support systems are important healthcare systems that help in improving healthcare quality and in reducing cost. These systems have multiple levels of outcomes on the individual, group, organization, and society levels. However, despite the abundant studies conducted to evaluate CDSS, most of these studies failed to recognize these multiple levels of impact and mostly focused on the clinical efficacy of these systems using randomized controlled trials (RCT) designs. In this paper, we propose that CDSS evaluation is a complex task that cannot be accomplished using a single research methodology. We propose a framework that identifies the multiple levels impacted by CDSS and the appropriate methodologies that can be used to evaluate these impacts. We also provide an example of using this framework to evaluate a pain management CDSS. This study informs future evaluation studies on what and how to evaluate at different levels of CDSS interventions.