Using theory of change to assess impact of knowledge translation initiatives

abstract

Abstract Issue The National Collaborating Centre for Methods and Tools (NCCMT) offers high quality supports to public health professionals on evidence-informed decision making (EIDM). We developed a theory of change evaluation framework to guide evaluations of our programs and services to ensure that the NCCMT's resources and training continue to be high quality and meet the needs of public health professionals globally. Description of the Problem Our objective was to develop an evaluation framework that illustrates our reach, quality, and impact. The NCCMT contracted an external program evaluation expert to assist in developing a revised framework to better capture these aspects. An inductive approach was used to identify evaluation themes through internal document review, and a review of applicable program evaluation literature. Results A revised evaluation framework, influenced by theory of change, was developed and is being implemented at the NCCMT. The theory of change visualizes how the NCCMT influences EIDM and public health practice through four stages that progress from creation of products and services, to knowledge and awareness, to engagement and capacity development, and finally to policy and practice changes embedded within organizational structures. Specific outcome indicators include: public health professionals' confidence, knowledge, and skills; improved public health programs; organizational support for evidence-informed decision making; and workforce engagement and competence. Lessons This innovative evaluation framework informed by a theory of change resulted in significant streamlining of the centre's evaluation activities, and has provided clearer direction with respect to the purpose of evaluation, as well as enhanced ability to capture the centre's reach, quality and impact. The lessons of the importance of using theory of change to underpin evaluation activities can be applied to other public health organizations globally. Key messages Applying an evaluation framework supports effective and efficient programming. The evaluation framework demonstrates the NCCMT’s impact and reach in public health.