Evaluation considerations: measures and methods
Conferences
Overview
Research
Identity
Additional Document Info
View All
Overview
abstract
There is a pressing need to effectively evaluate safety promotion actions worldwide, to inform the skeptics and funders about the real impact of injuries on societies in hindering their development, and the potential societal cost-benefits of the multiple activities and actions being carried out. The evaluation of programmatic or structural interventions for injury prevention or control is a complex activity. It can be carried out at different levels – the various components of a program or a program as a whole; in individuals in a specific community or a region, or at the community or regional level. There are multiple aspects of effectiveness, from process indicators, impact and outcome measures, to health related or financial considerations. Indicators can be quantified with different measures, and these in turn can be obtained through various types of experimental study designs. Of particular note is that most interventions tend to be carried out in community settings, and this impacts the choice of measures and study designs that are methodologically feasible and practical to be used. Process evaluation addresses the various aspects of the implementation of an intervention, and how well they worked. Impact evaluation, on the other hand, assesses changes in knowledge, attitudes, beliefs, behaviours and practices. The more commonly known measures are the outcome measures, since they address changes in behaviours directly related to injury events, injury events themselves (eg, frequency, type and pattern), morbidity and mortality. These measures are meaningful insofar as the study design used is appropriate. From a methodological perspective, sound evidence comes from controlled randomised experimental designs. Thus, experimental study designs such as randomised controlled trials, grouped randomised experiments, and community-randomised studies are judged as ‘better’ than quasi-experimental study designs such as non-randomised comparative studies, before-after studies, and observational studies like cohort studies, case–control studies and comparative cross-sectional studies. In this presentation, we review the various measures typically used and the most common types of study designs that provide statistical evidence on effectiveness. We contrast and compare the methodological advantages and limitations of the various choices. We also discuss incorporating ‘qualitative evidence’ to complement the quantitative indicators, as well as the consideration of economic indicators of effectiveness, such as disability adjusted life years (DALYs) or quality adjusted life years (QALYs). The challenge to us as active researchers in the injury control and safety promotion field is to develop convincing collective evidence of what aspects and what interventions work, and help reduce the societal burden from unnecessary injuries.
