Difficulties in clinical skills evaluation

A case-specific method of evaluating clinical skills is presented. The instrument is described together with its evaluation based on results from ninety-two medical students. The method involves direct observation, by pairs of teachers, of student encounters with simulated patients. The purpose of this paper is to explore the methodology of assessing clinical competence. Three aspects of this are addressed: reliability between teachers acting as observers, and the relationship of both observations of process measures and self-assessments with the students' understanding of the patient. Three aspects of clinical skills were evaluated; interviewing, problem orientation and physical examination. The results showed the observers to be highly reliable in observing physical examination but to have a wide range of reliability across cases in their observations of the other aspects of clinical skill. There was no correlation between both teachers' observations and the students' self-assessments with the students' understanding of the patient. This is a worrying finding. The implications of this study are that evaluation of the process of clinical skills is difficult methodologically and, added to which, it is not clear what relevance these observations have to clinical competence. Rather than effort being devoted to refining instruments to measure the process of clinical skills it would seem preferable to devote energy to assessing students' abilities in diagnosis and management, at least until we understand what observations of clinical skills are measuring.