Defensibility of Competency Assessments of Practising Family Physicians

When a physician’s licence to practice is at stake, professional acceptance and legal challenge are concerns for an organization undertaking competency assessments for practising physicians. The Physician Review and Enhancement Program (PREP), a program of McMaster University sponsored by the College of Physicians and Surgeons of Ontario, underwent an external review by evaluation experts. As a result, one of the 4 assessment tools, the Structured Office Oral, was dropped, as it was insufficiently structured to be reliable, and because it did not add significantly to the other tools. The content of all assessment tools was revised based on a PREP-developed blueprint for family practice. The MCQs were upgraded through collaboration with the Canadian physician accrediting body, the Medical Council of Canada (MCC), by PREP criterion physicians choosing MCQs, from the MCC question bank. The Standardized Patient Assessment was refined by PREP criterion physicians developing standardized clinical case scenarios with predefined performance criteria. Finally, through collaboration with the American Board of Emergency Medicine, Chart Stimulated Recall was restructured to ensure objectivity in standardization and interpretation. The result is a more standardized and structured program.