Turning words into numbers: establishing an empirical cut score for a letter graded examination

BACKGROUND: High stakes postgraduate specialist certification examinations have considerable implications for candidates' future careers. The cut score i.e. pass/fail mark of such examinations needs to be determined in a defensible and credible manner. A number of methods, suitable for use with numeric scoring methods, have been described. Determining the cut score of letter-graded examinations is, however, not described in the literature. AIM: The aim of this study was to determine a defensible and credible method for deriving the cut score of a letter-graded examination. METHOD: The cut score of the Fellowship examination of the College of Physicians of South Africa was estimated using a novel method. This method was validated by comparing the results obtained to those obtained using the contrasting groups method. RESULTS: By using the examiners' decision as the 'gold standard' we found that a cut score of 50% best approximated the cutpoint of this letter-graded examination, achieving a sensitivity and specificity of 83.7% and 82.8% respectively. CONCLUSION: This paper describes a useful strategy for estimating the cut score of letter-graded examinations.