PURPOSE: Although multiple independent sampling (MIS) has been adapted for admissions interviews, its application for assessing written materials in the admissions file has been limited. Currently, admissions file review at the University of Toronto medical school involves one rater per file to enable holistic assessment, which may introduce a halo effect-that is, impressions of one component influencing the evaluation of other components. The authors examined whether MIS file review, through which multiple raters evaluate specific file components independently, may reduce this effect. METHOD: The authors selected a stratified random sample of 300 applicant files from the 2010-2011 admissions cycle for rescoring by MIS. They divided each of the 300 applicant files into their four components (academic transcript, autobiographical sketch, personal statement, reference letters) and rebundled them into packages of 38 same-component items (purposely creating some overlap among packages to assess inter-rater reliability). The authors distributed each package to 1 of 36 raters; thus, each rater evaluated only one of four components across many applicants. The authors compared the inter-component reliability and factor analysis of MIS with that of holistic scoring. RESULTS: Ratings were returned for all applicants. Inter-component reliability (Cronbach alpha) was 0.69 for holistic scoring and 0.29 for MIS. Factor analysis showed all components loading heavily onto one factor in the holistic approach and onto three factors in the MIS method. CONCLUSIONS: Using MIS to assess the admissions file may reduce the halo effect and should be considered when evaluating applicants' written submissions.
