Comparison of manual and automated marking of student programs

Experiments are described with Automark, a prototype software package designed to mark student programs written in fortran-77. Correlations among marks assigned by Automark, student markers, and (presumably expert) instructors are analyzed. When the automark parameters are set to emphasize economy, modularity, simplicity and structure, a negative correlation with student marking is found; on the other hand, student marking correlates positively with documentation and layout. Marks assigned by students and by instructors correlate fairly highly. The conclusion is suggested that human markers place undue emphasis on the cosmetic aspects of programs; the role of Automark and similar systems may therefore best be viewed as the provision of a quantitative basis for the evaluation of the substantive aspects of computer programs.