Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability

In a previous article the distinction is made between objectivity and objectification. Objectivity is considered a generic goal of measurement, marked by freedom of subjective influences in general, whereas the latter term is used to describe strategies to reduce measurement error. A survey of several studies indicated that objectified methods are not intrinsically more reliable than subjective measures. In this paper the consequences of objectification are analysed for issues related to validity, efficiency, transparency, and effect of these methods on students and teachers. Several studies comparing objectified and subjective methods are surveyed for this propose. The studies indicate that--as in the previous article on reliability--objectification and objectivity are not identical, and that there are many pitfalls in the objectification of measurement procedures. As a consequence, it is argued that objectified methods should not exclusively be chosen on the basis of their unconditional appeal to objectivity, but that the application of measurement methods should follow the specific purpose of the testing situation. In the context of the testing situation, arguments against and in favour of objectification should be weighted, and trade-offs are to be evaluated. The outcome of this evaluation may vary from situation to situation, and from institution to institution.