Evaluating Information in Ill-Structured Decision Environments

The issue of evaluating information in an ill-structured decision environment is examined by comparing four alternative methods. Two are derived from the cognitive maps of the decision-makers: one is based on the intrinsic variety of the factor being evaluated, and the other on the number of links in the cognitive map that are affected by the factor's removal. The other two involve Likert scale and rank-ordering questions to measure the value of the factor directly. The four methods were tested using 10 district claim managers of an insurance company. The task involved evaluating the performance of their subordinates. The results were inconclusive, as no one measure clearly dominated the others. However, analyses of post-experiment interviews suggest that the decision-makers' evaluation of the information content of the relevant factors was inversely related to the ease with which they could infer the state (value) of a factor.