Use of screening phase data to evaluate observer variation of sputum cytodiagnosis as an outcome measure in a chemoprevention trial.

Sputum samples obtained during the screening phase of a chemoprevention trial in heavy smokers were evaluated independently by trained cytotechnologists and classified for degree of cellular atypia according to the method of Saccommano et al. (G. Saccommano et al., Cancer (Phila.), 33: 256-270, 1974). The level of agreement within and between Observers A and B was calculated as the percentage of agreement and, in addition, a statistic was used (kappa) to correct for chance-expected agreement. Between observer agreement on 300 specimens from 130 subjects was 68% (204 of 300) (kappa = 0.58). Of the 96 disagreements, only 17 were of more than one category in the six-category classification. Within observer agreement for both Observers A and B was evaluated on a subset of 60 specimens from 49 subjects examined on two separate occasions by each observer. The percentage of within observer agreement was 80% for Observer A (kappa = 0.73) and 62% for Observer B (kappa = 0.49) (P less than 0.04). Altogether, 71% (25 of 35) of within observer discordant readings were confined to only one category. These data, obtained from the screening phase of the study, will allow us to refine the outcome measure for the trial without introducing bias that could result from knowledge of the actual study results. The failure of conventional cancer treatments to impact significantly on overall cancer mortality has led to an emphasis on prevention. If premalignant changes can be reliably detected at a time when the malignant process is reversible, then it may be feasible to intervene to prevent the occurrence of some cancers. The appropriate selection of subjects for such intervention trials and the subsequent demonstration of the efficacy of chemoprevention strategies are therefore crucial.