Problem checklists and standardized diagnostic interviews: evidence of psychometric equivalence for classifying psychiatric disorder among children and youth in epidemiological studies

BACKGROUND: The standard approach for classifying child/youth psychiatric disorder as present or absent in epidemiological studies is lay-administered structured, standardized diagnostic interviews (interviews) based on categorical taxonomies such as the DSM and ICD. Converting problem checklist scale scores (checklists) to binary classifications provides a simple, inexpensive alternative. METHODS: Using assessments obtained from 737 parents, we determine if child/youth behavioral, attentional, and emotional disorder classifications based on checklists are equivalent psychometrically to interview classifications. We test this hypothesis by (1) comparing their test-retest reliabilities based on kappa (κ), (2) estimating their observed agreement at times 1 and 2, and (3) in structural equation models, comparing their strength of association with clinical status and reported use of prescription medication to treat disorder. A confidence interval approach is used to determine if parameter differences lie within the smallest effect size of interest set at ±0.125. RESULTS: The test-retest reliabilities (κ) for interviews compared with checklists met criteria for statistical equivalence: behavioral, .67 and .70; attentional, .64 and .66; and emotional, .61 and .65. Observed agreement between the checklist and interviews on classifications of disorder at times 1 and 2 was, on average, κ = .61. On average, the β coefficients estimating associations with clinical status were .59 (interviews) and .63 (checklists); and with prescription medication use, .69 (interviews) and .71 (checklists). Behavioral and attentional disorders met criteria for statistical equivalence. Emotional disorder did not, although the coefficients were stronger numerically for the checklist. CONCLUSIONS: Classifications of child/youth psychiatric disorder from parent-reported checklists and interviews are equivalent psychometrically. The practical advantages of checklists over interviews for classifying disorder (lower administration cost and respondent burden) are enhanced by their ability to measure disorder dimensionally. Checklists provide an option to interviews in epidemiological studies of common child/youth psychiatric disorders.