Strategies to Manipulate Reliability: Impact on Statistical Associations

OBJECTIVE: To examine the effects of improving measurement reliability on associations between risk factors and childhood psychiatric disorder. METHOD: Data were from a general population sample of parents (N = 211) with children aged 6 to 16 years. Reliability of measurement was improved in three ways: by increasing the number of items in a scale (internal-consistency reliability), by averaging assessments of the same variables collected on two different occasions, and by constructing latent variable measures. To assess the effects of improving reliability, selected risk factors were regressed on parental assessments of childhood oppositional defiant disorder (ODD) and overanxious disorder (OAD). RESULTS: Improving reliability led to systematic increases in the magnitude of standardized regression coefficients between family dysfunction and ODD (beta = .30-.51) and between family dysfunction and OAD (beta = .24-.48). In multiple regression, improving reliability served to strengthen the specificity of associations between ODD, OAD, and family dysfunction and maternal depressed mood. Although latent variable methods produced the largest associations, the standard errors of these estimates were also larger, resulting in wider confidence intervals and slightly larger significance values. CONCLUSIONS: Improving reliability of measurement results in larger associations between risk factors and childhood disorder and may increase the opportunity of revealing differential associations between variables.