Agreement Between Physician Evaluation and the Composite Response Index in Diffuse Cutaneous Systemic Sclerosis

OBJECTIVE: Diffuse cutaneous systemic sclerosis (SSc) is a highly heterogeneous disease. A provisionally approved Composite Response Index in diffuse cutaneous SSc (CRISS) was developed as a 1-year outcome measure for clinical trials. Our goal was to further validate the CRISS by examining agreement between CRISS definitions for improved/non-improved with physicians' evaluation of disease. METHODS: Patient profiles from a large observational cohort were created for 50 random diffuse cutaneous SSc patients of <5 years disease duration with improved CRISS scores after 1 year and 50 with non-improved CRISS scores. Profiles described disease features used during the initial CRISS development at baseline and at 1 year. Each profile was independently rated by 3 expert physicians. Majority opinion determined whether a patient was improved or not improved, and kappa agreement with the CRISS cutoff of 0.6 was calculated. RESULTS: Patients had mean ± SD disease duration of 2.2 ± 1.3 years. There was substantial agreement between the physician majority opinion about each case and the CRISS (κ = 0.76 [95% confidence interval (95% CI) 0.64-0.88]). The agreement between each individual physician opinion and the CRISS was also substantial (κ = 0.70 [95% CI 0.62-0.78]). All CRISS non-improvers were also rated as non-improved by physician majority; however, 12 CRISS improvers were rated as non-improved by physicians. CONCLUSION: There was substantial agreement between the dichotomous CRISS rating and physician assessment of diffuse cutaneous SSc patients after 1 year. This supports the use of a CRISS cutoff at 0.6 for improvement versus non-improvement, although the CRISS tended to rate more patients as improved than did physicians.