Multi-methods development and validation of a tool for use in measuring serious illness communication competence: Assessment of clinical encounters – Communication tool (ACE-CT)

OBJECTIVES: The assessment of serious illness communication (SIC) competence has, to date, primarily utilized tools that are profession-specific and not explicitly designed using competency-based or person-centered frameworks. To address these gaps, we developed and validated a new tool, the Assessment of Clinical Encounters - Communication Tool (ACE-CT). METHODS: We convened a national panel of interprofessional SIC experts to develop and validate the ACE-CT using a three-phase multi-method approach. Phase 1 focused on item development through review of existing validated tools, and a Bayesian process in which panel members assessed item quality and item-domain correlation. Phase 2 involved item refinement and preliminary validation through stimulated recall interviews using a think-aloud technique. Phase 3 consisted of psychometric analyses for which panel members used the tool to assess video-recorded standardized patient encounters from interprofessional clinicians completing a SIC professional development intervention. RESULTS: In Phase 1, 37 relevant items from previously validated tools were identified, of which 11 items were removed due to redundance. Through the Bayesian process, 14 items were removed and 1 item was generated. Through Phase 2, 2 items were generated, 2 items were combined into 1, and remaining items were refined to optimize measurability and understandability. In Phase 3, reliability was demonstrated through evidence of high internal consistency and moderate reproducibility, both over time and across raters. The tool was found to be responsive and have sound construct validity through evidence of congruence, convergence and credibility. Raters found the tool to be intuitive, easy to complete, and that it accurately captured their perception of the quality of communication observed. CONCLUSIONS: The ACE-CT provides a reliable and valid approach to assessing SIC competence among interprofessional clinicians. PRACTICAL IMPLICATIONS: Through its person-centered orientation, the ACE-CT provides an opportunity to objectively assess elements of SIC that patients and families value.