The MacTRAUMA TTL Assessment Tool: Developing a Novel Tool for Assessing Performance of Trauma Trainees: Initial Reliability Testing

OBJECTIVES: To develop a novel assessment tool for trainees-led trauma resuscitation. Assess psychometric properties of the proposed tool. Evaluate feasibility and utility of the tool. INTRODUCTION: Trauma resuscitation is a structured and complex process involving unique sets of skills. There is currently no published structured formative evaluation tool for trauma trainees. Therefore, many trauma trainees rely upon limited, unstructured feedback on their performance. We developed a tool to assess trainee performance while leading a trauma resuscitation and to assist faculty in providing trainee feedback after the encounter. METHODS: This study was conducted in a level I trauma centre in Ontario, Canada. Principles of learning theories, literature review, and clinical expert opinions were used to design a tool to assess clinical competence required to lead the resuscitation. In total, 5 critical domains were identified. High-fidelity simulation-based environment was used to test interrater reliability using intraclass correlation coefficients. To gauge feasibility, practicality, and utility of the tool, an online survey was sent to raters and trainees at the end of the study. RESULTS: We found "excellent" agreement for "initial critical assessment" domain (0.80) and "moderate to good" agreement for the "communication and leadership" (0.67) and "clinical performance" domains (0.53). "Poor" agreement was identified for the "decision-making" domain (0.33). The coefficients for individual items reached "good" agreement for 5 items, and "moderate" agreement for 8 items. Intraclass correlation coefficients for the remaining 7 items were "fair" or "poor." Most raters agreed that items in the medical training domain were not applicable. Feedback from raters and trainees confirmed the feasibility and acceptability of the tool for formative feedback, in addition to some suggestions to enhance the tool. CONCLUSION: MacTrauma TTL assessment tool is a novel tool for formative feedback for trainees' performance during trauma resuscitation. Initial psychometric property testing is promising. Further reliability and validity testing of the modified tool is needed. The tool has been shown to be feasible and acceptable by both trainees and faculty as a formative assessment tool.