Assessing the Variability and Quality of Lung Stereotactic Radiation Therapy Treatment Plans Using a Web-Based Crowdsourcing Platform

PURPOSE: The quality of stereotactic body radiation therapy (SBRT) treatment plans for early stage lung cancer are unknown outside of peer-reviewed publications. Thus, a study was conducted to crowdsource and analyze a variety of lung SBRT treatment plans from around the world. METHODS AND MATERIALS: This study had 2 parts, planning and contouring, and each was facilitated by a web-based technology platform. For planning, lung SBRT planners were invited to design, score, and submit their treatment plans (prescription of 11 Gy × 5) for a centralized stage I lung cancer case using standardized images and predefined contours. Each plan was scored with 20 weighted metrics adapted from currently recruiting phase 3 lung SBRT trials. For contouring, a separate image set was used to evaluate organ-at-risk contour accuracy using Dice coefficients and a StructSure score. RESULTS: For planning 227 plans were submitted in total with 7 different treatment planning systems and 7 different delivery methods represented. Variability was primarily user driven and not associated with the treatment planning system, delivery modality, total monitor units, or estimated beam-on time. Many of the highest-quality plans required the shortest amount of time to deliver, independent of the delivery technique. For contouring, organ-at-risk contours were frequently over- or undercontoured and often included only the luminal air of the trachea, proximal bronchial tree, and esophagus, even when the mucosal linings were within a few centimeters of the target tumor. CONCLUSIONS: These findings demonstrate the importance of quality assurance to help improve planning and contouring and the value of peer review and comparison. More readily accessible quality evaluation software solutions, such as the one used herein, may help meet this growing need.