Inter‐observer reproducibility of the 2021 AAGL Endometriosis Classification

BACKGROUND: Inter-observer agreement for the American Association of Gynecologic Laparoscopists (AAGL) 2021 Endometriosis Classification staging system has not been described. Its predecessor staging system, the revised American Society for Reproductive Medicine (rASRM), has historically demonstrated poor inter-observer agreement. AIMS: We aimed to determine the inter-observer agreement performance of the AAGL 2021 Endometriosis Classification staging system, and compare this with the rASRM staging system. MATERIALS AND METHODS: A database of 317 patients with coded surgical data was retrospectively analysed. Three independent observers allocated AAGL surgical stages (1-4), twice. Observers made their own interpretation of how to apply the tool in the first staging allocation. Consensus rules were then developed for a second staging allocation. RESULTS: First staging allocation: odds ratio (OR) (and 95% CI) for observer 1 to score higher than observer 2 was 8.08 (5.12-12.76). Observer 1 to score higher than observer 3 was 12.98 (7.99-21.11) and observer 2 to score higher than observer 3 was 1.61 (1.03-2.51). This represents poor agreement. Second staging allocation (after consensus): OR for observer 1 to score higher than observer 2 was 1.14 (0.64-2.03), observer 1 to score higher than observer 3 was 1.81 (0.99-3.28) and observer 2 to score higher than observer 3 was 1.59 (0.87-2.89). This represents good agreement. CONCLUSIONS: These findings suggest that in its current format the AAGL 2021 Endometriosis Classification staging system has poor inter-observer agreement, not superior to the rASRM staging system. However, performance improved when additional measures were taken to simplify and clarify areas of ambiguity in interpreting the staging system.