A Prospective Study Measuring Resident and Faculty Contour Concordance: A Potential Tool for Quantitative Assessment of Residents' Performance in Contouring and Target Delineation in Radiation Oncology Residency

Abstract

Purpose/objective(s): Accurate target delineation (ie, contouring) is essential for radiation treatment planning and radiotherapy efficacy. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. The purpose of this study was to track the concordance of radiation oncology residents' contours with those of faculty physicians over the course of 1 year to assess for patterns.

Materials/methods: Residents in postgraduate year (PGY) levels 2 to 4 were asked to contour target volumes that were then compared to the finalized, faculty physician-approved contours. Concordance between resident and faculty physician contours was determined by calculating the Jaccard concordance index (JCI), ranging from 0, meaning no agreement, to 1, meaning complete agreement. Multivariate mixed-effect models were used to assess the association of JCI to the fixed effect of PGY level and its interactions with cancer type and other baseline characteristics. Post hoc means of JCI were compared between PGY levels after accounting for multiple comparisons using Tukey's method.

Results: In total, 958 structures from 314 patients collected during the 2020-2021 academic year were studied. The mean JCI was 0.77, 0.75, and 0.61 for the PGY-4, PGY-3, and PGY-2 levels, respectively. The JCI score for PGY-2 was found to be lower than those for PGY-3 and PGY-4, respectively (all P < .001). No statistically significant difference of JCI score was found between the PGY-3 and PGY-4 levels. The average JCI score was lowest (0.51) for primary head and/or neck cancers, and it was highest (0.80) for gynecologic cancers.

Conclusions: Tracking and comparing the concordance of resident contours with faculty physician contours is an intriguing method of assessing resident performance in contouring and target delineation and could potentially serve as a quantitative metric, which is lacking currently, in radiation oncology resident evaluation. However, additional study is necessary before this technique can be incorporated into residency assessments.

Keywords: Contouring; concordance; quantitative; resident assessment; target delineation.