Early Emergency Medicine Milestone Assessment for Predicting First-Year Resident Performance

Abstract

Introduction: The Accreditation Council for Graduate Medical Education (ACGME) requires emergency medicine (EM) residency training programs to monitor residents' progress using standardized milestones. The first assessment of PGY 1 resident milestones occurs midway through the first year and could miss initial deficiencies. Early assessment of PGY 1 EM resident milestones has potential to identify at-risk residents prior to standard midyear evaluations. We developed an orientation syllabus for PGY 1 residents followed by a milestone assessment. Assessment scores helped predict future milestone scores and American Board of Emergency Medicine (ABEM) In-Training Examination (ITE) scores for PGY 1 residents.

Methods: From 2013 to 2020, we developed and implemented Milestone Evaluation Day (MED), a simulation-based day and written exam assessing PGY 1 EM residents during their first month on the 23 ACGME 1.0 milestones. MED stations included a history and physical with verbal presentation, patient simulation, vascular access, wound management, and airway management. MED, Clinical Competency Committee-generated (CCC-generated) milestone, and ABEM ITE scores were averaged and compared utilizing Pearson's correlation coefficient.

Results: Of 112 PGY 1 EM residents, 110 (98%) were analyzed over an 8-year period. We observed a moderate positive correlation of MED and CCC-generated milestone scores (r = .34, p < .001). There was a nonstatistically significant weak positive correlation of MED and ABEM ITE scores (r = .13, p = .17).

Discussion: An early assessment of EM milestones in the PGY 1 year can assist in the prediction of CCC-generated milestone scores for PGY 1 residents.

Keywords: Assessment; Clinical Skills Assessment/OSCEs; Clinical/Procedural Skills Training; Competency-Based Medical Education (Competencies, Milestones, EPAs); Emergency Medicine; Milestones; Simulation.