MSS849: GAPS IN ENTRUSTABLE PROFESSIONAL ACTIVITY (EPA) EVALUATION: LIMITATIONS OF SIMPL USE IN A SINGLE GENERAL SURGERY RESIDENCY PROGRAM
Megan C Breckheimer, MD1; Danielle M Isaacson2; Melissa M Burns, MD1; Robert W Krell, MD1; Christopher E Dyke, MD1; 1Walter Reed National Military Medical Center; 2Uniformed Services University of the Health Sciences
Objectives: The American Board of Surgery (ABS) launched its Entrustable Professional Activities (EPAs) framework for General Surgery residency programs in July 2023 to assess key tasks across all phases of patient care—preoperative, intraoperative, and postoperative. To improve efficiency of EPA assessment, the ABS adopted SIMPL (System for Improving and Measuring Procedural Learning) mobile tool, allowing faculty to document performance, case complexity, feedback, and entrustment levels in real time. Prior studies show variability in EPA assessments, particularly across different phases of care. Here, we review the first complete year of use at a military treatment facility (MTF). We hypothesize that faculty at our institution are primarily using SIMPL to evaluate the intraoperative phase of care which would in turn favor evaluations of senior residents. The purpose of this study was to analyze SIMPL evaluations collected at a single (MTF) over the first academic year the app was available to identify gaps and guide faculty education on EPA-aligned assessment.
Methods: Deidentified SIMPL evaluations from 01July 2024–30June 2025 were reviewed. Evaluations were categorized by phase of care (non-operative/preoperative, intraoperative, postoperative) and by postgraduate year (PGY). Descriptive statistics were used to characterize evaluation patterns.
Results: 293 SIMPL evaluations were reviewed. Of these, 86% of all evaluations were performed by faculty in the home institution, though 42% of rotations were performed outside the home institution. 50/293 (17.1%) were in the non-operative/preoperative phase, 225/293 (76.8%) intraoperative, and 18 /293 (6.1%) postoperative. Evaluations were distributed among PGY in the following way: PGY1 15/293 (5.1%), PGY2 58/293 (19.8%). PGY3 (86/29.4%), PGY4 44/293 (15.0%), PGY5 90/293 (30.7%). Intraoperative evaluations were the most common across all post graduate years making up the following percentages of evaluations for each year: PGY1 66.7%, PGY2 69.0%, PGY3 86.0%, PGY4 84.1%, PGY5 71.1%.
Conclusion: In the first year of its utilization at a military residency program, the SIMPL app was used mostly for intraoperative evaluations and rarely utilized by faculty outside the home institution. This led to missed opportunities for evaluation and feedback in two of the three phases of patient care and outside rotations which continue to increase in number across residency programs in the Military Health System. Both resident and faculty education are needed to encourage full use of the app in all phases of care to ensure comprehensive evaluations within the EPA framework.