Abstract
Purpose :
Resident surgical skill assessment is important to trainees, faculty, patients, and accreditors; however, conventional assessment forms suffer from lack of standardization and grade inflation. In this prospective observational study, we adapted a validated questionnaire (Ottawa Surgical Competency Operating Room Evaluation (O-SCORE)) in an online survey platform to collect surgical evaluations at multiple ophthalmology residency programs. The goal was to identify whether this tool can track the longitudinal progression of resident surgical skill acquisition and foster earlier detection of struggling residents, thus allowing for earlier intervention.
Methods :
The modified O-SCORE form evaluates a trainee’s ability to perform a procedure safely and independently, rather than evaluating a specific technical skill. Ophthalmology residency programs were recruited to participate via a program director network. Completed evaluations were stored in a REDCap database (Research Electronic Data Capture) and analyzed for score trends with time and across levels of training. A non-parametric one-way ANOVA was used for cross-sectional analysis and linear regression was used for longitudinal analysis.
Results :
Twelve programs participated including 94 residents and 64 attending evaluators, who completed evaluations via institution-specific weblinks. For the academic year 2017-2018, 902 cataract surgery evaluations were collected, including 695 routine cases and 207 complex cases. A cross-sectional analysis of cataract surgery evaluations demonstrated improving scores for total, routine and complex cases with increasing post graduate year (PGY) (p<0.05) except between PGY2 and PGY3 due to limited case numbers of PGY2 participants. Mean assessment scores showed upward progression over the 12 months for PGY4 residents (p<0.05). Assessment scores also trended upward significantly for 4 of 5 PGY4 residents who had most cataract cases throughout the study (p<0.05).
Conclusions :
This web-based surgical assessment tool demonstrates cross-sectional discriminative validity for cataract surgery and reflects the longitudinal progression over the time of training. It provides useful data for residency programs to monitor the quality of surgical training and early detection of non-progressing residents.
This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.