Abstract
Purpose :
Competency-based medical education requires a robust and multifaceted evaluation system. We performed a cross-sectional study of ophthalmology residents to determine the validity, reliability, and feasibility of 360-degree assessment tool for assessing competency in professionalism, interpersonal and communication skills, and system-based practice.
Methods :
A 5-point Likert-type online questionnaire was completed by attendings, nurses and allied health staffs, patients/families, and resident peers after each ophthalmology resident rotation from January 2021 to June 2021. The questionnaire was adapted from the International Council of Ophthalmology (ICO) 360-degree assessment tool. The Indonesian version questionnaire was developed using forward and backward translation.
Results :
A total of 534 evaluations were completed for 42 residents. Validity test using Pearson correlation analysis between all combinations of evaluators ranged from 0.51 to 0.94 and all were statistically significant (p<.01). The internal consistency reliability of the instrument was a Cronbach's alpha of 0.92, 0.80, 0.86, and 0.93 for attending doctor, nurse and allied health staff, patient/family, and resident peer, respectively. Sixty percent of study participants felt that this assessment contributed positively to the operation of ophthalmology service. All evaluators rated residents highly (mean Likert score range, 3.96 – 4.60). Patient/families rated the resident highest for interpersonal and communication skills (4.54±0.12) and system-based practice (4.60±0.16) among other evaluator groups. Nurse and allied health staff rated resident highest for professionalism (4.58±0.35). Ratings by year of training varied for each competency. No significant differences were found between rating from patient/families and their educational background (p>.05).
Conclusions :
The 360-degree assessment tool appears to be valid, reliable, and feasible to evaluate ophthalmology residents’ competency in professionalism, interpersonal and communication skills, and system-based practice. Information from the assessment may provide multidimensional formative feedback for residents’ development and improve residents’ performance.
This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.