Assessing the Impact of Photographer Experience on Gradability of Fundus Photography for Artificial Intelligence-based Diabetic Retinopathy Screening

Abstract

Purpose :
The ability of Artificial Intelligence (AI)-based screening to identify diabetic retinopathy (DR) in the primary care setting is affected by the gradability of the fundus photos taken. We performed a retrospective observational study of Temple Primary Care Clinic’s (TPCC) Screening Program to test the hypothesis that a photographer’s gradability rate improves with the number of photographs taken.

Methods : A retrospective chart review was conducted on 1354 DR screenings of patients 18 years or older with Eyenuk’s (Los Angeles, CA) EyeArt at 10 TPCC sites by 30 different medical assistant photographers (MAP) and the ophthalmic photographer (OP) between 5/12/2021-12/28/2022. Screening results were reported as “ungradable” by EyeArt if at least one eye photograph had uninterpretable images. MAPs who conducted at least 10 DR screenings were identified. Their screening gradability rates were compared to the gold standard screening rate of the OP. MAPs who conducted at least 100 DR screenings had their gradability rate assessed over time in intervals of 10 consecutive screenings. Linear correlation was determined if Pearson’s product-moment correlation coefficient r = ± 0.50 to ±1.

Results : 11 MAPs conducted at least 10 DR screenings and had an overall gradability rate of 56.9% (SD = 21%) with no linear relationship between the number of screenings and the gradability rate (r = +0.14). OP had a gradability rate of 83.4%. 3 MAPs conducted at least 100 screenings and 2 did not demonstrate any linear relationship of gradability rate over time (r = +0.13, r = +0.09), similar to the OP (r = +0.17). 1 MAP had a negative linear relationship (r = -0.68).

Conclusions : Surprisingly, screening gradability did not improve with increased photographer experience. MAP gradability rates were significantly worse than the OP rate of 83.4%, suggesting that either further training or role reassignment is needed. MAPs should have their gradability rate assessed promptly after starting to take fundus photos and also receive regular periodic feedback from the instructor, as there is no demonstrated improvement in gradability rate over time. Further research may be conducted to identify the reasons why MAP performance does not improve with experience.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.