Accuracy of Integrated Artificial Intelligence (AI) Grading at the Point of Care (POC) Using Handheld Retinal Imaging in a Community-Based Diabetic Retinopathy (DR) Screening Program (DRSP)

Abstract

Purpose : To prospectively evaluate handheld retinal images assessed by AI at the time of imaging as compared to standard retinal image graders at a centralized reading center (RC).

Methods : Prospective comparative study of AI assessment of referable DR [(refDR) moderate nonproliferative DR (NPDR) or worse, or any level of diabetic macular edema (DME)] and vision threatening DR [(vtDR) severe NPDR or worse, or any level of center involving DME (ciDME)]. AI assessment of disc and macular images performed at the time of imaging was compared with RC evaluation of validated 5-field handheld retinal images [(5F) disc, macula, temporal, superior and inferior]. RC evaluation of the 5F images followed the international DR/DME classification. Sensitivity and specificity (SN/SP) for ungradable images, refDR and vtDR were calculated.

Results : 1,733 eyes from 869 diabetic (DM) patients were enrolled in the study. Cohort demographic: age 59.5±10.0, 64.7% female, 98.4% type 2, DM duration 6.6±7.2 years. RC distribution of DR severity: no DR 70.5%, mild NPDR 9.3%, moderate NPDR 7.6%, severe NPDR 3.7%, PDR 2.9%, ungradable 6.0%. DME severity: no DME 82.2%, DME 6.2%, ciDME 4.2%, ungradable 7.3%. RefDR was present in 13.8% and vtDR in 7.8% of eyes. Images were ungradable for DR or DME in 7.7% by RC and 20.7% by AI. Table 1 summarizes the SN/SP and operating characteristics of POC AI grading. SN/SP of AI grading compared to RC evaluation was 0.83/0.95 for refDR and 0.96/0.91 for vtDR. 4 eyes with vtDR (3 severe NPDR and 1 proliferative DR) were missed by AI. Comparisons of performance of the POC AI with existing FDA approved algorithms are presented in table 2.

Conclusions : This study demonstrates that POC AI following a defined retinal imaging protocol at the time of imaging has SN/SP for refDR that meets the current acceptable thresholds of 0.80 and 0.95. Integrating AI at the POC could substantially reduce centralized reading center burden and speeds information delivery to the patient, allowing more prompt eye care referral.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Table 1. Agreement rates and measures of performance of point of care artificial intelligence evaluation against reading center evaluation

 

Table 2. Comparison with FDA approved artificial intelligence systems