Abstract
Purpose :
We previously developed and evaluated a DL model to identify eyes with moderately severe and severe NPDR from 7F-CFP leveraging a large cohort of patients with diabetes from the United States.1 In this study, we evaluate and report performances of the DL model on an independent cohort of patients with DR from Portugal.
Methods :
The DL model presented previously1 was developed using 7F-CFP collected from the eyes of 37,358 patients with diabetes between 1999 and 2016 (Source: Inoveon Corporation, Oklahoma City, OK). DR severity and presence of clinically significant macular edema were assessed from 7F-CFP (Zeiss FF 450+) by expert graders at a centralized reading center using the Early Treatment Diabetic Retinopathy Study Diabetic Retinopathy Severity Scale (DRSS). Prevalence of moderately severe or severe NPDR (DRSS 47–53) in this cohort was 2.2%. The dataset was used to develop a DL Inception v3 model with transfer learning to detect eyes with DRSS 47–53.
The independent evaluation dataset was collected from a natural history study of DR progression (NCT03010397). Enrolled patients had less than mild NPDR at baseline (32% female; mean age at baseline, 63 years) and were followed up annually over 5 years. From 172 patients who completed the study, 635 unique eyes/visits were imaged by 7F-CFP (Topcon TRC-50DX) and had a valid DRSS, of which 14 eyes were identified with DRSS 47–53. Model performance was assessed using the area under the receiver operating characteristic (AUROC) curve, specificity, and sensitivity, using a cutoff computed on the development dataset.
Results :
The model performed on the evaluation dataset with an AUROC of 0.900 (95% CI, 0.864, 0.944), sensitivity of 1.0 (95% CI, 1.0, 1.0), and specificity of 0.699 (95% CI, 0.667, 0.734) using a cutoff computed on the development set that maximizes the Youden index.
Conclusions :
Despite differences between the development and evaluation datasets in terms of geographic location of patients and CFP imaging instruments used, the DL model rendered good performance and generalizability for identification of eyes with DRSS 47–53. Provided additional validation of this model on more diverse datasets, the model could be used to inform screening of patients at high risk of progression to vision-threatening DR.
Reference:
1. Benmansour F et al. Invest Ophthalmol Vis Sci. 2021;62(8):115.
This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.