Validating a Deep Learning based Diabetic Retinopathy Model among Patients with Young-onset diabetes

Abstract

Purpose : Diabetic retinopathy (DR) is a leading cause of preventable blindness globally, and affects people with both Type 1 and Type 2 disease. Whereas many artificial intelligence (AI) models have been validated for individuals with Type 2 disease, fewer studies have focused on whether AI models work for individuals with Type 1 disease, which most commonly presents in childhood. These patients are a distinct population with unique characteristics, including a prominent retinal sheen that could be mistaken for exudates or cotton wool spots and potentially confound AI models. This study tests the hypothesis that a Deep Learning System (DLS) can perform effectively on a younger population of individuals.

Methods : In this prospective observational study, we recruited 321 participants aged 18-45, of whom 98.7% had Type 1 diabetes, for AI-based DR screening. All participants had their pupils dilated and had color fundus photographs taken, which were adjudicated by experienced graders to obtain reference DR grades. We defined a younger cohort (age 18-25) and an older cohort (age 26-45) and examined differences in AI model performance between the two cohorts.

Results : Eye level sensitivity for moderate-or-worse DR was 97.6% [95%CI 92.5 - 100.0] and 94.1% [89.26 - 98.23], respectively (p=0.369), for the young and old cohort. Specificity for moderate-or-worse DR differed by age cohort: 97.9% [95.4 - 99.0] for the younger cohort and 92.5% [88.2 - 95.4] for the older cohort (p=0.005). Similar trends were observed for Diabetic Macular Edema (DME); sensitivity was 78.6% [57.1 - 95.5] and 77.7% [63.2 - 91.4] (p=0.90), whereas specificity was 97.0% [94.4 - 99.0] and 92.1% [88.3 - 95.5] (p=0.008). Across the combined cohort, retinal sheen was more common in false negative DME cases (12/13, 92%) than in true positives (30/46, 65%). For both cohorts, the gradability rate for both DR and DME was near-perfect (99% for both).

Conclusions : AI-based DR screening performed well in a younger, predominantly type-1 diabetic patient population. Sensitivity was comparable between the older and younger age group, however, specificity was better for the younger age group. Algorithms may benefit from additional training data containing cases with retinal sheen.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Baseline characteristics of study participants

 

Eye-level analysis