Performance of Automated Machine Learning for Predicting Diabetic Retinopathy Progression from Ultrawide Field Retinal Images

Abstract

Purpose : To create and validate automated deep learning models (autoML) for diabetic retinopathy (DR) progression from ultrawide field (UWF) retinal images.

Methods : This was a prospective development and validation study of autoML models for predicting DR progression in UWF images. A total of 1,179 unique de-identified UWF images with mild or moderate nonproliferative DR (NPDR) from 703 persons with diabetes and 3 years of longitudinal follow-up retinal imaging had AutoML models generated from baseline on-axis 200-degree UWF retinal images. Baseline retinal images were graded for progression based on centralized reading center evaluation of baseline and follow-up images according to the clinical Early Treatment Diabetic Retinopathy Study severity scale. Images for model development were split 8-1-1 for training, optimization, and testing to detect DR progression of 1 or more steps. Algorithm validation was performed using a 328 image set from the same patient population but not used in model development. The outcomes measured were area under the precision-recall curve (AUPRC), sensitivity, specificity, positive predictive value, negative predictive value, (SN, SP, PPV, NPV, respectively) accuracy, and F1 scores.

Results : DR severity distribution was 32.2% mild and 67.8% moderate NPDR. DR progression was present in 50% of the training set. The model’s AUPRC for baseline mild NPDR was 0.717 and 0.863 for moderate NPDR. Performance on the validation set for mild NPDR was 0.70, 0.72, 0.31, 0.93, and 0.15 (SN, SP, PPV, NPV, and prevalence) and for moderate NPDR was 0.82, 0.74, 0.38, 0.93, and 0.22. In the validation set, 77.5% of mild and 85.4% moderate NPDR eyes that progressed ≥2-steps were identified. All eyes with mild NPDR that progressed ≥1-steps within 6 months and 1 year were identified, while 88.9% and 85.0% of eyes with moderate NPDR that progressed ≥1-steps within 6 months and 1 year were identified.

Conclusions : This study demonstrates the accuracy and feasibility of autoML models for identifying DR progression developed using UWF images. Machine learning algorithms may can potentially refine the risk of disease progression and personalize screening intervals that may reduce costs and improve vision-related outcomes.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.

 

Fig 1: Precision-Recall Curves in mild NPDR
Legend: AUPRC – Area under the Precision-Recall Curve

 

Fig 2: Precision-Recall Curves in moderate NPDR