Validation of a Simple Diabetic Retinopathy Risk Score to Identify Patients at Low Risk of Diabetic Retinopathy

Abstract

Purpose : To identify patients at low risk of retinopathy who might need less frequent screening, we developed and validated a Retinal Risk Score (RSS) which performed equally as well as 5 machine learning models. The RSS had an AUC of 0.733. However, the prior validation dataset was relatively small at 540 patients. This study aims to validate our predictive model on a second patient dataset.

Methods : A retrospective chart review of patients who were screened for DR via fundus photography using Eyenuk’s (Los Angeles, CA) EyeArt at Temple primary care clinics between 07/2022 and 12/2022 was performed. Patients with more than mild (MTM) DR (International Clinical Diabetic Retinopathy (ICDR) 2-4+/- evidence of macular edema) were considered as positive. Using our defined RSS (HgbA1c * Years with diabetes), we generated prediction probabilities and a receiver operator characteristics (ROC) curve. We calculated the area under the ROC curve for our predictor with this patient dataset and compared it to our index study dataset. We identified a dichotomous cutoff point by maximizing Cohen’s kappa statistic. Lift curves were generated.

Results : Figure 1 shows the ROC curves and the AUC for the product predictor model used for the previous and new dataset. In our previous validation study of 540 patients, we found the product predictive model AUC to be 0.733. Using 653 new patients, we determined the predictive model AUC to be identical to the previous study (0.737). Using the cutoff RSS of 53.5, we identified 316 (48.39%) patients at low risk for developing diabetic retinopathy. The error rate at this cutoff value was approximately 12%. This value is higher than the previous study (9.2%), consistent with the higher rate of retinopathy in this new patient population.

Conclusions : This study confirms that the RSS is a useful tool in identifying patients at a low risk for retinopathy and may prove useful as a means to determine which patients could be screened less frequently for diabetic retinopathy in low prevalence populations.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Figure 1: ROC curves for the index (RSS Study 1) and new (RSS Study 2) studies.

 

Figure 2: RSS Lift Curves generated for RSS study 1 (n=540) and RSS study 2 (n=653) with cutoff RSS at 53.5.