Abstract
Purpose :
To develop and compare machine learning models that can predict and identify patients at risk of being non-adherent to annual diabetic eye disease (DED) testing.
Methods :
We conducted a retrospective, cross-sectional study of all patients with diabetes mellitus managed across 30+ primary care sites at Johns Hopkins Medicine in the calendar year 2021. Collected patient data included age, race, ethnicity, language, sex, insurance type, national area deprivation index (ADI), inflation-adjusted median household income, and primary care clinic site type. A higher ADI score indicates more socioeconomic disadvantage. Primary care clinic site type was categorized based on whether autonomous artificial intelligence (AI) testing (IDx-DR, Digital Diagnostics, Coralville, IA) for DED was available or not. The primary outcome measure was adherence to DED testing by the end of the calendar year. Several machine learning models including random forest (RF), support vector machine (SVM), XGBoost (XGB), and logistic regression (LR) were trained using the aforementioned patient variables. The data was portioned at the patient level: 80% training and 20% testing. A p value of <0.05 was considered statistically significant.
Results :
A total of 17,590 patients were included. The four machine learning algorithms had high sensitivities in predicting non-adherent patients: 86% (RF), 99% (SVM), 70% (XGB), and 80% (LR). The top two predictive patient factors of non-adherence in both the RF and LR models were younger age and receiving care at a clinic site without AI implementation. In the XGB model, younger age and higher inflation-adjusted median household income were the two top predictors.
Conclusions :
Using data from a large cohort of patients with diabetes managed at an integrated healthcare system, we were able to train several machine learning models to predict non-adherence to annual DED testing in a given calendar year. Of the 4 models, SVM achieved the highest sensitivity of 99%. Of note, in two of the four models, being managed at a primary care site without autonomous AI was one of the two top predictors of patient non-adherence.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.