ENHANCED PHENOTYPE IDENTIFICATION OF COMMON OCULAR DISEASES IN REAL-WORLD DATASETS

Abstract

Purpose : Patients with the ocular diseases of interest in nearly all real-world data research are identified by using ICD billing codes. Yet with sole reliance on billing codes, some patients may get misclassified, possibly affecting study findings. Using machine learning (ML), we developed, trained, and validated a novel approach to identifying patients with common eye diseases by using additional elements in the electronic health record (EHR) beyond billing codes to improve accuracy.

Methods : Using data from 1 site in the SOURCE Ophthalmology Big Data consortium, we trained LASSO regression and other ML models, incorporating variables from structured data fields throughout the EHR to classify patients with glaucoma, macular degeneration (AMD), and diabetic retinopathy (DR). We compared the accuracy, PPV, NPV, AUC, and area under the precision recall curve (AUCPR) of the enhanced phenotype identification (EPI) models to models built using only ICD billing codes. Gold standard assessments for the presence or absence of these conditions were made by ophthalmologists. External validation was done by using data from a 2nd SOURCE site.

Results : Using EHR data from 1800 randomly selected eyes at 1 SOURCE site, we trained EPI and ICD-only models. Our EPI models outperformed models based solely on ICD billing codes for properly identifying patients with glaucoma (AUC 0.97 vs. 0.90), AMD (AUC 0.98 vs 0.95), and DR (AUC 0.997 vs 0.98). The AUPRC was also better for the EPI models compared with those using only the billing codes for glaucoma (0.79 vs 0.32), AMD (0.76 vs. 0.54), and DR (0.96 vs 0.84). In external validation using structured EHR data from a 2nd site, our EPI models worked well for glaucoma (AUC 0.93, AUPRC 0.75), AMD (AUC 0.96, AUPRC 0.68), and DR (AUC 0.98, AUPRC 0.80).

Conclusions : We developed, tested, and externally validated an ML approach that can accurately identify most patients with glaucoma, AMD, and DR, achieving AUCs of ≥ 0.93 for all 3 conditions. For these conditions, our EPI approach outperformed the conventional method involving ICD billing codes alone. The improved performance was most notable in the AUPRC comparisons for glaucoma and AMD. Higher AUPRCs mean the classifier is returning accurate results (high precision) and mostly all positive results (high recall). These models should greatly enhance researcher involving real-world data to identify patient cohorts with these common eye diseases.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.