Abstract
Purpose :
The electronic health record (EHR) contains a wealth of clinical data, but improved and automated classification approaches are needed to accurately identify patient cohorts in order to use this data for research. We aim to evaluate if clinical notes increase the positive predictive value (PPV) and sensitivity of a classification algorithm for proliferative diabetic retinopathy (PDR) and diabetic macular edema (DME) compared to diagnosis codes.
Methods :
De-identified EHR data from an academic medical center identified patients aged ≥ 18 years, with diabetes mellitus, with any history of diabetic retinopathy, and with available clinical notes. From these 2366 patients, 100 random patients underwent chart review by 3 ophthalmologists to establish the gold standard. International Classification of Diseases (ICD-9/10) codes were extracted for diagnosis codes. Natural language processing and text mining methods identified positive mention of PDR or DME from in-person office and procedure notes (notes method). Classification by diagnosis codes and clinical notes were then compared with the gold standard to determine sensitivity, specificity, PPV, and negative predictive value (NPV) of PDR and DME classification. McNemar’s test was used to compare the sensitivity and specificity of classification methods. To compare the PPV and NPV, we used the method proposed by Moskowitz and Pepe.The entire diabetic retinopathy cohort (N=2366) was then classified into PDR and/or DME using both the diagnosis codes and the notes method, and compared.
Results :
For PDR classification of the 100 random patients, the notes method had significantly higher sensitivity (94.3%) compared to diagnosis codes (82.9%, P=0.046), with similar PPV (97.1% vs 93.0%, P=0.27) respectively (Table 1). For DME classification on the 100 random patients, the notes method and the diagnosis codes had similar sensitivity (86.1% vs 80.6%, P=0.32) and PPV (70.5% vs 72.5%, P=0.74), respectively. For the entire diabetic retinopathy cohort, PDR classification by the notes method found 768 patients compared to 706 by diagnosis codes (Table 2). DME classification by the notes method also found more patients (N=880) compared to diagnosis codes (N=745).
Conclusions :
Clinical notes increased the sensitivity of PDR classification compared to diagnosis codes. Clinical notes and diagnosis codes had similar accuracy for classifying patients with DME.
This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.