Abstract
Purpose :
To demonstrate methods for Receiver-Operating Characteristic (ROC) analysis of correlated eye data.
Methods :
Using data from the Telemedicine Approaches to Evaluating Acute-Phase Retinopathy of Prematurity Study we previously developed a model for predicting development of treatment-requiring ROP (TR-ROP). The prediction model was based on data from 771 infants with birth weight <1251 grams who completed 1 retinal imaging session by 34 weeks of postmenstrual age and 1 subsequent retinopathy of prematurity (ROP) examination to determine TR-ROP. The factors in the model were: birth weight (BW), gestational age (GA), and findings from the first image session (IM). We calculated the AUC from a prediction model using BW, GA and IM and compared it to the AUC using BW and GA only. We used three methods to estimate the AUC and difference of AUC’s for correlated eye data: the Naïve method (NA) treating eyes as independent, the Obuchowski (OB) and Cluster Bootstrap (CB) methods. The OB method empirically estimates the design effect and effective sample size to derive SE’s and CI’s for AUC estimates and can be used with covariates that are either continuous or ordinal. The CB method selects bootstrap samples (BOOT) by randomly sampling with replacement the same number of subjects as in a given sample and includes all eligible eyes from those subjects. The AUC is computed using BOOT and the process is repeated B times. The 95% CI for AUC is based on the 2.5th and 97.5th percentiles of the ordered distribution of AUC from the B samples.
Results :
A comparison of the AUCs from the models predicting TR-ROP using BW and GA only, and using BW, GA and IM are shown in Figure 1 and Table 1. The point estimates of the AUC from the model with BW and GA were identically 0.802 from the NA and OB approaches, and quite similar to the CB approach, but the 95% CIs differed. The NA approach had a narrower width for the 95% CI (0.066) than the OB (0.093) or CB (0.090) approaches. The inclusion of IM findings significantly improved the AUC by 0.076, with narrower 95% CI of ΔAUC from the naive analysis (0.053) than from the OB (0.070) and CB approaches (0.070).
Conclusions :
In ROC analysis of correlated eye data, ignoring inter-eye correlation leads to an inappropriately narrower 95% CI, while the OB or CB approaches can properly account for inter-eye correlation.
This is a 2021 ARVO Annual Meeting abstract.