Abstract
purpose. The purpose of this study was to investigate the effect of the presence of atypical birefringence patterns, as measured by the typical scan score (TSS), on the diagnostic accuracy of a scanning laser polarimeter (the GDx VCC; Carl Zeiss Meditec, Inc., Dublin, CA) assessed by receiver operating characteristic (ROC) curves for discriminating between glaucoma and healthy eyes.
methods. Two hundred thirty-three glaucomatous eyes (repeatable abnormal visual fields by pattern standard deviation [PSD] and/or glaucoma hemifield test [GHT]) from 153 patients with glaucoma and 104 eyes from 71 healthy participants enrolled in the UCSD Diagnostic Innovations in Glaucoma Study (DIGS) were imaged using the GDx VCC. An ROC regression model was used to evaluate the influence of the covariates TSS; disease severity, defined as standard automated perimetry (SAP) mean deviation [MD]; and age in years on the diagnostic accuracy of the GDx parameters nerve fiber indicator [NFI], TSNIT (temporal, superior, nasal, inferior, temporal) average thickness, superior average thickness, inferior average thickness, and TSNIT standard deviation. Areas under the ROC curve were calculated for specific levels of the covariates according to the results provided by the model.
results. TSS and SAP MD significantly affected the diagnostic accuracy of each investigated GDx VCC parameter. Low TSSs, indicating the presence of atypical scans, were associated with decreased accuracy. For NFI, ROC curve areas ranged from 0.749 (when TSS = 20) to 0.904 (when TSS = 100). A similar influence of TSS was found for other parameters. In addition, diagnostic accuracy increased with increasing disease severity. For instance, for NFI, ROC curve areas ranged from 0.853 (when SAP MD = −3) to 0.954 (when SAP MD = −15).
conclusions. The diagnostic accuracy of GDx VCC parameters is affected by disease severity and is adversely affected by the presence of atypical retardation patterns (i.e., decreasing TSS). GDx VCC scans with atypical scan patterns should be interpreted with caution when used in clinical practice.
Scanning laser polarimetry (SLP) provides real-time, objective measurements for assessing retinal nerve fiber layer (RNFL) thickness in glaucoma. Recent studies have shown that using SLP with variable corneal compensation (GDx VCC; Carl Zeiss Meditec, Inc., Dublin, CA) improves diagnostic precision and strengthens cross-sectional structure-to-function associations, compared with using SLP with fixed corneal compensation (FCC).
1 2 3 4 5
However, GDx with VCC has been criticized for providing a relatively large number of artifact-laden images, called atypical scans. Atypical scans are scans with an atypical birefringence (i.e., retardance) pattern (ABP) that is not representative of RNFL thickness patterns found histologically (i.e., increased birefringence superiorly and inferiorly, indicating thicker RNFL compared with decreased birefringence temporally and nasally, indicating thinner RNFL). Rather, in addition to high birefringence superiorly and inferiorly, scans with ABP display increased birefringence in the temporal and nasal quadrants in radial patterns centered on and surrounding the entire optic disc. Current GDx VCC software includes an exportable parameter called typical scan score (TSS) that provides a numerical representation of the degree of “typicalness” in each scan, ranging from 1 (extremely atypical) to 100 (typical; see
Fig. 1 ). This parameter is the output of a support vector machine machine-learning classifier trained to identify scans that were subjectively assessed as atypical by instrument developers. Previous studies have suggested cutoff values for this parameter that result in the exclusion of subjectively classified atypical scans.
6 7 However, although these cutoffs provide a way to exclude atypical scans, they do not directly address the effect of atypical scans on the diagnostic ability of GDx VCC.
The purpose of the present study was to investigate the effect of TSS on the accuracy of the GDx VCC for classifying eyes as healthy or glaucomatous. We used a receiver operating characteristic (ROC) regression technique to analyze the covarying effects of TSS, glaucoma severity, and age on ROC curves describing the diagnostic performance of several GDx VCC parameters.
One or two eyes were selected from each of 242 participants enrolled in the University of California, San Diego-based longitudinal Diagnostic Innovations in Glaucoma Study (DIGS) for study. All participant eyes had had GDx VCC imaging and a reliable visual field test within 6 months. A total of 337 eyes were studied: 104 were classified as healthy and 233 were classified as glaucomatous.
Each study participant underwent a comprehensive ophthalmic evaluation including review of medical history, best corrected visual acuity testing, slit lamp biomicroscopy, IOP measurement with Goldmann applanation tonometry, gonioscopy, dilated fundus examination with a 78-D lens, simultaneous stereoscopic optic disc photography (TRC-SS; Topcon Instruments Corp. of America, Paramus, NJ), and standard automated perimetry (SAP) using the 24-2 Swedish Interactive Threshold Algorithm (SITA; Humphrey Field Analyzer II; Carl Zeiss Meditec, Inc.). To be included in the study, participants had to have best corrected acuity better than or equal to 20/40, spherical refraction within ±5.0 D and cylinder correction within ±3.0 D, and open angles on gonioscopy.
Participants were excluded if they had a history of intraocular surgery except for uncomplicated cataract or glaucoma surgery. We also excluded all participants with nonglaucomatous secondary causes of elevated IOP (e.g., iridocyclitis, trauma), other intraocular eye disease, other diseases affecting the visual field (e.g., pituitary lesions, demyelinating diseases, HIV or AIDS, or diabetic retinopathy), medications known to affect visual field sensitivity, or problems other than glaucoma that affect color vision.
Glaucomatous eyes were defined as those with repeatable (two consecutive) SAP results outside normal limits by pattern standard deviation (PSD; P < 5%) or glaucoma hemifield test (GHT). The first abnormal SAP was on or before the imaging date. If only one eye of a study participant had repeatable abnormal visual fields, the nonqualifying eye was excluded from study. Average SAP mean deviation (MD) of the glaucomatous eyes within 6 months of GDx VCC imaging was −5.67 dB (median = −3.51, SD = 5.71; range = −31.46 to +1.00) and average PSD was 5.35 dB (median = 3.58, SD = 3.73; range = 1.38 to 15.58). The mean age of the patients with glaucoma (i.e., those with glaucoma in at least one eye, n = 173) was 68.4 years (median = 69.8, SD = 11.6; range = 33.2 to 91.8), 91 (53%) were women, and 146 (84%) were self-reported white. Eighty-two (35%) of the glaucomatous eyes were pseudophakic.
Healthy eyes were defined as those with healthy-appearing optic discs on clinical examination, SAP results (MD, PSD, and GHT) within normal limits, and no history of IOP > 22 mm Hg. Average SAP MD of the healthy eyes was −0.86 dB (median = −0.75, SD = 1.38; range = −4.78 to +1.73) and average PSD was 1.62 dB (median = 1.54, SD = 0.45; range = 0.99 to 3.61). Both global indices differed significantly and were better in healthy than in glaucomatous eyes (t-tests, P < 0.001). The mean age of the healthy participants (n = 69) was 51.0 years (median = 55.3, SD = 16.4; range = 18.3 to 83.6) and was significantly lower than that of the patients with glaucoma (t-test, P < 0.001). Fifty (72%) healthy participants were women and 56 (81%) were self-reported white. Two of the healthy eyes were pseudophakic.
This research adhered to the tenets of the Declaration of Helsinki and the Health Insurance Portability and Accountability Act (HIPAA).
All participants’ eyes were imaged with a commercially available scanning laser polarimeter (GDx VCC, software version 5.0.1; Carl Zeiss Meditec, Inc.). Principles of this technology have been provided in detail elsewhere. In general, scanning laser polarimetry measures the retardation of light reflected from the birefringent RNFL fibers and provides an estimated RNFL thickness based on the linear relationship between observed retardation, measured using a prototype instrument, and RNFL thickness determined histologically. The GDx VCC employs a variable corneal polarization compensator that allows eye-specific compensation of anterior chamber birefringence. After determining the axis and magnitude of corneal polarization in each eye by macular scanning,
8 three appropriately compensated retinal polarization images per eye were automatically obtained and combined to form each mean image used for analysis. Only well focused, evenly illuminated and centered scans with residual anterior segment retardation ≤ 12 nm and SD ≤ 7 μm, determined by the perimeter software, were included (cutoffs suggested by Qienyuan Zhou PhD, Carl Zeiss Meditec, Inc., verbal communication, June 2005).
In this study we examined the effect of the software-determined TSS on the ability of several parameters to discriminate between healthy and glaucomatous eyes. The TSS is a continuous variable ranging from 0 to 100 and is the result of a support vector machine analysis of VCC data labeled for training based on the subjective appearance of each scan (typical versus atypical). TSS is based on the slope, standard deviation, and average magnitude of RNFL thickness measurements from the edge of the optic disc extending outward to 20°. Low TSSs indicate very atypical scans (
Fig. 1 , leftmost image, TSS = 20) and high typical scan scores indicate very typical scans (
Fig. 1 , rightmost image, TSS = 100).
RNFL measurement parameters investigated in this study were the nerve fiber indicator (NFI), TSNIT (circumpapillary RNFL thickness measured under the automatically defined a 3.2-mm diameter calculation circle: T, temporal sector; S, superior sector; N, nasal sector; I, inferior sector) average, superior average, inferior average, and TSNIT SD. These parameters were selected because they are those provided on the instrument print-out designed for clinical use. We also examined the effects of the covariates disease severity (defined as SAP MD) and age in an ROC regression model with TSS.
In the present study, we used an ROC regression modeling technique to evaluate the influence of atypical scan patterns on the diagnostic accuracy of the GDx VCC in glaucoma. This modeling approach has been recently applied by Medeiros et al.
9 to evaluate the influence of covariates on the performance of diagnostic tests in glaucoma. This methodology allows the evaluation of the influence of covariates on the diagnostic performance of the test, so that ROC curves for specific values of the covariate of interest can be obtained. Also, it allows adjustment for the possible confounding effects of other covariates. Details of the modeling procedure have been described previously.
9 10 In brief, the ROC
x,x D (q) is the probability that a subject with diseased eyes with disease-specific covariates
X D (that is, covariates specific to subjects with diseased eyes such as disease severity, for example) and common covariates
X (covariates common to both subjects with diseased eyes and healthy subjects) has test results
Y D that are greater than or equal to the
qth quantile of the distribution of tests results from subjects with nondiseased eyes. That is, when the specificity of the test is 1 −
q, the sensitivity is ROC
x,x D (q).
The general ROC regression model can be written as
\[\mathrm{ROC}_{X,X_{\mathrm{D}}}(q){=}{\Phi}({\alpha}_{1}{+}{\alpha}_{2}{\Phi}^{{-}1}(q){+}{\beta}X{+}{\beta}_{\mathrm{D}}X_{\mathrm{D}})\]
where the coefficients α
1 and α
2 are the intercept and slope of the ROC curve, respectively, and Φ is the normal cumulative distribution function. If the coefficient for a specific variable
X(β) is greater than 0, then the discrimination between subjects with and without diseased eyes increases with increasing values of this covariate. Similarly, if the coefficient for the disease-specific covariate
X D(β
D) is greater than 0, then subjects with diseased eyes who have larger values of this covariate are more distinct from those with nondiseased eyes than are those with diseased eyes who have smaller values of
X D.
In the present study, an ROC model was fitted to assess the influence of the disease-specific covariate severity and the common covariates age and TSS on the diagnostic performance of the GDx VCC parameters. The following ROC regression model was fitted for each GDx VCC parameter evaluated:
\[\mathrm{ROC}_{X,X_{\mathrm{D}}}(q){=}{\Phi}({\alpha}_{1}{+}{\alpha}_{2}{\Phi}^{{-}1}(q){+}{\beta}_{1}\mathrm{TSS}{+}{\beta}_{2}\mathrm{TSS}{\times}{\Phi}^{{-}1}(q){+}{\beta}_{3}\ \mathrm{severity}{+}{\beta}_{4}\ \mathrm{age}\]
where TSS is a continuous variable quantifying the presence of atypical patterns of retardation, severity is the variable indicating severity of glaucomatous damage as measured by the MD, and age is a variable indicating the patient’s age. An interaction term between the variable TSS and Φ
−1(
q) was included to allow the effects of this covariate to differ by various amounts depending on the false-positive rate
q (or specificity 1 −
q)—that is, to influence the shape of the ROC curve.
Parameters were estimated using probit regression. To obtain confidence intervals for regression parameters, a bootstrap resampling procedure was used (n = 500 resamples). Statistical analyses were performed with commercially available software (Stata ver. 9.0; StataCorp, College Station, TX, and SPSS ver. 13.0; SPSS Inc., Chicago, IL). The α level (type I error) was set at 0.05.