Comparability of Retinal Nerve Fiber Layer Thickness Measurements of Optical Coherence Tomography Instruments

Abstract

purpose. To compare retinal nerve fiber layer (RNFL) measurements between two ocular coherence tomography (OCT) instruments (OCT 2000 and Stratus OCT; Carl Zeiss Meditec, Dublin, CA) and compare their diagnostic precision.

methods. One hundred thirty-nine consecutive subjects were imaged (3 × 3.4-mm diameter circular scans) on the same day with each instrument. Thirty-five patients were excluded due to poor-quality images. RNFL thicknesses measured by the two instruments were compared, and receiver operating characteristic (ROC) curves were used to determine diagnostic precision.

results. A randomly selected eye of each of 104 participants (28 with open-angle glaucoma, 40 with suspected glaucoma, and 36 healthy subjects) was analyzed. RNFL thickness measurements generally were thicker with OCT 2000 than with Stratus OCT. The difference in global RNFL thickness between instruments was within 20 μm in 66 (65%) of subjects and within 10 μm (the instrument’s limit of resolution) in 25 (25%) subjects. Application of a correction factor to OCT 2000 measurements predicted Stratus OCT RNFL thickness within 10 μm of the observed measurement in 75% of the eyes. For both instruments, highest ROC curve areas (better discrimination between glaucomatous and normal eyes) were found in the inferior sector. Discrimination using global RNFL thickness was better with Stratus OCT than OCT 2000 (P = 0.043).

conclusions. RNFL thickness measurements measured by OCT 2000 can be approximated to measurements made by Stratus OCT using correction factors calculated by this study. However, there remains considerable variability that exceeds the limits of resolution afforded by the instruments themselves. Therefore comparisons between instruments using these approximations should be interpreted with caution.

The Optical Coherence Tomograph (OCT; Carl Zeiss Meditec, Dublin, CA) is designed to provide real-time, objective, cross-sectional measurements of various layers of the retina, including the RNFL. The measurements have been shown to have good reproducibility ^{1 2 3 4} ; however, there is little published evidence ^{5 6} to show that the OCT can be used to detect changes in RNFL thickness over time. There have been several versions of software and hardware since its introduction. To ascertain change, a given feature measured by a diagnostic instrument can be compared with previous measurements, perhaps made with an earlier version of the respective technology (backward compatibility). Whether features measured with one version are comparable to those of another version remains unknown. The original instrument (OCT 1) measures 100 A-scans and has an axial resolution of 10 to 20 μm. The OCT 2000 is an upgrade to enhance the user interface and uses the same imaging technical specifications as the OCT 1. The OCT 3000 (Stratus OCT), the most recent model, differs from the two previous instruments, in that it measures between 128 and 512 A-scans at increased sampling density (1024 samples per A-scan compared with 500 samples per A-scan with the previous instruments). ⁷ In addition, the Stratus OCT has a reported axial resolution of 10 μm or less. Both the original and Stratus OCT instruments have the same transverse resolution (20 μm) and scan range in tissue (2 mm). To detect the progression of RNFL damage in patients who have been serially examined with these various OCT instruments as they become available, it is important to know whether the measurements are comparable. 

This study was designed to compare the RNFL measurements of the same patients directly by using two instruments (OCT 2000 and Stratus OCT). We also compared the diagnostic precision of the two instruments in the same group of subjects. To our knowledge, this is the first study to compare these instruments in this way. 

Methods

This observational cross-sectional study included 139 consecutive subjects who were imaged with both instruments on the same day between February and October 2002. After exclusion of 35 patients due to poor image quality (see the Results section for details), one randomly selected eye of each of the remaining 104 participants (64 women and 40 men) was analyzed. Twenty-eight patients had open-angle glaucoma, 40 patients had suspected glaucoma, and 36 were healthy. Mean ± SD age of patients with glaucoma, patients with suspected glaucoma, and healthy individuals was 68.5 ± 9.1, 60.7 ± 14.5, and 61.6 ± 12.4 years, respectively (patients with glaucoma were significantly older than those with suspected glaucoma and healthy individuals, P = 0.03, analysis of variance). 

All patients were evaluated at the Hamilton Glaucoma Center, University of California, San Diego, and retrospectively selected from our research database. These patients were part of the Diagnostic Innovation in Glaucoma Study (DIGS), a prospective longitudinal study designed to evaluate optic nerve structure and visual function in glaucoma. All patients who met the inclusion criteria described were enrolled in the present study, after informed consent was obtained. The University of California, San Diego, Human Subjects Committee approved all protocols, and the methods described adhered to the tenets of the Declaration of Helsinki. 

Each subject underwent a comprehensive ophthalmic examination, including review of the medical history, best corrected visual acuity, slit lamp biomicroscopy, intraocular pressure (IOP) measurement with Goldmann applanation tonometry, gonioscopy, dilated fundoscopic examination using a 78-D lens, stereoscopic optic disc photography, and automated perimetry using the 24-2 Swedish Interactive Threshold Algorithm (Humphrey visual field analyzer; Carl Zeiss Meditec). To be included, subjects had to have best corrected visual acuity of 20/40 or better, spherical refraction within ±5.0 D, cylinder correction within ±3.0 D, and open angles on gonioscopy. Eyes with coexisting retinal disease, uveitis, or nonglaucomatous optic neuropathy were excluded from the investigation. 

The eyes of the healthy control subjects had IOPs of 22 mmHg or less, with no history of increased IOP and a normal visual field result. A normal visual field was defined as a pattern SD (PSD) within the 95% confidence limits and Glaucoma Hemifield Test results within normal limits. Healthy control eyes also had a healthy appearance of the optic disc and RNFL (no diffuse or focal rim thinning cupping, optic disc hemorrhage, or RNFL defects), as evaluated by optic disc photographs. 

Eyes were classified as glaucomatous if they had repeatable (two consecutive) abnormal visual field test results, defined as a PSD outside the 95% normal confidence limits or Glaucoma Hemifield Test results outside 99% normal confidence limits, regardless of the appearance of the optic disc. Average mean deviation (MD) of the glaucomatous eyes from the visual field test nearest the imaging date was −4.19 ± 2.71 dB (SD; range, −10.25 to −0.69 dB). 

Patients with suspected glaucoma had ocular hypertension (IOP >22 mm Hg on more than two separate visits) and/or glaucomatous appearance of the optic disc but normal results on visual field tests. Glaucomatous damage to the optic disc was defined as the presence of neuroretinal rim thinning, excavation, notching, or characteristic RNFL defects. Of the 38 patients with suspected glaucoma with normal visual fields, 12 (31.6%) had ocular hypertension and optic nerves with a normal appearance, whereas 26 (68.4%) had a glaucomatous appearance of the optic disc. 

The OCT is a noninvasive diagnostic imaging device that obtains cross-sectional images of ocular microstructures. ^{3 8 9} In brief, low-coherence interferometry is used to measure the time delay of backscattered light from different layers of the retina. This involves the analysis of two light beams created when the incident wave is directed onto a partially reflecting mirror. One beam is used for reference, the other for measurement. The RNFL is differentiated from other retinal layers by using a thresholding algorithm. Nerve fiber layer thickness is defined as the number of pixels between the anterior and posterior edges of the RNFL. 

For the OCT 2000 (software ver. A4 × 1; Carl Zeiss Meditec) the RNFL thickness was measured at 100 points along a 360° circular path of 3.4 mm diameter. Three circular scans (of the same diameter) were acquired by standard fast-scanning acquisition (acquisition in a rapid automatic sequence) using the Stratus OCT (software ver. 3.1; Carl Zeiss Meditec), which measures RNFL thickness at 256 points. For both instruments, the RNFL thickness is presented on two circular charts, one with 12 equal sectors each representing 1 hour around the clock face and the other with four equal 90° hour-glass sectors, each representing one quadrant. These charts display RNFL thickness (in micrometers) within each sector. A single mean RNFL thickness for the full 360° scan is also displayed. 

After undergoing mydriasis, each subject was imaged with the two instruments on the same day. Three 3.4-mm diameter circular scans centered on the optic disc judged to be of acceptable quality were obtained for each test eye, the mean of which was used for the analysis. In each subject, RNFL thickness was assessed in four quadrants (superior, inferior, temporal, and nasal). Average RNFL thickness (global thickness) was also assessed (0°–360° on a unit circle). Two investigators (RB, FM) assessed each scan for quality. Scans were not incorporated in the mean if the optic disc was not centered or focused within the circular scan and/or if the computer-generated lines demarcating the borders of the RNFL (generated by the thresholding algorithm) were discontinuous. 

Statistical Analysis

Statistical software (SPSS, ver. 11.0; SPSS Inc., Chicago, IL) was used for statistical analyses. Analysis of variance was used to assess differences in RNFL thickness among patients with glaucoma, patients with suspected glaucoma, and healthy individuals. The Fisher least-significant difference test was used to perform post hoc multiple comparisons. The correlation between the RNFL thickness measurements made by the two instruments for each parapapillary sector was evaluated by Pearson product moment correlation coefficients. Comparisons of RNFL thickness between the two instruments was also graphically represented by displaying the differences between measurements by the two instruments against the mean of the two measurements. ¹⁰ Receiver operating characteristic (ROC) curves were calculated to describe the ability of each instrument to differentiate glaucomatous from healthy eyes. The ROC curve shows the tradeoff between sensitivity and 1 − specificity. An area under the ROC curve of 1.0 represents perfect discrimination, whereas an area of 0.5 represents chance discrimination. The method of DeLong et al. ¹¹ was used to compare areas under the ROC curve. P < 0.05 was considered statistically significant. Sensitivities at fixed specificities for the detection of glaucoma were compared between the two instruments by using the McNemar test cross-tabulation. 

Results

Image Quality with the Two OCT Instruments

Good-quality images were obtained in 104 patients with both OCT instruments. Poor-quality images were obtained in 35 (25.2%) of the total of 139 patients. Of the 35 excluded patients, 24 (68.5%) were excluded on account of a poor-quality OCT 2000 image only, and 3 (8.6%) were excluded on account of a poor-quality Stratus OCT image only. Eight (22.8%) patients were excluded on account of poor-quality images from both instruments. Poor-quality images were significantly more likely to be obtained with the OCT 2000 than with the Stratus OCT (χ² test, P < 0.001). 

RNFL Measurements by Diagnostic Group

Table 1gives the mean values of RNFL thickness in each of four parapapillary sectors in patients with glaucoma, patients with suspected glaucoma, and healthy control subjects. Significant differences between patients with glaucoma and healthy subjects and between patients with glaucoma and patients with suspected glaucoma were found in the global thickness and in the superior and inferior sector thicknesses, with both instruments. Significant differences between patient groups were found by both instruments in the nasal sector, yet when pair-wise comparisons were made between patient groups (Fisher least-significant difference test), there were no significant differences between patients with glaucoma and those with suspected glaucoma, with both instruments. No significant differences were found between the patient groups for the temporal sector with the OCT 2000. With the Stratus OCT, significant differences were found between the patient groups and for the pair-wise comparisons with the exception of the pairing of patients with suspected glaucoma and healthy eyes when the difference was not significant. 

Comparison of RNFL Thickness Measurements between the OCT 2000 and OCT Stratus

Correlation coefficients for the associations of average RNFL thickness between the two instruments for global thickness and for superior, temporal, inferior, and nasal sector thicknesses were significant at P < 0.001. 

Figure 1shows Bland-Altman ¹⁰ plots of the agreement in RNFL thickness measurements between OCT 2000 and Stratus OCT. The difference (OCT 2000 RNFL thickness − Stratus OCT RNFL thickness) was plotted against the average of the two measurements for each sector around the optic disc. These scatterplots demonstrate that in most of the eyes, measurements of RNFL thickness were thicker with the OCT 2000 than with the Stratus OCT. Also, the inspection of the plots reveals a considerable discrepancy between RNFL thickness measurements obtained by the two instruments. For the global RNFL thickness, the difference between the OCT 2000 and Stratus OCT measurements was within 20 μm in 66 (65.3%) subjects, and within 10 μm in 25 (24.7%). 

We also evaluated whether the differences between the measurements obtained by the two OCT instruments were related to the magnitude of RNFL thickness. The slope of the regression line of the difference between the two measurements on the average of the two measurements in the Bland-Altman plots was statistically significant for the nasal and temporal sectors (P = 0.002 and P = 0.032, respectively), indicating the existence of proportional bias. In these sectors, a greater difference in RNFL thickness measurements (OCT 2000 − Stratus OCT) was observed with increasing RNFL thicknesses. This was not the case for global thickness (P = 0.09) or for thicknesses in the superior (P = 0.58), or inferior (P = 0.954) parapapillary sectors. 

An attempt was made to apply a correction factor to the OCT 2000 measurements to predict the Stratus OCT measurements. Table 2gives the equation for the regression lines for the global and the four RNFL sectors, comparing measurements made with the OCT 2000 and the Stratus OCT. These equations were used to predict the Stratus OCT measurement with an observed OCT 2000 measurement. This principle is illustrated in Figure 2for global RNFL thickness by showing the relationship between the measurements from the two OCT instruments in the first graph, and, in the second graph, a Bland-Altman plot shows the agreement between the observed and the predicted Stratus OCT measurements. 

Table 2shows that after application of a correction formula to an OCT 2000 measurement, the resultant predicted Stratus OCT RNFL thickness was within approximately 27 μm of the observed Stratus OCT measurement in 95% of eyes examined, when using these sector measurements of RNFL thickness. If 10 μm is taken to be the limit of axial resolution for the Stratus OCT instrument, ¹² 75% of eyes had a predicted global RNFL thickness within 10 μm of the observed measurement. For the four sectoral measurements, the percentage of eyes ranged from 49% (inferior sector) to 66% (temporal sector). 

Ability of Each Instrument to Differentiate Glaucomatous from Healthy Eyes

Table 3gives the areas under the ROC curves for the global measure of RNFL thickness and the four quadrants for the OCT 2000 and the Stratus OCT. 

The variable with the largest ROC curve area for the OCT 2000 was the inferior sector, whereas for the Stratus OCT, the inferior sector and global RNFL thickness variables had the largest areas. With both instruments, the highest ROC curve areas were found in the inferior sector, followed by the superior, the nasal, and the temporal sectors. With the OCT 2000, these areas were 0.89, 0.79, 0.69, and 0.56, respectively, whereas with the Stratus OCT, they were 0.93, 0.81, 0.75, and 0.65, respectively. The ROC curves for the global and inferior RNFL sectors are shown in Figure 3for both OCT instruments. The ROC curve area of the global measure of RNFL thickness using the Stratus OCT was significantly better than that of the OCT 2000 at discriminating glaucomatous eyes from healthy eyes (Table 3 ; P = 0.043). No significant differences were found between the ROC curve areas of the OCT instruments in the inferior, superior, temporal, and nasal RNFL sectors (P = 0.069, 0.582, 0.167, and 0.366, respectively). 

Table 3also shows the sensitivities at fixed specificities (≥80% and ≥95%) for the sectoral and global measurements of RNFL thickness. At 95% specificity, the global RNFL thickness had a sensitivity of 71% for the detection of glaucoma with the Stratus OCT, compared with a sensitivity of 54% with the OCT 2000. With 80% specificity, the global RNFL thickness had a sensitivity of 93% for the detection of glaucoma with the Stratus OCT, compared to a sensitivity of 68% with the OCT 2000. At 95% specificity, there were no statistically significant differences (P > 0.05) in sensitivities between the two instruments for the global or sector measures. At 80% specificity, only the sensitivity of the global RNFL thickness was statistically significant different between the two instruments (P = 0.039). 

Discussion

In most of the cases, direct comparison of RNFL measurements on the same eyes using the two OCT instruments showed that the Stratus OCT gave a thinner measurement of the RNFL than the original instrument. As would be expected, there was high correlation between measurements by the two instruments, yet when the differences between measurements were evaluated using Bland-Altman plots, considerable discrepancy between the two instruments was observed. An attempt was made to apply a correction factor to the OCT 2000 measurements to predict the Stratus OCT measurement. Although the correction factor reduced the difference between the OCT measurements, with 75% of eyes having a predicted global RNFL thickness within 10 μm of the observed measurement, the 95% limits of agreement were relatively wide and larger (18 μm) than the limit of resolution (approximately 10 μm) of the Stratus OCT. The correction factors offered by the results of this study may be used to correct OCT 2000 measurements to approximate the result expected from the Stratus OCT on the same eye. However, this remains an approximation. In addition, our estimates are conservative as these were not tested on an independent data set. 

A limitation of our study was that the inclusion criteria for normal subjects included a normal optic nerve appearance judged from examination of stereoscopic optic disc photographs. This criterion was necessary to avoid including subjects with glaucomatous optic neuropathy but normal visual fields in the control group. This inclusion criterion could have overestimated the diagnostic accuracy of OCT instruments. However, this problem is a limitation common to case–control studies of this type and no practical solution to it is currently available. Further, the overestimation of specificity is likely to have affected both instruments to the same degree and therefore would not affect the comparison between the two instruments—the primary purpose of the study. 

Diagnostic accuracy was assessed in this study by using ROC curve areas and sensitivities at fixed specificities. In our study, the greatest area under the curve (AUC) was found with inferior RNFL thickness with both instruments. Other studies ¹³ have also reported this sector to have the greatest AUC when measured with the Stratus OCT and have also shown the inferior RNFL thickness to have the highest AUC, followed by the superior, the nasal, and the temporal sector thicknesses. Zangwill et al. ¹² and Kanamori et al., ¹⁵ using the OCT 2000, also reported that the OCT parameters with the greatest AUC were related to the inferior sector. Our findings differ from those in a study of the OCT 2000 by Nouri-Mahdavi et al., ¹⁶ who reported the superior sector and global thickness to have the greatest AUC. Our finding that the inferior sector had the greatest AUC concurs with findings of others who have reported that the superior visual field is more often affected by glaucoma than the inferior hemifield, with increased susceptibility of the inferior part of the optic disc to glaucomatous damage. ^{17 18}  

An interesting finding was the difference between the two OCT instruments in the discrimination of glaucomatous eyes from healthy eyes, when considering the global RNFL thickness. The better performance of the Stratus OCT may be related to the higher resolution achievable with this instrument. Another advantage of the Stratus OCT was that significantly fewer images were disqualified on account of poor image quality compared with the OCT 2000 instrument. Despite the fact that all patients had a best corrected visual acuity of 20/40 or better, 25% of patients were excluded from the analysis on account of poor-quality images, most having been obtained using the OCT 2000. The lower proportion of poor-quality images obtained with the Stratus OCT may be due, in part, to the higher resolution afforded by this instrument, with the result that fewer images were excluded due to misalignment of the line that demarcates the borders of the RNFL (generated by the thresholding algorithm). 

In conclusion, the RNFL thickness measurements measured by the OCT 2000 can be approximated to measurements made by the Stratus OCT using correction factors calculated by this study. Even with the application of the proposed correction, the agreement between the two instruments may be considered by some as unacceptably low from a clinical standpoint, suggesting that comparisons between instruments using these approximations should be interpreted with caution. Both OCT instruments were found to have high AUCs for RNFL thickness when discriminating glaucomatous from healthy eyes, with the Stratus OCT showing a significantly higher discriminative ability than the OCT 2000, when considering average RNFL thickness. 

 Supported in part by The TFC Frost Trust, United Kingdom (RRAB); National Eye Institute Grant EY11008 (LMZ); and the Albert Einstein College of Medicine (KJ).

 Submitted for publication August 19, 2004; revised October 11, 2004; accepted December 8, 2004.

 Disclosure: R.R.A. Bourne, None; F.A. Medeiros, None; C. Bowd, None; K. Jahanbakhsh, None; L.M. Zangwill, None; R.N. Weinreb, Carl Zeiss Meditec (F)

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Table 1.

 

RNFL Thickness Measured Using the Two Instruments

Variable	Glaucoma Patients (n = 28)	Patients with Suspected Glaucoma (n = 40)	Healthy Control Subjects (n = 36)	P (ANOVA)
OCT 2000
Global	91 ± 18	105 ± 14	112 ± 12	<0.001* , † , ‡
Superior	109 ± 28	126 ± 20	137 ± 17	<0.001* , † , ‡
Temporal	83 ± 18	88 ± 19	85 ± 15	0.49
Inferior	97 ± 27	127 ± 20	135 ± 16	<0.001* , †
Nasal	76 ± 26	80 ± 21	92 ± 19	0.012* , ‡
Stratus OCT
Global	74 ± 15	90 ± 9	99 ± 9	<0.001* , † , ‡
Superior	93 ± 28	115 ± 14	123 ± 15	<0.001* , †
Temporal	60 ± 16	69 ± 15	67 ± 11	0.023* , †
Inferior	82 ± 23	110 ± 17	126 ± 13	<0.001* , † , ‡
Nasal	60 ± 20	68 ± 12	79 ± 17	<0.001* , ‡

 Measurements were taken in each of four sectors around the optic disc in patients with glaucoma, patients with suspected glaucoma, and healthy control subjects. The paired comparisons were analyzed using the Fisher least-significant difference test. Data are expressed as the mean ± SD.

*  Corresponding pairs (glaucomatous and healthy eyes) were significantly different (P < 0.05).

†  Corresponding pairs (patients with glaucoma and patients with suspected glaucoma) were significantly different (P < 0.05).

‡  Corresponding pairs (patients with suspected glaucoma and healthy subjects) were significantly different (P < 0.05).

Figure 1.

 

Bland-Altman plots of the agreement in RNFL thickness measurements between OCT 2000 and Stratus OCT. The difference (OCT 2000 RNFL thickness − Stratus OCT RNFL thickness) was plotted against the average of the two measurements for each sector around the optic disc: (A) global, (B) temporal, (C) superior, (D) nasal, and (E) inferior RNFL thicknesses.

Table 2.

 

Equations for the Regression Lines between OCT 2000 and Stratus OCT Measurements

Variable	Regression between OCT 2000 (x) and Stratus OCT (y)			Difference between Predicted Stratus OCT RNFL Thickness (from OCT 2000 Result) and Observed Stratus OCT
		Equation	R 2	Mean (μm)	95% Confidence Limits (μm)	Eyes (n) in which Difference Is within 10 μm (% of all 109 eyes)
Global	y = 0.6747x + 19.016	0.589	0.08	−18.2, 18.4	82 (75.2)
Superior	y = 0.7466x + 18.567	0.616	0.12	−27.2, 27.4	64 (58.7)
Inferior	y = 0.8337x + 6.4834	0.739	0.12	−24.4, 24.7	53 (48.6)
Temporal	y = 0.4953x + 23.561	0.382	0.10	−21.6, 21.8	72 (66.1)
Nasal	y = 0.484x + 30.11	0.386	0.13	−27.3, 27.6	57 (52.2)

 These equations were used to predict a Stratus OCT result from the observed OCT 2000 measurement; 95% confidence limits of agreement for the difference between observed and predicted Stratus OCT measurements are given in addition to the percentage of predicted Stratus OCT measurements that are within 10 μm of the observed Stratus OCT RNFL thickness.

Figure 2.

 

Data show the relationship between the measurements of global RNFL thickness from the two OCT instruments. (A) The regression line between absolute measurements by the two instruments. The equation for this regression line was then used to predict a value for the Stratus OCT from the OCT 2000 result. (B) Bland-Altman plot showing the differences between observed and predicted Stratus OCT measurements compared with the average of these two measurements.

Table 3.

 

The OCT 2000 and Stratus OCT Performance Parameters

Variable	Area under ROC Curve (Mean ± SE)					Sensitivity (Specificity ≥ 95%)					Sensitivity (Specificity ≥ 80%)
		OCT 2000	Stratus OCT	P	OCT 2000	Stratus OCT	P *	OCT 2000	Stratus OCT	P *
Global	0.84 ± 0.05	0.93 ± 0.03	0.043	54	71	0.063	68	93	0.039
Superior	0.79 ± 0.06	0.81 ± 0.06	0.582	46	54	0.219	71	71	0.999
Inferior	0.89 ± 0.04	0.93 ± 0.04	0.069	61	86	0.063	79	89	0.250
Temporal	0.56 ± 0.07	0.65 ± 0.07	0.167	14	21	0.453	25	36	0.453
Nasal	0.69 ± 0.07	0.75 ± 0.06	0.366	14	43	0.109	64	64	0.999

 Areas under the ROC curves for discriminating between 36 healthy and 28 glaucomatous eyes. Sensitivities are for diagnosing glaucoma at fixed specificities.

*  Calculated with a McNemar test crosstabulation.

Figure 3.

 

ROC curves for global and inferior sector RNFL thickness measurements by the OCT 2000 and Stratus OCT instruments.