Reproducibility of Retinal Nerve Fiber Thickness Measurements Using the Stratus OCT in Normal and Glaucomatous Eyes

Abstract

purpose. To determine the reproducibility of Stratus Optical Coherence Tomography (OCT) retinal nerve fiber layer (RNFL) measurements around the optic nerve in normal and glaucomatous eyes.

methods. One eye was chosen at random from 88 normal subjects and 59 glaucomatous subjects distributed among mild, moderate, and severe glaucoma, determined by visual field testing. Subjects underwent six RNFL thickness measurements performed by a single operator over a 30-minute period with a brief rest between sessions. Three scans were taken with the high-density Standard RNFL protocol, and three were taken with the Fast RNFL protocol, alternating between scan protocols.

results. Reliability, as measured by intraclass correlation coefficient (ICC), was calculated for the overall mean RNFL thickness and for each quadrant. The ICC for the mean Standard RNFL thickness (and lower 95% confidence interval [CI]) in normal and glaucomatous eyes was 0.97 (0.96 CI) and 0.98 (0.97 CI), respectively. The ICC for the mean Fast RNFL thickness in normal and glaucomatous eyes was 0.95 (0.93 CI) and 0.97 (0.95 CI), respectively. Quadrant ICCs ranged between 0.79 and 0.97, with the nasal quadrant being the least reproducible of all four quadrants, using either the Standard or Fast RNFL program. The test–retest variability ranged from 3.5 μm for the average RNFL thickness measurements in normal eyes to 13.8 μm for the nasal quadrant measurements in glaucomatous eyes, which appeared to be the most variable.

conclusions. Reproducibility of RNFL measurements using the Stratus OCT is excellent in normal and glaucomatous eyes. The nasal quadrant appears to be the most variable measurement. Standard RNFL and Fast RNFL scans are equally reproducible and yield comparable measurements. These findings have implications for the diagnosis of glaucoma and glaucomatous progression.

Glaucoma is an optic neuropathy characterized by loss of retinal ganglion cells (RGCs). Currently, the diagnosis of glaucoma and determination of glaucomatous progression are made with a combination of clinical examinations, optic disc photography, and achromatic static perimetry. ¹ Because of variability in the appearance of the optic disc, it is actually a change in the appearance of the cup relative to the disc, ideally documented by optic disc photographs, that typically is used to diagnose glaucoma and its progression by optic disc evaluation. ² These diagnostic tools are relatively insensitive for identifying early progression, because many RGCs are lost before there is a detectable change in optic disc photographs or visual field results. A clinically detectable change in the optic cup represents loss of thousands of axons and may be a relatively insensitive measure of glaucoma damage. Autopsy studies have shown that 30% to 50% of RGCs may be lost before an abnormality appears on standard achromatic perimetry. ^{3 4} In addition, the subjectivity involved in the interpretation of these examination techniques makes them less than ideal. 

Optical coherence tomography (OCT), first described in 1991 by Huang et al., ⁵ is a high-resolution, cross-sectional imaging technique that allows in vivo measurements of the retinal nerve fiber layer (RNFL). The third-generation machine, Stratus OCT (Carl Zeiss Meditec, Dublin, CA), is able to quantify the thickness of the NFL at a resolution of approximately 8 to 10 μm. Thus, clinicians potentially have a more objective tool in helping to diagnose glaucoma and glaucomatous progression earlier than standard techniques. However, in order for any instrument to measure accurately a disease and its progression, it first must be shown to be reproducible for two reasons. First, RNFL thickness measurements obtained with the Stratus OCT are compared with data in a normative database, available since July 2003. If there were significant variability in measurements obtained by the instrument using a single measurement, then comparison to this database would be problematic unless repeated measurements were taken. And second, if there were large variability in measurements when assessing change, then there would have to be an even larger pathologic change in the RNFL thickness for the test to be useful. Demonstrating the level of reproducibility of the RNFL thickness measurements is an important first step in validating Stratus OCT for the diagnosis of glaucoma and glaucomatous progression. The purpose of the present study was to demonstrate the reproducibility of Stratus OCT on the same day, by the same operator, in normal and glaucomatous eyes. 

Materials and Methods

The Human Subjects Subcommittee of the Institutional Review Board of the University of Miami School of Medicine approved the study. Informed consent was obtained from each subject and the study protocol conformed with the Declaration of Helsinki. Normal subjects were primarily recruited from the employee group at the Anne Bates Leach Eye Hospital and the Department of Ophthalmology, University of Miami School of Medicine. Subjects with glaucoma were recruited from the outpatient glaucoma service of the Department of Ophthalmology, University of Miami School of Medicine. 

Each subject had a complete ophthalmic evaluation, including visual acuity and intraocular pressure measurements and slit lamp and fundus examinations. Visual field examinations with the Humphrey Visual Field Analyzer (Carl Zeiss Meditec) were performed on all subjects with glaucoma or suspected glaucoma within 3 months of recruitment. 

Normal subjects had to have best-corrected visual acuity >20/40; normal intraocular pressure (IOP ≤ 21 mm Hg); normal fundus examinations without evidence of optic disc or macular disease, including optic cup-to-disc ratio >0.5 in either eye or asymmetry between the cup and disc ≥0.2, focal thinning of the optic disc rim, optic disc drusen, optic disc pallor, optic disc hemorrhage, age-related macular degeneration, or diabetic retinopathy. Subjects with glaucoma included those with diagnoses of any form of glaucoma, defined as glaucomatous visual field loss with accompanying optic nerve abnormality, and patients designated as having suspected glaucoma, defined as a cup-to-disc ratio >0.5 or asymmetry of cup and disc ≥0.2. Patients with ocular hypertension only, defined as IOP > 21 mm Hg without cup-to-disc ratio >0.5 or optic nerve asymmetry of ≥0.2, were excluded from the glaucoma group. An attempt was made to recruit subjects with glaucoma across a wide spectrum of severity, including those with suspected glaucoma and those with mild, moderate, and severe visual field defects, according to a standard visual field grading scale. ⁶ All subjects with glaucoma had visual field defects that were reproducible on multiple repeat testing. 

The reasons for exclusion in normal subjects included IOP >21 mm Hg in either eye or suspicious-appearing optic nerves; history of amblyopia or of any disease, surgery, or trauma in the eye being tested; abnormal findings in a pupillary examination or history of use of miotic or other medications that might affect pupil size; a history of systemic medication use (e.g., Plaquenil; Sanofi Winthrop, New York, NY) that might affect the visual field or retina; or a history of diabetic retinopathy or age-related macular degeneration. Subjects were not eligible if they had recently used an agent with photosensitizing properties; a history of diabetes mellitus, refractive surgery, or retinal surgery; a history of acute anterior segment disease (e.g., uveitis, conjunctivitis); peripapillary atrophy extending beyond 1.7 mm from the disc center; concomitant retinal or macular disease; unreliable visual fields (fixation losses >33%, false-positive responses >33%, or false negative responses >33%); or inability to obtain at least two adequate OCT images. Reasons for the latter exclusions are presented in the Results section. 

OCT Technique

Subjects were scanned with the Stratus OCT (model 3000, software ver.2.0; Carl Zeiss Meditec, Dublin, CA) three times during the same day, with short breaks between each measurement. In normal subjects, one eye was randomly selected. Measurements involved having the subject seated with the chin in a chin rest and the machine properly aligned. The OCT lens was adjusted for the patient’s refractive error. The contralateral eye was covered. The subject was then instructed to fixate the eye being measured on a target, to bring the optic nerve head within view of the examiner real-time. The Z-offset was adjusted to bring the OCT image into view. Polarization was optimized to maximize the reflective signal. The aiming circle was adjusted by the operator to match the optic nerve head so that the nerve head scan would acquire an OCT image of an even radius of 1.73 mm around the optic nerve head. The scan circle that is placed around the disc has a diameter of 3.46 mm. If the amount of peripapillary atrophy exceeded the scan circle, which was visible and controlled by the operator, the patient was excluded. A single operator collected all measurements, because this initial study was designed to determine the reproducibility of the instrument, rather than of operators. 

Subjects with glaucoma were asked if they wanted to participate during their usual clinical examination. Normal patients received a complete eye examination with refraction to confirm that they had no ocular disease. 

Scan type (Standard and Fast) was alternated, starting with Standard, followed by a Fast scan, then a Standard scan, and so forth, until three sets of each type of scan was obtained. Standard RNFL scans consisted of 512 measurements taken in a circle three times around the optic disc, having a standardized diameter of 3.4 mm, for a total of 1536 data points per scan. Fast RNFL scans were acquired all at once, so such that 256 measurements were taken three times for a total of 768 data points per scan. Between each scan, the patient was asked to sit back and rest for a few minutes before proceeding to the next scan, and the Stratus OCT settings (polarization, Z-offset) were changed between scans so that each scan was treated as a new session. The following criteria were used to assess scan quality: The fundus image must have been clear enough to see the optic disc and the scan circle, color saturation must have been even and dense across the entire scan, and there must have been red color visible in the RPE and RNFL, with no missing or blank area within the scan pattern. The RNFL analysis uses an automated computer algorithm to identify the anterior and posterior margins of the NFL. This delineation is done by the computer’s calculating the boundary where the red reflectivity exceeds a set threshold. The data points between the two white lines delineating the NFL then make up the NFL thickness. The analysis algorithm averages the measurements around the circular scan to obtain 17 numbers per scan. These include the single mean RNFL thickness, the four quadrant averages (temporal, superior, nasal, and inferior), and the 12 clock-hour averages. 

Statistical Methods

The number of subjects (n) and measurements (k) were selected to assure that the lower confidence interval (CI) for an ICC of 0.8 would not be lower than 0.75, which is the generally accepted lower cutoff for good reproducibility. ⁷ If one uses only two measurements per subject, 100 subjects in each group would be necessary to yield an ICC of 0.8 with a lower CI of 0.73. With three measurements per subject, 75 subjects are needed to yield an ICC of 0.8 with a lower CI of 0.74. With five measurements per subject, only 40 subjects are needed for an ICC of 0.8 and a lower CI of 0.73. A design involving three measurements per subject and 75 subjects in each group (normal and glaucoma) was thought to be a reasonable balance between the number of subjects needed and the number of measurements performed on each subject. 

A repeated-measures analysis of variance (Stata, ver. 7.0; Stata Corp., College Station, TX) was performed for each of the RNFL parameters (mean and four quadrants) for both the standard RNFL and Fast RNFL scans. The intraclass correlation coefficient and lower 95% CIs were calculated. Coefficient of variation was calculated using the square root of the residual mean square value divided by the mean thickness. The test–retest variability in RNFL thickness, measured in micrometers, was calculated as two times the SD of the three repeated measurements for each measure of RNFL thickness. Paired Student’s t-tests were used to compare the mean RNFL thicknesses obtained using the Standard RNFL and Fast RNFL scan techniques. Linear regression analysis was performed to compare the mean RNFL thickness by severity of glaucoma (preperimetric or mild, moderate, or severe perimetric defects). 

Results

One hundred fifty-six subjects were enrolled in the study. One normal subject was excluded because of the diagnosis of optic disc drusen. Eight subjects were excluded because of poor analysis quality. One hundred forty-seven subjects, 88 with normal eyes and 59 with eyes having or suspected of having glaucoma were included. In the normal group, there were 66 women and 22 men. In the glaucoma group, there were 36 women and 23 men. The mean (±SD) age of the normal group was 53 ± 18 years (range, 19–88) and the mean age of the glaucoma group was 68 ± 13 years (range, 29–88; P < 0.001). Within the glaucoma group, there were 15 subjects classified as having suspected glaucoma, since they had suspicious-appearing optic discs but no visual field defects. 

ICCs for normal and glaucomatous eyes are presented in Tables 1 and 2 , respectively. ICCs were all excellent, with the mean Standard RNFL and Fast RNFL measurements having the highest ICCs (0.95–0.98) and the nasal quadrant measurements having the lowest, but still excellent, ICCs (0.79–0.88). Even the lower 95% CIs were greater than 0.70, indicating excellent reproducibility of all measurements. 

Test–retest variability in RNFL thickness measurements (Tables 1 and 2)ranged from 3.5 to 4.7 μm in normal eyes and 5.2 to 6.6 μm in glaucomatous eyes. Of the quadrant measurements, the nasal quadrant showed the least reproducibility, from 10.2 to 13.0 μm in normal eyes to 10.2 to 13.8 μm in glaucomatous eyes. Measurements made with the Standard RNFL scan had consistently less variability than measurements made with the Fast RNFL scan. Glaucomatous eyes were slightly more variable than normal eyes. 

Tables 3 and 4provide summary statistics for RNFL thicknesses in normal and glaucomatous eyes, respectively. There were no statistically significant differences between RNFL measurements by the Standard RNFL versus the Fast RNFL scans, although the Fast RNFL scans consistently produced thicker RNFL measurements than did the Standard scans in normal subjects and those with glaucoma. The mean and superior RNFL thickness measurements were significantly thicker when measured with the Fast RNFL scan than with the Standard RNFL scan in normal eyes (Table 3) . 

Discussion

The diagnosis of glaucoma and glaucoma progression is currently relatively subjective, involving a set of characteristic optic nerve findings usually accompanied by visual field changes. It is known that pathologic changes of the optic nerve precede visual field changes to such an extent that 30% to 50% of axons can be lost before any change is seen on the visual field. ^{3 4} This observation has stimulated interest in measuring the RNFL thickness as a potential method of diagnosing glaucoma and its progression before visual field loss is detectable. An important aspect of any test’s ability to detect change is the reproducibility of the measurements. If there is excellent test–retest reproducibility, with little variance in measurements, then identifying small changes in those measurements over time will be meaningful, as long as one accounts for changes that would normally occur with age. However, if there is a great amount of variance on retesting, then relatively large pathologic changes will be needed to diagnose true progression of disease. This concept is commonly referred to as the signal-to-noise ratio. 

Several previous studies using OCT1 and OCT2 technology have addressed the reliability of NFL thickness measurements in normal and glaucomatous eyes. Schuman et al. ⁸ performed RNFL measurements with OCT1 on 21 normal or glaucomatous subjects on five separate occasions during a 1-month period and found ICCs between 0.42 and 0.61, concluding that NFL thickness measurements are reproducible, particularly when fixation is maintained with the eye being tested (internal fixation), as opposed to fixation with the contralateral eye (external fixation). A study in 2000 by Blumenthal et al. ⁹ determined, in a group of 10 normal and 10 glaucomatous eyes, that RNFL thickness measurements by OCT2 were also reproducible within the same session. Similar to our study, the nasal quadrant was the least reproducible, whereas the temporal quadrant was the most reproducible. Also in that study, glaucomatous eyes showed considerably less reproducibility than normal eyes, an effect that was seen to a lesser degree in our study. Jones et al. ¹⁰ studied OCT2 on 15 normal patients in two separate sessions, showing reproducible measurements of the RNFL. Reproducibility was poorer in the nasal peripapillary RNFL measurements in this study as well. In 2003, Carpineto et al. ¹¹ published a study in which they used OCT I to compare 24 patients with glaucoma with 24 age- and gender-matched normal subjects and found that it provided fairly reliable NFL thickness measurements, with ICCs ranging between 0.49 and 0.54. Measurements were performed on five occasions during a 15-day period, which may account for the lower ICCs obtained in this study. Paunescu et al. ¹² reported on the reproducibility of the Stratus OCT in 10 normal subjects aged 23 to 43 years who were scanned six times per day on three different days over a 5-month period. ¹² This study yielded lower ICCs for RNFL thickness measurements than the present study, implying that there may be variation in RNFL thickness in the same person on different days. The reasons for this variability are not known. As in the present study, Paunescu et al. also found that the Standard (high-density scanning) program yielded slightly thinner RNFL measurements than the Fast (lower-density scanning) program, although these differences were not statistically different for most measurements. 

A study by Gurses-Ozden et al. ¹³ showed that reproducibility can be increased by increasing the sampling density, or number of scans performed, used to obtain each measurement. The present study shows better reproducibility using the Standard RNFL scan, which has greater sampling density than the Fast RNFL scan, confirming this result. Paunescu et al. ¹² also generally found that the higher-density scanning program yielded better ICCs. Because the Fast RNFL scan does not save much time compared with the Standard RNFL scan, perhaps the latter should be used to take advantage of the improved reproducibility derived from the increased sampling density. 

There are a few limitations of OCT, especially in relation to the need for clear media and the lack of longitudinal data. Moderate to severe cataracts, particularly posterior subcapsular cataracts, corneal opacities, and vitreous opacities can prevent image acquisition. The patient’s cooperation during the examination is necessary. The patient must be able to see the fixation light and not move during the scan. Last, with increasing age, there is a loss of NFL thickness. This potential problem has been overcome by the incorporation of a normative database program into the software. 

The present study demonstrates that very reproducible measurements of RNFL can be obtained with the Stratus OCT by the same operator on the same day. This is important information, because the usefulness of any scientific test depends on its reproducibility. In addition to the instrument’s being reproducible, reproducibility of RNFL measurements on different days and by different operators must be demonstrated. Also, sensitivity and specificity for glaucoma at different levels of abnormality in a wide range of patients with known glaucoma must be tested. In addition, the effect of other conditions such as cataract, intraocular pressure, pseudophakia, optic disc size, and fixation of the subject must be examined. 

 Presented in part at the annual meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, Florida, April 2004.

 Supported by National Eye Institute Grant P30EY014801 and an unrestricted grant from Research to Prevent Blindness. The Stratus OCT was on loan from Carl Zeiss Meditec, Dublin, California, during the course of this study.

 Submitted for publication October 4, 2004; revised November 12 and December 8, 2004; accepted December 16, 2004.

 Disclosure: D.L. Budenz, Carl Zeiss Meditec (F, R); R.T. Chang, None; X. Huang, Carl Zeiss Meditec (F); R.W. Knighton, Carl Zeiss Meditec (F); J.M. Tielsch, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Table 1.

 

Intraclass Correlation Coefficient, Coefficient of Variation, and Test–Retest Variability in Normal Eyes

	ICC*	Coefficient of Variation, † (%)	Test-Retest Variability, ‡ (μm)
Mean RNFL	0.97 (0.96)	1.7	3.5
Mean Fast RNFL	0.95 (0.93)	2.2	4.7
Temporal RNFL	0.92 (0.88)	5.1	7.5
Temporal Fast RNFL	0.88 (0.84)	6.4	9.5
Superior RNFL	0.91 (0.88)	3.8	9.6
Superior Fast RNFL	0.88 (0.84)	4.5	11.8
Nasal RNFL	0.88 (0.84)	6.7	10.2
Nasal Fast RNFL	0.84 (0.79)	8.2	13.0
Inferior RNFL	0.93 (0.90)	3.7	9.7
Inferior Fast RNFL	0.91 (0.88)	4.3	11.5

*  ICC, with lower 95% CI in parentheses.

†  Calculated as the square root of the residual mean squared values of three repeated measures, divided by mean thickness.

‡  Test–retest variability calculated as two times the SD of three repeated measurements.

Table 2.

 

Intraclass Correlation Coefficient, Coefficient of Variation, and Test–Retest Variability in Glaucomatous Eyes

	ICC*	Coefficient of Variation, † (%)	Test-Retest Variability, ‡ (μm)
Mean RNFL	0.98 (0.97)	3.7	5.2
Mean Fast RNFL	0.97 (0.95)	4.7	6.6
Temporal RNFL	0.97 (0.95)	5.3	5.6
Temporal Fast RNFL	0.94 (0.92)	7.1	7.7
Superior RNFL	0.95 (0.93)	6.4	10.7
Superior Fast RNFL	0.93 (0.90)	7.8	13.7
Nasal RNFL	0.86 (0.81)	9.0	10.2
Nasal Fast RNFL	0.79 (0.72)	11.9	13.8
Inferior RNFL	0.96 (0.95)	6.6	10.6
Inferior Fast RNFL	0.96 (0.94)	7.8	12.7

*  ICC, with lower 95% CI in parentheses.

†  Calculated as the square root of the residual mean squared values of three repeated measures, divided by mean thickness.

‡  Test-retest variability calculated as two times the SD of three repeated measures.

Table 3.

 

RNFL Thickness in Normal Eyes

	RNFL Scan	Fast RNFL Scan	P *
Mean RNFL	101.5 ± 9.8	104.8 ± 10.4	0.009
	(99.4–103.5)	(102.6–107.1)
Temporal RNFL	72.7 ± 13.1	74.8 ± 14.0	0.33
		(70.6–76.2)	(71.8–77.7)
Superior RNFL	125.5 ± 15.8	131.4 ± 17.1	0.017
	(122.2–128.9)	(127.7–135.0)
Nasal RNFL	76.3 ± 14.7	79.0 ± 16.1	0.20
	(73.2–79.4)	(75.6–82.4)
Inferior RNFL	131.5 ± 18.1	134.6 ± 19.0	0.28
	(127.7–135.4)	(130.6–138.6)

 All data are given in micrometers as the mean ± SD, with 95% CI in parentheses.

*  Results of paired t-tests.

Table 4.

 

RNFL Thickness in Glaucomatous Eyes

	RNFL Scan	Fast RNFL Scan	P *
Mean RNFL	69.2 ± 17.1 (64.7–73.6)	70.8 ± 18.8 (65.8–75.8)	0.54
Temporal RNFL	52.6 ± 15.2 (48.6–56.5)	54.3 ± 16.1 (49.9–58.7)	0.73
Superior RNFL	84.5 ± 23.5 (78.4–90.6)	88.5 ± 25.6 (81.6–95.5)	0.37
Nasal RNFL	57.4 ± 13.8 (53.6–60.8)	58.5 ± 15.2 (54.6–62.5)	0.71
Inferior RNFL	81.0 ± 28.2 (74.0–88.1)	81.3 ± 30.6 (73.2–89.4)	0.85

 All data are given in micrometers as the mean ± SD (range).

*  Results of paired t-tests.