August 2012
Volume 53, Issue 9
Free
Clinical and Epidemiologic Research  |   August 2012
Retinal Nerve Fiber Layer Thickness Reproducibility Using Seven Different OCT Instruments
Author Affiliations & Notes
  • Luisa Pierro
    From the Department of Ophthalmology, Vita-Salute University, San Raffaele Scientific Institute, Milan, Italy; and
  • Marco Gagliardi
    From the Department of Ophthalmology, Vita-Salute University, San Raffaele Scientific Institute, Milan, Italy; and
  • Lorenzo Iuliano
    From the Department of Ophthalmology, Vita-Salute University, San Raffaele Scientific Institute, Milan, Italy; and
  • Alessandro Ambrosi
    Vita-Salute San Raffaele University, Milan, Italy.
  • Francesco Bandello
    From the Department of Ophthalmology, Vita-Salute University, San Raffaele Scientific Institute, Milan, Italy; and
  • Corresponding author: Luisa Pierro, San Raffaele Scientific Institute, Via Olgettina, 60, 20132 Milan, Italy; pierro.luisa@hsr.it
Investigative Ophthalmology & Visual Science August 2012, Vol.53, 5912-5920. doi:10.1167/iovs.11-8644
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Luisa Pierro, Marco Gagliardi, Lorenzo Iuliano, Alessandro Ambrosi, Francesco Bandello; Retinal Nerve Fiber Layer Thickness Reproducibility Using Seven Different OCT Instruments. Invest. Ophthalmol. Vis. Sci. 2012;53(9):5912-5920. doi: 10.1167/iovs.11-8644.

      Download citation file:


      © 2015 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements

Purpose. The clinical utility of new optical coherence tomography (OCT) instruments strongly depends on measurements reproducibility. The aim of this study was to assess retinal nerve fiber layer (RNFL) thickness reproducibility using six different spectral-domain OCTs (SD-OCTs) and one time-domain OCT.

Methods. RNFL thickness (average and four quadrant) from six SD-OCTs (Spectral OCT/SLO OPKO/OTI, 3D-OCT 2000 Topcon, RS-3000 NIDEK, Cirrus HD-OCT Zeiss, RTVue-100 Optovue, and Spectralis Heidelberg) and one time-domain OCT (Stratus OCT Zeiss) was measured twice in 38 right eyes of 38 randomly chosen healthy volunteers by two masked operators. Inter- and intraoperator reproducibility was evaluated by the intraclass correlation coefficient (ICC), coefficient of variation (CV), and Bland-Altman test analysis. Instrument-to-instrument reproducibility was determined by ANOVA for repeated measures. We also tested how the devices disagree in terms of systemic bias and random error using a structural equation model.

Results. Mean RNFL average thickness ranged from 90.08 μm to 106.51 μm. Cirrus and Heidelberg showed the thinnest RNFL values in all measurements, Topcon the highest. ICC, CV, and Bland-Altman plots showed variable inter- and intraoperator agreement depending on the instrument. Heidelberg demonstrated the best interoperator (ICC, 0.92; CV, 1.56%) and intraoperator (ICC, 0.94 and 0.95; CV, 1.28% and 1.26%, respectively, for operator A and operator B) agreement for average RNFL thickness.

Conclusions. Heidelberg demonstrated the higher agreement in inter- and intraoperator reproducibility, Optovue the worst. In light of our error analysis results, we found that a scale bias among instruments could interfere with a thorough RNFL monitoring, suggesting that best monitoring is obtained with the same operator and the same device.

Introduction
In the past decades, great attention has been focused on the early diagnosis and monitoring of optic nerve diseases. Indeed, retinal nerve fiber layer (RNFL) thickness is a key diagnostic feature for glaucoma, multiple sclerosis, and optic neuropathies. 113 Quantitative RNFL thickness measurement has become possible with the development of imaging technologies, such as optical coherence tomography (OCT). Recently, the introduction of spectral-domain OCT (SD-OCT) has greatly enhanced the image resolution, decreasing the scan acquisition time, and has also improved measurement reproducibility. 9  
Owing to the increasing number of commercially available SD-OCTs, some patients examined with one OCT instrument may receive subsequent examinations with other OCT devices during the follow-up. Therefore, it is of the utmost importance to evaluate the agreement in RNFL thickness measurements among the different OCT instruments. 
Many studies have offered variable and contrasting results regarding both RNFL thickness and RNFL reproducibility with different OCTs. 5,6,8,1123  
The present investigation was designed to appraise for the first time in literature the agreement among seven commercially available OCT devices, and to evaluate intra- and interoperator reproducibility in assessment of average peripapillary RNFL thickness and four-quadrant values in healthy subjects without ocular pathologies. For a comprehensive approach, we enhanced the statistical analysis by further investigations on device disagreement in terms of systematic bias and random error, using a structural equation model. We highlight the limits of classical statistical test results for a fully comprehensive description of an instrument's reproducibility, even though conventional tests provide an easier approach for the clinical ophthalmologist. 
Methods
Thirty-eight healthy volunteers from the staff of the Department of Ophthalmology of Scientific Institute San Raffaele in Milan, Italy, were recruited between January and December 2010. Informed consent was obtained from the subjects after explanation of the nature and possible consequences of the study. Our research was approved by the institutional review board of the Scientific Institute San Raffaele. Our research adhered to the tenets of the Declaration of Helsinki. The inclusion criteria for all participants consisted of a best-corrected visual acuity of 20/20 or better, spherical refraction between +2.0 and −2.0 diopters (D), axial length <24 mm, normal optic nerve without abnormality of the neuroretinal rim, cup-to-disc ratio greater than 0.2, and normal anterior chamber with open angle. Exclusion criteria were as follows: any ocular disease, history of ocular hypertension or glaucoma, refractive error greater than 2 D, history of ocular surgery, axial length >24 mm. The right eye of each subject was scanned using all of the OCT instruments. The peripapillary RNFL thickness (average and four-quadrant values) was analyzed, comparing six SD-OCTs and one time-domain OCT (Table 1), by two experienced, masked operators in an observational prospective study. Both operators repeated two consecutive measurements on the same day at the same time. For each OCT, results from two separate scan sets were then averaged to generate the final data for each eye. 
Table 1. 
 
Device Technical Specifications
Table 1. 
 
Device Technical Specifications
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Technology Spectral domain Spectral domain Spectral domain Spectral domain Time domain Spectral domain Spectral domain
Software version May 14, 2010 5.0.0.326 4.13 1.2.03 4.02 5.1 5.1
Eye tracking Yes No No No No No No
Scan type (mm) Circle scan (3.46) Raster scan* (6 × 6) Circle scan (3.40) Circle scan (3.45) Circle scan (3.40) Circle scan (3.45) Circle scan (3.45)
A-scan 512 256 1024 1024 256 965 1024
Scanning time, s 1.50 2.40 0.05 1.50 1.92 0.39 Variable†
Measurements of the peripapillary RNFL thickness were obtained by using Spectral OCT/SLO, (OPKO Health Instrumentation, Miami, FL), 3D-OCT 2000, (Topcon, GB Ltd., Newbury, Berkshire, UK), Cirrus HD 100 (Carl Zeiss Meditec, Dublin, CA), OCT RS-3000 (NIDEK, Gamagori, Japan), RTVue-100 (Optovue Inc., Fremont, CA), Spectralis (Heidelberg Engineering, Heidelberg, Germany), and Stratus OCT (Carl Zeiss Meditec) (detailed specifications in Table 1). All subjects underwent a complete ophthalmologic examination, including assessment of LogMAR visual acuity, refractive error, slit-lamp biomicroscopy, intraocular pressure measurement, and fundus examination. For each instrument, the subject was seated and properly aligned, as carried out in everyday practice. If a scan showed a segmentation error, the information was not included in the statistical analysis. Only good-quality images were included in the study. 
Statistical Analysis
Average and four-quadrant RNFL thickness with standard deviation for each eye was computed. 
To determine instrument-to-instrument reproducibility, analysis of variance (ANOVA) for repeated measurements was performed. The reproducibility of all measurements was also evaluated using both the intraclass correlation coefficient (ICC) and the coefficient of variation (CV). Bland-Altman analysis was used to better investigate operator agreement. 
We tested how the devices disagree in terms of systemic bias and random error by using a structural equation model (SEM). This analysis extracts the calibration equations and the scale-adjusted imprecision standard deviations. As ANOVA provides a statistical test of whether or not the means of several groups are all equal, the calibration equations describe the relative bias (systematic error) between each pair of instruments, and may convert measurements from one device into the equivalent measurement for another device. On the other hand, while the CV is a general normalized measure of measurement dispersion, the scale-adjusted imprecision standard deviations measure the spread of the random portion of the measurement error, estimating the single-device imprecision. 
Statistical Package for Social Science (SPSS) software (version 18; SPSS Inc., Chicago, IL) was used for ANOVA test, ICC, CV, and Bland-Altman analysis. 
Measurement error was analyzed with the merror package 24 for the R statistical software, 25 using the functions merror.pairs, ncb.od, lrt, and cplot. 
Results
Thirty-eight healthy volunteers were enrolled in the study. Mean age was 42 ± 10.8 years; 25 were females. Mean RNFL average thickness measured by different instruments ranged from 90.08 μm to 106.51 μm. 
Cirrus HD-OCT and Spectralis HRA+OCT showed thinner RNFL thickness (average RNFL thickness was 90.08 μm and 93.30 μm, respectively), whereas Topcon 3D-OCT 2000 showed the highest value (average RNFL thickness was 106.51 μm). As expected, RNFL thickness was higher in superior and inferior quadrants than in nasal and temporal quadrants, using all the instruments (instrument factor, P < 0.001; post hoc test for differences among instruments, P < 0.005 against all in all sectors) (Table 2). 
Table 2. 
 
RNFL Thickness Mean Values and Standard Deviations of Seven Different OCT Instruments
Table 2. 
 
RNFL Thickness Mean Values and Standard Deviations of Seven Different OCT Instruments
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Average 103.58 7.26 90.08 6.28 106.51 8.35 102.43 6.54 99.63 8.62 103.90 6.16 93.30 5.18
Nasal 88.76 13.50 69.77 7.52 95.77 15.48 86.32 11.27 80.08 12.87 83.2 13.53 70.73 8.12
Temporal 71.98 14.53 63.31 12.35 81.82 15.19 71.76 12.49 70.14 14.38 79 11.84 70 15.46
Inferior 128.77 9.90 116.96 11.39 127.83 14.25 128.41 7.71 126.16 12.64 129.73 13.14 120.44 8.78
Superior 124.25 13.49 109.72 10.97 120.11 15.91 122.72 14.90 121.27 16.35 123.65 16.39 111.74 11.15
Interoperator average reproducibility showed that the worst correlation was obtained with Optovue RTVue-100 and Topcon 3D-OCT 2000, whereas the best correlation was found with Heidelberg Spectralis (ICC, 0.92; CV, 1.65%), Zeiss Cirrus HD-OCT (ICC, 0.90; CV, 2.20%), and Zeiss Stratus (ICC, 0.91; CV, 2.01%), according to the ICC and the CV tests (Table 3) and the Bland-Altman test (Fig. 1). 
Figure 1. 
 
Interoperator RNFL average thickness reproducibility using seven different OCT instruments: Bland-Altman plots.
Figure 1. 
 
Interoperator RNFL average thickness reproducibility using seven different OCT instruments: Bland-Altman plots.
Table 3. 
 
Interoperator RNFL Reproducibility of Seven Different OCT Instruments
Table 3. 
 
Interoperator RNFL Reproducibility of Seven Different OCT Instruments
OPKO OTIOCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, %
Average 0.87 2.89 0.90 2.20 0.80 3.18 0.87 2.46 0.91 2.01 0.57 3.35 0.92 1.65
Nasal 0.71 7.32 0.88 4.19 0.57 8.29 0.78 7.13 0.78 6.36 0.46 7.95 0.72 4.78
Temporal 0.83 6.30 0.64 4.99 0.54 8.10 0.63 7.29 0.89 4.47 0.75 7.18 0.86 4.20
Inferior 0.76 4.84 0.92 3.47 0.69 4.89 0.62 3.92 0.85 3.56 0.86 4.20 0.76 3.27
Superior 0.82 5.52 0.82 3.77 0.81 5.42 0.94 4.63 0.92 3.08 0.60 5.22 0.86 3.03
When comparing the interoperator sector reproducibility, Heidelberg Spectralis, Zeiss Stratus, NIDEK RS-3000, and Topcon 3D-OCT 2000 gave the best results for the superior sector. Zeiss Cirrus and Optovue RTVue-100, on the contrary, gave the best results for the inferior sector, while OPKO OTI OCT/SLO and Heidelberg Spectralis, once again, gave the best results for the temporal sector. On the contrary, the lowest reproducibility was observed with Heidelberg Spectralis, Zeiss Cirrus, Optovue RTVue-100, and OPKO OTI OCT/SLO in the nasal sector; with Topcon 3D-OCT 2000 and Zeiss Cirrus in the temporal sector; and with NIDEK RS-3000 in the inferior sector. 
As expected, intraoperator reproducibility was better than interoperator reproducibility for all instruments (Tables 4 and 5), both in average and sectorial thickness measurements. 
Table 4. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: ICC
Table 4. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: ICC
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B
Average 0.94 0.94 0.92 0.93 0.91 0.92 0.92 0.92 0.97 0.97 0.76 0.70 0.94 0.95
Nasal 0.76 0.81 0.81 0.69 0.74 0.87 0.70 0.63 0.90 0.92 0.79 0.70 0.73 0.78
Temporal 0.86 0.89 0.94 0.62 0.77 0.73 0.73 0.75 0.97 0.96 0.50 0.53 0.95 0.85
Inferior 0.67 0.88 0.87 0.89 0.82 0.89 0.80 0.86 0.90 0.88 0.50 0.42 0.85 0.87
Superior 0.82 0.89 0.89 0.89 0.86 0.90 0.93 0.83 0.96 0.97 0.86 0.69 0.91 0.89
Table 5. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: CV (%)
Table 5. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: CV (%)
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B
Average 1.85 1.86 1.90 1.89 2.08 2.10 1.79 1.79 1.17 1.16 2.17 2.18 1.28 1.26
Nasal 5.68 5.70 3.63 3.61 6.15 6.02 6.50 6.40 3.96 3.90 3.96 4.02 4.01 3.91
Temporal 4.12 4.10 2.88 2.90 5.53 5.55 5.07 5.08 2.43 2.44 7.35 7.34 3.06 3.07
Inferior 4.45 4.35 2.95 3.97 3.66 3.61 3.16 3.10 2.65 2.66 2.82 3.02 2.67 2.71
Superior 3.88 3.90 3.20 3.22 3.83 3.80 3.26 3.32 2.24 2.24 2.89 3.03 2.71 2.71
Calibration equations for the seven tested instruments are shown in Table 6. Any measurement from a given device could be converted to an equivalent measurement from another device. The calibration equations describe the relative bias (systematic error) between each pair of instruments. A substantial bias is evident between many of the instruments. By way of example, comparing Optovue RTVue-100 to OPKO OTI OCT/SLO, measurements made on Optovue RTVue-100 must be almost doubled (1.916) after first subtracting 96.248 in order to obtain the equivalent measurement on the Optko OTI OCT/SLO instrument. Accordingly, 1.916 RTVue units equal 1 OTI OCT/SLO unit. On the other hand, NIDEK OCT RS-3000 units are nearly equal to OTI OCT/SLO units (1.066 NIDEK equals 1 OTI). These relationships are illustrated in the calibration curves (Fig. 2): the closer the calibration line (red) is to the no-bias line (blue), the smaller is the systematic error between two instruments, and vice versa. Solid black circles show the paired measurements on each device. An average graphic overview of all the calibration equations for the seven instruments is presented in Figure 3. Blue lines represent the no-bias line. 
Figure 2. 
 
Calibration curves for Optovue RTVue-100 (right) and NIDEK OCT RS-3000 (left) using OPKO OTI OCT/SLO as reference. OPKO OTI OCT/SLO values are plotted on the y-axis. Black spots describe the true corresponding measurement among coupled devices.
Figure 2. 
 
Calibration curves for Optovue RTVue-100 (right) and NIDEK OCT RS-3000 (left) using OPKO OTI OCT/SLO as reference. OPKO OTI OCT/SLO values are plotted on the y-axis. Black spots describe the true corresponding measurement among coupled devices.
Figure 3. 
 
Overview of all calibration equations for the seven instruments. Green lines represent the no-bias line, while black spots describe the true corresponding measurement among coupled devices.
Figure 3. 
 
Overview of all calibration equations for the seven instruments. Green lines represent the no-bias line, while black spots describe the true corresponding measurement among coupled devices.
Table 6. 
 
Calibration Equations for the Seven Tested Devices
Table 6. 
 
Calibration Equations for the Seven Tested Devices
Device Calibration Equation
OPKO OTI OCT/SLO −22.033 + 1.391* Zeiss Cirrus
−36.345 + 1.296* Topcon 3D-OCT 2000
−5.997 + 1.066* NIDEK OCT RS-3000
−45.559 + 1.52 Zeiss Stratus
−96.248 + 1.916* Optovue RTVue-100
−17.904 + 1.311* Heidelberg Spectralis
Zeiss Cirrus 15.843 + 0.719* OPKO OTI OCT/SLO
−10.292 + 0.932* Topcon 3D-OCT 2000
11.531 + 0.767* NIDEK OCT RS-3000
−16.917 + 1.093* Zeiss Stratus
−53.366 + 1.377* Optovue RTVue-100
2.969 + 0.942* Heidelberg Spectralis
Topcon 3D-OCT 2000 28.05 + 0.772* OPKO OTI OCT/SLO
11.046 + 1.073* Zeiss Cirrus
23.422 + 0.823* NIDEK OCT RS-3000
−7.111 + 1.173* Zeiss Stratus
−46.231 + 1.478* Optovue RTVue-100
14.232 + 1.012* Heidelberg Spectralis
NIDEK OCT RS-3000 5.623 + 0.938* OPKO OTI OCT/SLO
−15.0.37 + 1.304* Zeiss Cirrus
−28.459 + 1.215* Topcon 3D-OCT 2000
−37.099 + 1.425* Zeiss Stratus
−84.632 + 1.796* Optovue RTVue-100
−11.166 + 1.229* Heidelberg Spectralis
Zeiss Stratus 29.982 + 0.658* OPKO OTI OCT/SLO
15.482 + 0.915* Zeiss Cirrus
6.063 + 0.853* Topcon 3D-OCT 2000
26.035 + 0.702* NIDEK OCT RS-3000
−33.359 + 1.261* Optovue RTVue-100
18.199 + 0.863* Heidelberg Spectralis
Optovue RTVue-100 50.243 + 0.522* OPKO OTI OCT/SLO
38.742 + 0.726* Zeiss Cirrus
31.271 + 0.676* Topcon 3D-OCT 2000
47.113 + 0.557* NIDEK OCT RS-3000
26.461 + 0.793* Zeiss Stratus
40.897 + 0.684* Heidelberg Spectralis
Heidelberg Spectralis 13.66 + 0.763* OPKO OTI OCT/SLO
−3.15 + 1.061* Zeiss Cirrus
−14.07 + 0.989* Topcon 3D-OCT 2000
9.085 + 0.804* NIDEK OCT RS-3000
−21.1 + 1.159* Zeiss Stratus
−59.774 + 1.462* Optovue RTVue-100
Scale-adjusted imprecision standard deviations for every instrument, measuring the random portion of the measurement error, are shown in Table 7. The units were transformed to the same scale as that of OPKO Spectral OTI OCT/SLO for interinstrument comparison (although the equivalent comparisons could be made using any of the devices as the reference). Evaluation of device imprecision showed the same trend of interoperator reproducibility. Optovue RTVue-100 and Zeiss Stratus were nearly the same and had the highest imprecision, while Heidelberg Spectralis had the lowest (slightly less than the OPKO OTI) (Table 7). 
Table 7. 
 
Overall* Scale-Adjusted Imprecision Standard Deviations for the Seven Tested Devices and Ratios with OPKO OTI OCT/SLO
Table 7. 
 
Overall* Scale-Adjusted Imprecision Standard Deviations for the Seven Tested Devices and Ratios with OPKO OTI OCT/SLO
Device Overall Operators Ratio with OTI
OPKO OTI OCT/SLO 2.52 1.00
Zeiss Cirrus 3.855 1.53
Topcon 3D-OCT 2000 5.903 2.34
NIDEK OCT RS-3000 2.957 1.17
Zeiss Stratus 6.435 2.55
Optovue RTVue-100 6.386 2.53
Heidelberg Spectralis 2.046 0.81
Discussion
The capability of imaging instruments to provide additional information to traditional examination is essential to achieve an early diagnosis in optic nerve disorders. 
Our study showed that average RNFL thickness measurement carried out by several different instruments generates significantly different average and sector values, as already reported 20,26 (Table 2). It is plausible that segmentation differences in the definition of the outer border of RNFL and optical interaction with tissue due to different light sources and laser camera system (LCS) sensor may determine this variability. 27  
As expected, we found that each sector followed the anatomic thickness distribution with all tested instruments, with the maximum thickness in the inferior sector and reduction through superior, nasal, and temporal sectors. 26  
Our study is different from those already published because previously published studies have compared no more than groups of two or three instruments, while in our research, all the commercially available instruments were compared. Furthermore, all RNFL quadrants were considered. Thus, the present study offered a wider evaluation of all the currently available OCT instruments. 123,26,28  
We obtained a greater variability among instruments either for thickness or reproducibility. In particular, we analyzed the potential influence of different conditions related to RNFL thickness. 
First of all, we considered different standard diameters. It is well known that RNFL thickness increases with increasing proximity to the optic disc. We hypothesize that measurements closer than 3.4-mm diameter around the disc, as with Stratus and Topcon 3D-OCT2000, may explain the greater thickness of the nerve. 26,29  
Spectral OCT/SLO, Cirrus, RTVue, Spectralis, and NIDEK work with a 3.46-mm diameter. Nevertheless, Topcon's RNFL assessment had a higher value (106.51 mm) than that of Stratus (99.63 mm), even if both devices have the same scanning diameter (3.40 mm). 
We also analyzed the influence of image quality on RNFL variability. Even if we acquired only images with the same quality level value (>6), the signal strengths differ among the instruments 13 and may determine the variability that we found among RNFL thickness measurements. 
The effect of blood vessels was also studied. The presence of blood vessels around the optic disc can modify the optic nerve profile and may interfere with thickness. Specifically, comparing Topcon, Heidelberg, and OPKO OCTs, the vascular patterns around the optic disc were obtained by the test–retest function of the instrument and automatically identified in successive scans. It is noteworthy that even though Topcon, Heidelberg, and OPKO have the same options, the results greatly differed. 
In an attempt to investigate the effects on reproducibility, other aspects should be considered. 
First of all, the differences among the instrument scan circle placement may greatly influence the RNFL thickness measurement. In clinical practice, accurate centering of the measurement circle can be difficult. Imprecise measurement caused by an off-center scan circle may be a source of measurement variability. Cirrus has completely automatic scan circle placement obtained by a raster cube scan, whereas all the other OCTs require manual placement of the circular scan set down of the optic disc edge by the operator. RTVue uses a combination of radial and circular scans that require more software interpolation. 27 However, reproducibility results are good also in devices with manual placement of the circle scan. 
Another condition refers to the incidence angle of the illuminating beam that may produce different responses. Although some authors have demonstrated that the angle of incidence of the illuminating beam makes the RNFL image on the nasal side dimmer, and therefore less identifiable by the measurement algorithm and also less reproducible, 9,13,17,19 we noticed a wide reproducibility variability involving all the sectors, and not only in the nasal sector, as we have previously reported. 9,13,19  
Moreover, we tried to evaluate the effect of eye tracking. Eye tracking can improve reproducibility, as happens with Spectralis and OCT/SLO, but it does not seem fundamental if we consider that Stratus, which is the least updated of the available instruments and does not have this function, shows excellent reproducibility. 3,14,29  
Eventually, the scanning time may also theoretically affect reproducibility. The instrument scanning time is also very different among OCTs. Spectral OCT/SLO and NIDEK have similar scanning time (1.5 seconds for three circle scans), whereas that of Stratus and Spectralis is 1.92 seconds, and that of Cirrus is 2.4 seconds. On the other hand, Topcon has 0.05 seconds for each circle scan and RTVue has 0.39 seconds, suggesting that fast scanning times do not necessarily mean good reproducibility. Therefore, scanning time should not be considered as an influencing factor for reproducibility. 
In conclusion, great variability in thickness and reproducibility can be registered among different OCT instruments, for both average and sector values. As already shown in another study, we confirmed that worse reproducibility and higher imprecision was found in RTVue, probably owing to the different scanning pattern (requiring alignment) and interpolation, while a raster scan or a circular scan associated with eye tracking are less heavily processed. 27  
A comprehensive analysis of the cited data must include the evaluation of device disagreement in terms of systematic bias and random error. Classical measurement error model can be described as an SEM, whose parameters include the distorting factors of the unknown true values. All cited results must be therefore interpreted by considering the single-device imprecision, which can be estimated by the scale-adjusted imprecision standard deviation. This value is calculated along with the calibration equation, which makes it possible to convert measurements from one device into an equivalent measurement for another device. These quantities have the same scale and are comparable between devices and operators. Follow-up measurements using different devices could be theoretically compared using these equations, but further prospective studies with larger samples are required. 
Statistical methods such as ANOVA, CV, and ICC are only appropriate when it has first been determined that there is no scale bias. When the calibration results indicate a scale bias, these statistical methods are not appropriate. 27 Therefore, we recommend to consider the limits of classical statistical test results for a fully comprehensive description of an instrument's reproducibility, even though conventional tests provide an easier approach for the clinical ophthalmologist. 
In the absence of a clear gold standard demonstrating the real RNFL thickness, it is difficult to establish the most accurate assessment of each instrument. In light of our error analysis results, we found that a scale bias among instruments could interfere with a thorough RNFL monitoring, suggesting that best monitoring is obtained with the same operator and the same device. Furthermore, our results seem to overlap those obtained for macular thickness. 30  
This research confirmed that SD-OCTs best apply in qualitative and morphologic evaluation and show a great variance in quantitative measurements, which requires careful data interpretation. 
References
Li JP Wang XZ Fu J Li SN Wang NL. Reproducibility of RTVue retinal nerve fiber layer thickness and optic nerve head measurements in normal and glaucoma eyes. Chin Med J . 2010;123:1898–1903.
Antón A Castany M Pazos-Lopez M Cuadrado R Flores A Castilla M. Reproducibility of measurements and variability of the classification algorithm of Stratus OCT in normal, hypertensive, and glaucomatous patients. Clin Ophthalmol . 2009;3:139–145.
Labiris G Giarmoukakis A Katsanos A Reproducibility of OCT/SLO measurements in healthy eyes. Eur J Ophthalmol . 2010;20:552–558.
Strouthidis NG Fortune B Yang H Sigal IA Burgoyne CF. Longitudinal change detected by spectral domain optical coherence tomography in the optic nerve head and peripapillary retina in experimental glaucoma. Invest Ophthalmol Vis Sci . 2011;52:1206–1219.
Seibold LK Mandava N Kahook MY. Comparison of retinal nerve fiber layer thickness in normal eyes using time-domain and spectral-domain optical coherence tomography. Am J Ophthalmol . 2010;150:807–814.
Shin CJ Sung KR Um TW Comparison of retinal nerve fibre layer thickness measurements calculated by the optic nerve head map (NHM4) and RNFL3.45 modes of spectral-domain optical coherence tomography (RTVue-100). Br J Ophthalmol . 2010;94:763–767.
Lee SH Kim SH Kim TW Park KH Kim DM. Reproducibility of retinal nerve fiber thickness measurements using the test-retest function of spectral OCT/SLO in normal and glaucomatous eyes. J Glaucoma . 2010;19:637–642.
Bourne RR Medeiros FA Bowd C Jahanbakhsh K Zangwill LM Weinreb RN. Comparability of retinal nerve fiber layer thickness measurements of optical coherence tomography instruments. Invest Ophthalmol Vis Sci . 2005;46:1280–1285.
Leite MT Rao HL Weinreb RN Agreement among spectral-domain optical coherence tomography instruments for assessing retinal nerve fiber layer thickness. Am J Ophthalmol . 2011;151:85–92.
Leung CK Choi N Weinreb RN Retinal nerve fiber layer imaging with spectral-domain optical coherence tomography: pattern of RNFL defects in glaucoma. Ophthalmology . 2010;117:2337–2344.
Blumenthal EZ Williams JM Weinreb RN Girkin CA Berry CC Zangwill LM. Reproducibility of nerve fiber layer thickness measurements by use of optical coherence tomography. Ophthalmology . 2000;107:2278–2282.
Gyatsho J Kaushik S Gupta A Pandav SS Ram J. Retinal nerve fiber layer thickness in normal, ocular hypertensive, and glaucomatous Indian eyes: an optical coherence tomography study. J Glaucoma . 2008;17:122–127.
Sung KR Kim DY Park SB Kook MS. Comparison of retinal nerve fiber layer thickness measured by Cirrus HD and Stratus optical coherence tomography. Ophthalmology . 2009;116:1264–1270.
Moreno-Montañés J Antón A García N Olmo N Morilla A Fallon M. Comparison of retinal nerve fiber layer thickness values using Stratus Optical Coherence Tomography and Heidelberg Retina Tomograph-III. J Glaucoma . 2009;18:528–534.
González-García AO Vizzeri G Bowd C Medeiros FA Zangwill LM Weinreb RN. Reproducibility of RTVue retinal nerve fiber layer thickness and optic disc measurements and agreement with Stratus optical coherence tomography measurements. Am J Ophthalmol . 2009;147:1067–1074.
Johnson DE El-Defrawy SR Almeida DR Campbell RJ. Comparison of retinal nerve fibre layer measurements from time domain and spectral domain optical coherence tomography systems. Can J Ophthalmol . 2009;44:562–566.
Hong S Kim CY Lee WS Seong GJ. Reproducibility of peripapillary retinal nerve fiber layer thickness with spectral domain cirrus high-definition optical coherence tomography in normal eyes. Jpn J Ophthalmol . 2010;54:43–47.
Marsh BC Cantor LB WuDunn D Optic nerve head (ONH) topographic analysis by stratus OCT in normal subjects: correlation to disc size, age, and ethnicity. J Glaucoma . 2010;19:310–318.
Vizzeri G Weinreb RN Gonzalez-Garcia AO Agreement between spectral-domain and time-domain OCT for measuring RNFL thickness. Br J Ophthalmol . 2009;93:775–781.
Huang J Liu X Wu Z Macular and retinal nerve fiber layer thickness measurements in normal eyes with the Stratus OCT, the Cirrus HD-OCT, and the Topcon 3D OCT-1000. J Glaucoma . 2011;20:118–125.
Kim JS Ishikawa H Gabriele ML Retinal nerve fiber layer thickness measurement comparability between time domain optical coherence tomography (OCT) and spectral domain OCT. Invest Ophthalmol Vis Sci . 2010;51:896–902.
Funaki S Shirakashi M Abe H. Relation between size of optic disc and thickness of retinal nerve fibre layer in normal subjects. Br J Ophthalmol . 1998;82:1242–1245.
Mwanza JC Durbin MK Budenz DL; Cirrus OCT Normative Database Study Group. Interocular symmetry in peripapillary retinal nerve fiber layer thickness measured with the Cirrus HD-OCT in healthy eyes. Am J Ophthalmol . 2011;151:514–521.
Bilonick RA. merror: Accuracy and Precision of Measurements . R package version 2.0.1. 2012. Available at: http://CRAN.R-project.org/package=merror. Accessed January 18, 2012.
R Core Team 2012. R: A Language and Environment for Statistical Computing . Vienna, Austria: R Foundation for Statistical Computing; 2012. Available at: http://www.R-project.org/. Accessed January 18, 2012.
Menke MN Knecht P Sturm V Dabov S Funk J. Reproducibility of nerve fiber layer thickness measurements using 3D fourier-domain OCT. Invest Ophthalmol Vis Sci . 2008;49:5386–5391.
Buchser NM Wollstein G Ishikawa H Comparison of retinal nerve fiber layer thickness measurement bias and imprecision across three spectral-domain optical coherence tomography devices [published online ahead of print April 26, 2012]. Invest Ophthalmol Vis Sci . doi:10.1167/iovs.11-8432 .
Savini G Carbonelli M Barboni P. Retinal nerve fiber layer thickness measurement by Fourier-domain optical coherence tomography: a comparison between cirrus-HD OCT and RTVue in healthy eyes. J Glaucoma . 2010;19:369–372.
Bendschneider D Tornow RP Horn FK Retinal nerve fiber layer thickness in normals measured by spectral domain OCT. J Glaucoma . 2010;19:475–482.
Pierro L Giatsidis SM Mantovani E Gagliardi M. Macular thickness interoperator and intraoperator reproducibility in healthy eyes using 7 optical coherence tomography instruments. Am J Ophthalmol . 2010;150:199–204.
 
 
 
 
Figure 1. 
 
Interoperator RNFL average thickness reproducibility using seven different OCT instruments: Bland-Altman plots.
Figure 1. 
 
Interoperator RNFL average thickness reproducibility using seven different OCT instruments: Bland-Altman plots.
Figure 2. 
 
Calibration curves for Optovue RTVue-100 (right) and NIDEK OCT RS-3000 (left) using OPKO OTI OCT/SLO as reference. OPKO OTI OCT/SLO values are plotted on the y-axis. Black spots describe the true corresponding measurement among coupled devices.
Figure 2. 
 
Calibration curves for Optovue RTVue-100 (right) and NIDEK OCT RS-3000 (left) using OPKO OTI OCT/SLO as reference. OPKO OTI OCT/SLO values are plotted on the y-axis. Black spots describe the true corresponding measurement among coupled devices.
Figure 3. 
 
Overview of all calibration equations for the seven instruments. Green lines represent the no-bias line, while black spots describe the true corresponding measurement among coupled devices.
Figure 3. 
 
Overview of all calibration equations for the seven instruments. Green lines represent the no-bias line, while black spots describe the true corresponding measurement among coupled devices.
Table 1. 
 
Device Technical Specifications
Table 1. 
 
Device Technical Specifications
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Technology Spectral domain Spectral domain Spectral domain Spectral domain Time domain Spectral domain Spectral domain
Software version May 14, 2010 5.0.0.326 4.13 1.2.03 4.02 5.1 5.1
Eye tracking Yes No No No No No No
Scan type (mm) Circle scan (3.46) Raster scan* (6 × 6) Circle scan (3.40) Circle scan (3.45) Circle scan (3.40) Circle scan (3.45) Circle scan (3.45)
A-scan 512 256 1024 1024 256 965 1024
Scanning time, s 1.50 2.40 0.05 1.50 1.92 0.39 Variable†
Table 2. 
 
RNFL Thickness Mean Values and Standard Deviations of Seven Different OCT Instruments
Table 2. 
 
RNFL Thickness Mean Values and Standard Deviations of Seven Different OCT Instruments
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD
Average 103.58 7.26 90.08 6.28 106.51 8.35 102.43 6.54 99.63 8.62 103.90 6.16 93.30 5.18
Nasal 88.76 13.50 69.77 7.52 95.77 15.48 86.32 11.27 80.08 12.87 83.2 13.53 70.73 8.12
Temporal 71.98 14.53 63.31 12.35 81.82 15.19 71.76 12.49 70.14 14.38 79 11.84 70 15.46
Inferior 128.77 9.90 116.96 11.39 127.83 14.25 128.41 7.71 126.16 12.64 129.73 13.14 120.44 8.78
Superior 124.25 13.49 109.72 10.97 120.11 15.91 122.72 14.90 121.27 16.35 123.65 16.39 111.74 11.15
Table 3. 
 
Interoperator RNFL Reproducibility of Seven Different OCT Instruments
Table 3. 
 
Interoperator RNFL Reproducibility of Seven Different OCT Instruments
OPKO OTIOCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, % ICC CV, %
Average 0.87 2.89 0.90 2.20 0.80 3.18 0.87 2.46 0.91 2.01 0.57 3.35 0.92 1.65
Nasal 0.71 7.32 0.88 4.19 0.57 8.29 0.78 7.13 0.78 6.36 0.46 7.95 0.72 4.78
Temporal 0.83 6.30 0.64 4.99 0.54 8.10 0.63 7.29 0.89 4.47 0.75 7.18 0.86 4.20
Inferior 0.76 4.84 0.92 3.47 0.69 4.89 0.62 3.92 0.85 3.56 0.86 4.20 0.76 3.27
Superior 0.82 5.52 0.82 3.77 0.81 5.42 0.94 4.63 0.92 3.08 0.60 5.22 0.86 3.03
Table 4. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: ICC
Table 4. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: ICC
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B
Average 0.94 0.94 0.92 0.93 0.91 0.92 0.92 0.92 0.97 0.97 0.76 0.70 0.94 0.95
Nasal 0.76 0.81 0.81 0.69 0.74 0.87 0.70 0.63 0.90 0.92 0.79 0.70 0.73 0.78
Temporal 0.86 0.89 0.94 0.62 0.77 0.73 0.73 0.75 0.97 0.96 0.50 0.53 0.95 0.85
Inferior 0.67 0.88 0.87 0.89 0.82 0.89 0.80 0.86 0.90 0.88 0.50 0.42 0.85 0.87
Superior 0.82 0.89 0.89 0.89 0.86 0.90 0.93 0.83 0.96 0.97 0.86 0.69 0.91 0.89
Table 5. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: CV (%)
Table 5. 
 
Intraoperator RNFL Reproducibility of Seven Different OCT Instruments: CV (%)
OPKO OTI OCT/SLO Zeiss Cirrus Topcon 3D-OCT 2000 NIDEK RS-3000 Zeiss Stratus Optovue RTVue-100 Heidelberg Spectralis
Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B Operator A Operator B
Average 1.85 1.86 1.90 1.89 2.08 2.10 1.79 1.79 1.17 1.16 2.17 2.18 1.28 1.26
Nasal 5.68 5.70 3.63 3.61 6.15 6.02 6.50 6.40 3.96 3.90 3.96 4.02 4.01 3.91
Temporal 4.12 4.10 2.88 2.90 5.53 5.55 5.07 5.08 2.43 2.44 7.35 7.34 3.06 3.07
Inferior 4.45 4.35 2.95 3.97 3.66 3.61 3.16 3.10 2.65 2.66 2.82 3.02 2.67 2.71
Superior 3.88 3.90 3.20 3.22 3.83 3.80 3.26 3.32 2.24 2.24 2.89 3.03 2.71 2.71
Table 6. 
 
Calibration Equations for the Seven Tested Devices
Table 6. 
 
Calibration Equations for the Seven Tested Devices
Device Calibration Equation
OPKO OTI OCT/SLO −22.033 + 1.391* Zeiss Cirrus
−36.345 + 1.296* Topcon 3D-OCT 2000
−5.997 + 1.066* NIDEK OCT RS-3000
−45.559 + 1.52 Zeiss Stratus
−96.248 + 1.916* Optovue RTVue-100
−17.904 + 1.311* Heidelberg Spectralis
Zeiss Cirrus 15.843 + 0.719* OPKO OTI OCT/SLO
−10.292 + 0.932* Topcon 3D-OCT 2000
11.531 + 0.767* NIDEK OCT RS-3000
−16.917 + 1.093* Zeiss Stratus
−53.366 + 1.377* Optovue RTVue-100
2.969 + 0.942* Heidelberg Spectralis
Topcon 3D-OCT 2000 28.05 + 0.772* OPKO OTI OCT/SLO
11.046 + 1.073* Zeiss Cirrus
23.422 + 0.823* NIDEK OCT RS-3000
−7.111 + 1.173* Zeiss Stratus
−46.231 + 1.478* Optovue RTVue-100
14.232 + 1.012* Heidelberg Spectralis
NIDEK OCT RS-3000 5.623 + 0.938* OPKO OTI OCT/SLO
−15.0.37 + 1.304* Zeiss Cirrus
−28.459 + 1.215* Topcon 3D-OCT 2000
−37.099 + 1.425* Zeiss Stratus
−84.632 + 1.796* Optovue RTVue-100
−11.166 + 1.229* Heidelberg Spectralis
Zeiss Stratus 29.982 + 0.658* OPKO OTI OCT/SLO
15.482 + 0.915* Zeiss Cirrus
6.063 + 0.853* Topcon 3D-OCT 2000
26.035 + 0.702* NIDEK OCT RS-3000
−33.359 + 1.261* Optovue RTVue-100
18.199 + 0.863* Heidelberg Spectralis
Optovue RTVue-100 50.243 + 0.522* OPKO OTI OCT/SLO
38.742 + 0.726* Zeiss Cirrus
31.271 + 0.676* Topcon 3D-OCT 2000
47.113 + 0.557* NIDEK OCT RS-3000
26.461 + 0.793* Zeiss Stratus
40.897 + 0.684* Heidelberg Spectralis
Heidelberg Spectralis 13.66 + 0.763* OPKO OTI OCT/SLO
−3.15 + 1.061* Zeiss Cirrus
−14.07 + 0.989* Topcon 3D-OCT 2000
9.085 + 0.804* NIDEK OCT RS-3000
−21.1 + 1.159* Zeiss Stratus
−59.774 + 1.462* Optovue RTVue-100
Table 7. 
 
Overall* Scale-Adjusted Imprecision Standard Deviations for the Seven Tested Devices and Ratios with OPKO OTI OCT/SLO
Table 7. 
 
Overall* Scale-Adjusted Imprecision Standard Deviations for the Seven Tested Devices and Ratios with OPKO OTI OCT/SLO
Device Overall Operators Ratio with OTI
OPKO OTI OCT/SLO 2.52 1.00
Zeiss Cirrus 3.855 1.53
Topcon 3D-OCT 2000 5.903 2.34
NIDEK OCT RS-3000 2.957 1.17
Zeiss Stratus 6.435 2.55
Optovue RTVue-100 6.386 2.53
Heidelberg Spectralis 2.046 0.81
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×