Signal Normalization Reduces Systematic Measurement Differences Between Spectral-Domain Optical Coherence Tomography Devices

Abstract

Purpose.: To test the effect of a novel signal normalization method for reducing systematic optical coherence tomography (OCT) measurement differences among multiple spectral-domain (SD) OCT devices.

Methods.: A total of 109 eyes from 59 subjects were scanned with two SD-OCT devices (Cirrus and RTVue) at the same visit. Optical coherence tomography image data were normalized to match their signal characteristics between the devices. To compensate signal strength differences, custom high dynamic range (HDR) processing was also applied only to images with substantially lower signal strength. Global mean peripapillary retinal nerve fiber layer (RNFL) thicknesses were then measured automatically from all images using custom segmentation software and were compared to the original device outputs. Structural equation models were used to analyze the absolute RNFL thickness difference between original device outputs and our software outputs after signal normalization.

Results.: The device-measured RNFL thickness showed a statistically significant difference between the two devices (mean absolute difference 10.58 μm, P < 0.05), while there was no significant difference after normalization on eyes with 62.4-μm or thicker RNFL (mean absolute difference 2.95 μm, P < 0.05).

Conclusions.: The signal normalization method successfully reduces the systematic difference in RNFL thickness measurements between two SD-OCT devices. Enabling direct comparison of RNFL thickness obtained from multiple devices would broaden the use of OCT technology in both clinical and research applications.

Introduction

Optical coherence tomography (OCT) has become a staple of ocular imaging in daily clinical care in ophthalmology. ^1–3 There are many commercially available spectral-domain (SD) OCT devices, providing a wide variety of options in terms of cost, scan protocols, image processing, and presentation. This diversity, however, poses a serious clinical challenge when it comes to OCT measurement data comparability. Significant differences exist among the various devices in their reported measurements, such as total retinal thickness and retinal nerve fiber layer (RNFL) thickness. ^4–8 The systematic differences make OCT measurements obtained using different OCT devices not directly comparable, leading to inflexibility with regard to switching various OCT devices and limiting the uses of OCT devices. 

In glaucoma practice, where OCT RNFL thickness is one of the major clinical outcome measures, longitudinal observation is essential as glaucoma is a slowly progressing disease. Yet the measurement differences prevent us from establishing a long-term clinical record of RNFL thickness measurements, in situations such as those in which patients move from one ophthalmologist to another, or even different iterations of the device from the same manufacturer requiring the re-establishment of baseline measurements. Several studies have attempted to overcome this problem by establishing conversion equations of the retinal thickness measurement among different OCT devices ^9,10 ; however, these equations are heavily dependent on the studied population and are specific for each parameter, so that for any given parameter a different equation is needed. 

We hypothesized, first, that OCT signals carry similar information from the same eye but show different signal characteristics from diverse OCT devices because various light sources, wavelengths, and device settings cause systematic measurement differences; and second, that normalizing OCT signal minimizes the systematic OCT measurement differences among different devices. To test these hypotheses, a novel signal normalization method developed previously ¹¹ was tested to evaluate its effect on reducing the systematic differences in peripapillary RNFL thickness measurements obtained from two SD-OCT devices, namely, Cirrus (Zeiss, Dublin, CA) and RTVue (Optovue, Fremont, CA). 

Methods

Subjects

This was an observational cross-sectional study. Subjects included in this study were recruited at the University of Pittsburgh Medical Center Eye Center Glaucoma Clinic (both healthy and glaucomatous eyes). University of Pittsburgh Review Board and ethics committee approval was obtained for the study, and informed consent was obtained from all subjects. This study followed the tenets of the Declaration of Helsinki and was conducted in compliance with the Health Insurance Portability and Accountability Act. 

Inclusion criteria were best-corrected visual acuity of 20/40 or better, refractive error within ±6.0 diopters (D), and no media opacities. Subjects were excluded if they were using medications known to affect the retina, or if they had had any previous intraocular surgeries other than uneventful cataract extraction or glaucoma surgery. 

Instruments and Image Acquisition

The peripapillary region from all eyes was scanned using Cirrus HD-OCT (software version 5.1; Zeiss) and RTVue (software version 6.1; Optovue) at the same visit. Scan patterns, which allow the devices to measure the RNFL thickness using their own segmentation algorithms, were used on both devices. 

Cirrus HD-OCT.

Optic Disc Cube 200 × 200 scan was used to obtain the three-dimensional (3-D) cube data. The scanning protocol collected 200 × 200 sampling points from a 6 × 6-mm² area centered on the optic disc and 1024 samplings within a 2.0-mm axial scan depth. Images with signal strength (SS) less than 7 (the manufacturer recommended cutoff) or apparent eye movement during scanning were considered poor-quality images and discarded. Eye movement was subjectively defined as image artifacts on OCT en face (or OCT fundus) images showing a horizontal frame shift larger than a retinal blood vessel diameter or a major distortion of the optic disc region. 

RTVue OCT.

RNFL 3.45 Circle scan pattern was used to obtain a conventional peripapillary circular scan along a 3.45-mm-diameter circle centered on the optic disc. An RNFL 3.45 Circle scan consisted of 1019 A-scans and 768 samplings along each A-scan for a 2.3-mm axial scan depth. Images with signal strength index (SSI) less than 40 (the manufacturer recommended cutoff) were considered as poor-quality images and were excluded. 

Both Cirrus and RTVue image raw data were exported to a stand-alone computer for signal normalization and further analysis. 

Signal Normalization

Signal normalization was performed based on the previously reported method. ¹¹ In this study, Cirrus OCT data format was used as the normalization data format reference, and therefore RTVue OCT data format was “normalized” (or converted) to the Cirrus equivalent OCT data format. The reasoning behind using Cirrus data as a reference signal has been discussed elsewhere. ¹¹ In short, this process would decrease the artifacts due to scaling discrete numbers and make the outcome data more compatible with readily available image processing tools. The normalization process had three stages: z-scaling and sampling density normalization, amplitude normalization, and SS normalization. Since a modified median filter was applied as part of the preprocessing stage in our segmentation algorithm to remove the speckle noise, ¹² the speckle noise reduction step in the original signal normalization method was removed. On the other hand, SS normalization was added to compensate for the signal quality differences, which have been recognized as an important factor inducing variability to the RNFL thickness measurements. ¹³  

For SS normalization, quality index (QI) was first calculated and used as the image quality index for each OCT image. ¹⁴ As the data range of QI was different between Cirrus and RTVue images, QI values were normalized by calculating the percentiles of the QI distribution on Cirrus and RTVue separately. Then the difference in QI percentiles between the matched Cirrus and RTVue images obtained from the same eye was calculated to assess SS disparity. Finally, on the histogram of the QI percentile difference, the top and bottom 5% of the differences were classified as cases showing substantial QI difference, which became subject to SS normalization. 

In each pair, the image with worse quality was processed with our custom high dynamic range (HDR) processing to compensate for poor SS. ¹⁵ In brief, HDR processing remaps the signal dynamic range at three signal levels (low, mid, and high) separately and then combines them into one so that the OCT retinal signal is enhanced and boosted selectively on the poor signal portion of the images. 

RNFL Thickness Measurements

Original machine-measured global mean peripapillary RNFL thicknesses on the original Cirrus and RTVue data were exported from the commercial devices (comparison I, Table 1). In order to test the hypothesis that applying the same segmentation algorithm can reduce the measurement differences, RNFL thickness was also measured automatically using a universal RNFL segmentation algorithm of our own design (comparison II). ¹² As the word “universal” indicates, our segmentation software is able to open, read, and perform retinal layer segmentation well on various SD-OCT data with the same core segmentation algorithm, unlike algorithms integrated in the commercial devices; these have an optimized approach and parameters targeting the signal characteristics of a specific SD-OCT device and thus may not generate equally good segmentation results when processing OCT data from a different SD-OCT device. Further, RNFL thickness was measured using the universal algorithm with parameters tuned specifically to Cirrus and RTVue images in order to assess the effect of fine-tuning on the universal algorithm (comparison III). Finally, RNFL thickness was measured after signal normalization using the universal segmentation algorithm without any specific tuning (comparison IV). The segmentation performance was subjectively evaluated for any potential erroneous border detection. Image data were excluded if the images demonstrated one or both of the following: (1) apparent inaccurate border detection for more than consecutive 15% or additive 20% of the total image or (2) collapse of borders of the RNFL, meaning that the RNFL thickness was recoded as a string of zeros for at least 10 consecutive points. 

Table 1. 

 

Definitions of Different Methods for Comparison of RNFL Thickness Measurements

Comparison Methods	Segmentation Software	Measurement Description
Comparison I	Commercial software	Original device outputs
Comparison II	Self-designed universal  segmentation software 12	Algorithm parameters were optimized only for Cirrus and this algorithm was applied to both original Cirrus and RTVue data
Comparison III	Self-designed universal  segmentation software 12	Algorithm parameters were optimized for Cirrus and RTVue separately and applied to both original Cirrus and RTVue data
Comparison IV	Self-designed universal  segmentation software 12	Normalized RTVue signals to Cirrus specification and used algorithm parameters optimized only for Cirrus

Statistical Analysis

In order to appropriately handle the comparison between RNFL thickness measurements from Cirrus and RTVue with multiple measurements of the same RNFL thickness from data including both eyes from the same subject, we constructed a comprehensive measurement error model. This measurement error model describes how the true unknown RNFL thickness of each eye is linked to the measurements from each device and processing method, and provides calibration equations to delineate the relationship between Cirrus and RTVue for different comparison methods. 

The simplified basic measurement error model is given by:  

 

where μ_j indicates the unknown true RNFL thickness for the jth eye, x_ij indicates an RNFL thickness observation measured by device i (Cirrus or RTVue) for eye j, α_i and β_i describe the bias (systematic error) introduced by device i, and ε_i denotes a random error whose distribution describes the imprecision for each device. Based on the measurement error model, the calibration equation between two devices for each comparison method can be derived as:  

 

where E denotes the expectation operator (which averages out the random error) and C stands for Cirrus while R stands for RTVue. When the ratio of two device slopes (β_R/β_C) equals one, the calibration line is parallel to the no-bias line, E[x_C] = E[x_R], and the bias is considered to be a constant bias and equal to the horizontal or vertical distance between the calibration line and the no-bias line.  

Structural equation models (SEMs) were used to estimate the parameters in the measurement error model and further derive parameters for the calibration equations. The R Environment and Language for Statistics (version 2.13.1) ¹⁶ with OpenMx (version 1.1.2-1818) ¹⁷ and merror (version 1.0) ¹⁸ were used to describe the SEMs. Full information maximum likelihood (FIML) was used to estimate the measurement error model parameters. 

Furthermore, to assess the effect of our signal normalization method in reducing the systematic difference between Cirrus and RTVue over a wide range of disease severity (measured using the visual field mean deviation [MD] value), linear mixed effect models were constructed to estimate the relationship of the differences of RNFL thickness between two devices to the MD value. 

Results

A total of 109 eyes from 59 subjects were included in this study. Subject demographic and clinical characteristics are presented in Table 2. Disease severity, as measured by the visual field MD, ranged from −9.23 to 2.13 dB, including healthy subjects as well as early and moderate glaucoma subjects. 

Table 2. 

 

Subject Demographics

Subject Demographics
Male:female	16:43
Age, y	61.67 ± 8.0 (59.60, 63.75)
Visual field mean deviation, MD, dB	−0.76 ± 1.90 (−1.13, −0.40)

 Data are expressed as the mean ± standard deviation and the 95% confidence interval within parentheses.

Table 3 shows the global mean peripapillary RNFL thicknesses from Cirrus and RTVue data measured using four different methods and the systematic differences in RNFL thickness measurements between Cirrus and RTVue, along with the slope and intercept values of the corresponding calibration lines. The RNFL thicknesses between Cirrus and RTVue were statistically significantly different before normalization regardless of the choice of segmentation algorithms (device built in or our custom design, comparison I to comparison III). Before signal normalization, there were significant differences in RNFL thickness measurements in comparison I (mean absolute difference 10.58 μm, P < 0.05, Fig. 1A) as well as in comparisons II and III (18.14 and 10.90 μm, both P < 0.05, Figs. 1B, 1C, respectively). After signal normalization, although the RNFL thickness showed a nonconstant difference between devices (comparison IV; Fig. 1D), the difference was reduced substantially. The difference between Cirrus and RTVue was statistically significantly reduced by signal normalization for the eyes with RNFL thicker than 62.4 μm according to Cirrus device measurements (dotted blue vertical line in Fig. 2), representing 95% of the studied population. The mean absolute difference between Cirrus and RTVue for eyes with the Cirrus RNFL thickness larger than 62.4 μm was 2.95 μm, which was calculated by averaging the absolute difference between the two devices in that range. 

Figure 1. 

 

Scatter plot of global mean RNFL thickness measured from Cirrus and RTVue using four different measurement methods. The calibration curve (red line) and no-bias curve (green line) were drawn on each plot. Vertical gray line indicates the average of the RNFL thickness measured from Cirrus data, and the constant differences between two SD-OCT devices were measured as the distance between the red line and green line at this point. Paired eyes from the same subject are connected by gray lines. See Table 1 for definition of the comparisons. (A) Comparison I: original device outputs from Cirrus and RTVue devices. (B) Comparison II: processing the original Cirrus and RTVue data using our universal segmentation software with the same parameters for Cirrus and RTVue, where the parameters were optimized for Cirrus data. (C) Comparison III: original Cirrus and RTVue data processed using our universal segmentation software with the parameters optimized for Cirrus and RTVue separately. (D) Comparison IV: processing the normalized Cirrus and RTVue data using our universal segmentation software with the same parameters, which were optimized for Cirrus only.

Figure 2. 

 

Calibration curve between Cirrus and RTVue after signal normalization (blue line), from original machine outputs (red line) and no-bias curve (green dotted line). Note how the calibration curve after normalization (blue line) is closer to the no-bias curve (green dotted line), showing less bias overall. However, the blue line is not parallel to the green dotted line, indicating that the systematic measurement difference depends on the measured thickness. The dotted blue vertical line is at the threshold of RNFL thickness where differences between devices below this level are statistically significant.

Table 3. 

 

Global Mean Peripapillary RNFL Thickness Measurements and Systematic Measurement Differences Between Cirrus and RTVue, Along With the Slope and Intercept Values of the Corresponding Calibration Lines, Using Four Comparison Methods

	RNFL Thickness From Cirrus, μm	RNFL Thickness From RTVue, μm	Difference Between Cirrus and RTVue, μm	Slope	Intercept
Comparison I	82.65 ± 11.99   (80.37, 84.92)	93.04 ± 12.71   (90.63, 95.46)	10.58 (9.75, 11.41)	1.05 (0.97, 1.14)	6.24 (−1.81, 13.31)
Comparison II	96.77 ± 15.10   (93.90, 99.64)	114.51 ± 17.18   (111.25, 117.77)	18.14 (16.34, 19.95)	1.05 (0.91, 1.20)	13.49 (−3.26, 28.50)
Comparison III	96.77 ± 15.10   (93.90, 99.64)	107.35 ± 16.06   (104.30, 110.40)	10.90 (9.23, 12.57)	0.97 (0.85, 1.11)	13.46 (−1.97, 27.32)
Comparison IV	97.70 ± 15.04   (94.85, 100.56)	99.62 ± 14.77   (96.82, 102.43)	*	0.92 (0.80, 1.06)	9.16 (−6.31, 22.7)

 Data are expressed as the mean ± standard deviation and the 95% confidence interval within parentheses.

*  Nonconstant difference ranging from −0.13 to 4.98 μm.

The relationship between the RNFL thickness differences (RTVue − Cirrus) and visual field MD was also analyzed. Since the effect of the present signal normalization on the eyes with Cirrus-measured RNFL thinner than 62.4 μm was not significant, six eyes with such condition were excluded for this analysis. The RNFL difference showed no significant correlation with the visual field MD either before or after normalization (correlation coefficient 0.24 vs. −0.34 μm/dB, respectively, P > 0.30), indicating that the residual difference between two devices was independent of disease severity before and after normalization. 

Discussion

The systematic measurement differences between two commercial SD-OCT devices, Cirrus and RTVue, were statistically significantly reduced after processing with the developed signal normalization method for most of the cases (95%) in which the Cirrus RNFL thickness was larger than 62.4 μm. This encompasses a wide range of subjects, including early and moderate glaucoma participants along with healthy subjects. For advanced glaucoma subjects whose Cirrus RNFL thickness was less than or equal to 62.4 μm, the reduction of systematic measurement difference was limited with the present method, probably because of an insufficient number of observations of such thin RNFL cases. Further investigation with more observations of severe glaucoma cases is warranted. 

The behavior of the systematic measurement difference can also be observed from Figure 2. The absolute difference in RNFL measurements between Cirrus and RTVue after normalization was always smaller than the difference between the original device outputs. Although the calibration curve after normalization (blue line) in Figure 2 is closer to the no-bias curve (green line), showing less difference overall, it is not parallel to the no-bias curve, indicating that the systematic measurement difference depends on the measured thickness. 

Previous studies showed that RTVue-measured RNFL thicknesses were thicker than the corresponding Cirrus measurements. ^7,8 Our analysis results for the devices' original outputs agreed with the previous findings. Heussen et al. ¹⁹ suggested in a recent study that similar RNFL thickness measurements could be generated by both manually segmenting and correcting the outer retinal boundary to a standardized reference location. Their results support our first hypothesis that during scanning of the same eye, OCT signals from different devices contain the same information although the signal characteristics vary because of different device settings and thus react differently to the same segmentation algorithm. However, correcting only the boundary position still cannot resolve all the systematic difference in RNFL measurements, and it is not practical to manually correct segmentation in regular clinical settings. ¹⁹ The same results were also obtained in our study. Even with use of the same segmentation software, RNFL thickness measured on the RTVue images was still larger than that measured on the Cirrus images, which implies that the factors causing this systematic difference in RNFL thickness are not only due to the use of a different segmentation algorithm but also due to various signal characteristics. 

The systematic RNFL measurement difference between Cirrus and RTVue was 10.58 μm from the original device outputs and 18.14 μm when processing both the original Cirrus and RTVue image data with our universal segmentation software without optimizing the parameters for each SD-OCT device separately. The increased systematic measurement difference indicated that simply processing OCT data from different SD-OCT machines with the same segmentation algorithm cannot reduce the systematic measurement differences between SD-OCT devices and even makes the differences larger. The results also proved our hypothesis that OCT data from different SD-OCT devices have different signal characteristics so that they react differently to the same segmentation algorithm. 

After fine-tuning of the parameters in our universal segmentation software for Cirrus and RTVue separately, the systematic measurement difference became 10.90 μm. With the optimization, the systematic measurement difference decreased to the same level as that obtained from machine outputs, 10.58 μm. This result was expected, since tuning parameters in the same algorithm for each specific SD-OCT device worked similarly to using different algorithms that were optimized for specific SD-OCT devices and would present the best performance for the image captured from the specific device. However, software optimization did not fix the systematic measurement difference between Cirrus and RTVue data. Therefore, an approach other than adjusting the segmentation algorithm is needed to solve this problem. 

With the present signal normalization method, RNFL thickness from the two devices could be reduced to the inherent device measurement variability level and become directly comparable. By unifying the sampling density in the axial direction using z-scaling and sampling density normalization, normalization of the pixel dynamic range using histogram-based amplitude normalization, and compensating SS through HDR process, the proposed normalization method succeeded in transforming OCT signals obtained with one device into virtually similar signals obtained with the other device. Although the systematic differences in RNFL measurement between Cirrus and RTVue could not be reduced to a statistically significant level with RNFL thickness thinner than 62.4 μm, the largest difference between two devices after normalization was 4.98 μm, which was within the inherent device measurement variability. 

It was interesting that the RNFL thickness measurement differences between devices were independent from the disease severity. With thinner RNFL on glaucomatous eyes, one may expect a smaller difference if the effect is proportional. But instead, the results suggest that the effect is more of a fixed bias regardless of the disease status. Likely this bias stems from the characteristic difference in the slope of the intensity profiles at the inner and outer borders of the RNFL. We hypothesize that normalizing such intensity profile characteristic would further reduce the systematic difference in OCT measurements. Further investigation is needed. 

Although we tested only the effect of the signal normalization method on reducing the systematic RNFL thickness measurement differences with Cirrus and RTVue devices, in principle this normalization method can be applied to all SD-OCT devices. Further investigation of this aspect is warranted. 

In conclusion, our signal normalization method successfully reduced the systematic difference in RNFL thickness measurements between Cirrus and RTVue to the level of the reported inherent device measurement variability. This enables the direct comparison of RNFL thicknesses obtained from multiple devices and would broaden the use of OCT technology in both clinical and research applications. 

Presented in part at the annual meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, Florida, May 2012. 

Supported in part by National Institutes of Health contracts NIH R01-EY013178, R01-EY011289, and P30-EY008098 (Bethesda, MD); The Eye and Ear Foundation (Pittsburgh, PA); and Research to Prevent Blindness (New York, NY). 

Disclosure: C.-L. Chen, None; H. Ishikawa, None; Y. Ling, None; G. Wollstein, Allergan (C); R.A. Bilonick, None; J. Xu, None; J.G. Fujimoto, Optovue (I), P; I.A. Sigal, None; L. Kagemann, None; J.S. Schuman, P 

Footnotes

 C-LC and HI contributed equally to the work presented here and should therefore be regarded as equivalent authors.