October 2013
Volume 54, Issue 10
Free
Glaucoma  |   October 2013
Reproducibility of SD-OCT–Based Ganglion Cell–Layer Thickness in Glaucoma Using Two Different Segmentation Algorithms
Author Affiliations & Notes
  • Mona K. Garvin
    Center for the Prevention and Treatment of Visual Loss, Iowa City VA Health Care System, Iowa City, Iowa
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa
  • Kyungmoo Lee
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa
  • Trudy L. Burns
    Department Epidemiology, University of Iowa, Iowa City, Iowa
  • Michael D. Abràmoff
    Center for the Prevention and Treatment of Visual Loss, Iowa City VA Health Care System, Iowa City, Iowa
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa
    Department Ophthalmology and Visual Sciences, University of Iowa, Iowa City, Iowa
    Department Biomedical Engineering, University of Iowa, Iowa City, Iowa
  • Milan Sonka
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, Iowa
    Department Ophthalmology and Visual Sciences, University of Iowa, Iowa City, Iowa
  • Young H. Kwon
    Department Ophthalmology and Visual Sciences, University of Iowa, Iowa City, Iowa
  • Correspondence: Mona K. Garvin, 4318 Seamans Center for the Engineering Arts and Sciences, The University of Iowa, Iowa City, IA 52242; mona-garvin@uiowa.edu
Investigative Ophthalmology & Visual Science October 2013, Vol.54, 6998-7004. doi:https://doi.org/10.1167/iovs.13-12131
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Mona K. Garvin, Kyungmoo Lee, Trudy L. Burns, Michael D. Abràmoff, Milan Sonka, Young H. Kwon; Reproducibility of SD-OCT–Based Ganglion Cell–Layer Thickness in Glaucoma Using Two Different Segmentation Algorithms. Invest. Ophthalmol. Vis. Sci. 2013;54(10):6998-7004. https://doi.org/10.1167/iovs.13-12131.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose.: To compare the reproducibility of spectral-domain optical coherence tomography (SD-OCT)–based ganglion cell–layer-plus-inner plexiform–layer (GCL+IPL) thickness measurements for glaucoma patients obtained using both a publicly available and a commercially available algorithm.

Methods.: Macula SD-OCT volumes (200 × 200 × 1024 voxels, 6 × 6 × 2 mm3) were obtained prospectively from both eyes of patients with open-angle glaucoma or with suspected glaucoma on two separate visits within 4 months. The combined GCL+IPL thickness was computed for each SD-OCT volume within an elliptical annulus centered at the fovea, based on two algorithms: (1) a previously published graph-theoretical layer segmentation approach developed at the University of Iowa, and (2) a ganglion cell analysis module of version 6 of Cirrus software. The mean overall thickness of the elliptical annulus was computed as was the thickness within six sectors. For statistical analyses, eyes with an SD-OCT volume with low signal strength (<6), image acquisition errors, or errors in performing the commercial GCL+IPL analysis in at least one of the repeated acquisitions were excluded.

Results.: Using 104 eyes (from 56 patients) with repeated measurements, we found the intraclass correlation coefficient for the overall elliptical annular GCL+IPL thickness to be 0.98 (95% confidence interval [CI]: 0.97–0.99) with the Iowa algorithm and 0.95 (95% CI: 0.93–0.97) with the Cirrus algorithm; the intervisit SDs were 1.55 μm (Iowa) and 2.45 μm (Cirrus); and the coefficients of variation were 2.2% (Iowa) and 3.5% (Cirrus), P < 0.0001.

Conclusions.: SD-OCT–based GCL+IPL thickness measurements in patients with early glaucoma are highly reproducible.

Introduction
While the peripapillary retinal nerve fiber–layer (RNFL) thickness measured from optical coherence tomography (OCT) is a commonly used structural parameter for the evaluation of glaucoma, 13 there is increasing evidence regarding the utility of the ganglion cell–layer-plus-inner plexiform–layer (GCL+IPL) thickness 4,5 or RNFL+GCL+IPL thickness 6,7 within the macula for the detection and management of glaucoma. Although originally limited to evaluation within research groups with access to customized layer segmentation algorithms, GCL+IPL thickness measurements are now commercially available in the analysis software of spectral-domain OCT (SD-OCT) devices. One example is the Ganglion Cell Analysis module of Cirrus version 6 software (Cirrus; Carl Zeiss Meditec, Inc., Dublin, CA). In this case, GCL+IPL thickness measurements are provided within six sectors of an elliptical annulus surrounding the fovea and within the entire elliptical region. In addition, any thickness values that are below the 5% and 1% limits of the normative range are marked accordingly with a yellow and red color, respectively. Using the Cirrus algorithm that was to be included in version 6 of the software, Mwanza et al. 8 reported a high intervisit reproducibility of the GCL+IPL thickness measurements in glaucoma patients. Choi et al. 9 also reported a high intravisit reproducibility of the GCL+IPL thickness measurements in glaucoma patients by using Cirrus software. 
We previously reported a three-dimensional (3D) graph-theoretical approach to the automated segmentation of intraretinal layers (including the GCL+IPL) within SD-OCT volumes 10,11 ; that approach was a core component of our SD-OCT–based glaucoma analyses. 12,13 Recently, we made this software for automatic segmentation of retinal layers, including the GCL+IPL, publicly available as part of a suite of Iowa Reference Algorithms. 14 Now that such a software suite is freely available for research use (in combination with the emerging utility of the GCL+IPL thickness in glaucoma diagnosis), it is important to understand the reproducibility of resulting GCL+IPL thickness measurements. The purpose of the present study was to compare the reproducibility of SD-OCT–based GCL+IPL thickness measurements from a publicly available algorithm (the Iowa Reference Algorithm 14 ) with that from a commercially available algorithm (Cirrus; Carl Zeiss Meditec, Inc.) in patients with glaucoma or those suspected of having glaucoma. 
Methods
Data Acquisition
Patients were recruited consecutively from the outpatient Glaucoma Service at the University of Iowa as previously described. 13 Briefly, both cases of open-angle glaucoma and those suspected of having glaucoma were included in the study; patients with angle-closure or combined-mechanism glaucoma were excluded. Unstable glaucoma patients (i.e., those with elevated intraocular pressure undergoing escalation in laser therapy or surgery) were also excluded. Glaucoma was defined as optic disc cupping consistent with glaucoma (either diffuse or focal thinning of the neuroretinal rim or nerve fiber layer defects) and visual field defects consistent with optic disc cupping, with or without elevated intraocular pressure. Primary as well as secondary open-angle glaucoma (e.g., pigmentary or exfoliative) were included. Suspicion of glaucoma was defined as ocular hypertension (>21 mm Hg) without evidence of glaucomatous optic neuropathy or suspicious optic discs (vertical cup-to-disc ratio >0.7 or asymmetry >0.2 between fellow eyes) with normal visual fields. 13,15  
Patient data were obtained prospectively in both eyes on two separate visits. The visits were less than 4 months apart. At each visit, macula- and optic nerve head (ONH)–centered SD-OCT volumes (Cirrus) were acquired; only the macula-centered volumes were used in the present study. Each SD-OCT volume was 200 × 200 × 1024 voxels, corresponding to physical dimensions of 6 × 6 × 2 mm3. Humphrey visual fields (Carl Zeiss Meditec, Inc.) and simultaneous stereo fundus photographs (3Dx; Nidek, Inc., Freemont, CA) were obtained on the same day as SD-OCT images. SD-OCT volumes with a signal strength of <6 were excluded. Additional scans were excluded if the SD-OCT volume did not contain imaging information within an elliptical annulus centered at the fovea (with a vertical inner and outer radius of 0.5 mm and 2.0 mm, respectively; and a horizontal inner and outer radius of 0.6 mm and 2.4 mm, respectively) due to the fovea being insufficiently centered or if the tissue imaged within the elliptical annulus had been clipped in the depth direction. The study was approved by the Institutional Review Board of the University of Iowa and adhered to the tenets of the Declaration of Helsinki. All participants gave written informed consent. 
Ganglion Cell Structural Parameters Within Macula-Centered SD-OCT Volumes
Eleven surfaces of each volumetric ONH-centered scan were first segmented using a previously published graph-theoretical segmentation approach 10,11 (and as part of the publicly available set of Iowa Reference Algorithms 14 ). It is important to note that this algorithm is completely automated and is without any human input for segmentation (as is also true of the Cirrus algorithm). The following surfaces were retained to enable computation of the fovea center and the GCL+IPL thickness: (1) the internal limiting membrane; (2) the interface between the RNFL and the GCL; (3) the interface between the IPL and the inner nuclear layer (INL); and (4) the posterior surface of the retinal pigment epithelial (RPE) layer (Figs. 1A, 1B). The location of the fovea center was automatically defined as the A-scan location with the minimum distance between the first and fourth surfaces. For each A-scan location, the GCL+IPL thickness was defined as the distance between the second surface (the interface between the RNFL and GCL) and the third surface (the interface between the IPL and INL). 
Figure 1
 
Computation of macular GCL+IPL parameters. (A) Central slice of macula-centered SD-OCT volume of a left eye from a patient. (B) Automated 3D layer segmentation results shown on a central slice of macula-centered volume. The GCL+IPL is between the second (yellow) and third (orange) surfaces. (C) Projection image of macula-centered SD-OCT volume with elliptical annular sectors, used for computing regional GCL+IPL thicknesses. For each sector (ST, S, SN, IN, I, and IT) and the combined elliptical annular region, the mean GCL+IPL thickness is measured. SN, superior nasal; S, superior; ST, superior temporal; IT, inferior temporal; I, inferior; IN, inferior nasal. (D) Color-coded thickness map with overlaid sectoral thickness measurements.
Figure 1
 
Computation of macular GCL+IPL parameters. (A) Central slice of macula-centered SD-OCT volume of a left eye from a patient. (B) Automated 3D layer segmentation results shown on a central slice of macula-centered volume. The GCL+IPL is between the second (yellow) and third (orange) surfaces. (C) Projection image of macula-centered SD-OCT volume with elliptical annular sectors, used for computing regional GCL+IPL thicknesses. For each sector (ST, S, SN, IN, I, and IT) and the combined elliptical annular region, the mean GCL+IPL thickness is measured. SN, superior nasal; S, superior; ST, superior temporal; IT, inferior temporal; I, inferior; IN, inferior nasal. (D) Color-coded thickness map with overlaid sectoral thickness measurements.
Given the GCL+IPL thickness measurements at each A-scan and the location of the fovea, the following parameters were computed (Figs. 1C, 1D): 
  •  
    The mean GCL+IPL thickness within an elliptical annulus (with a vertical inner and outer radius of 0.5 mm and 2.0 mm, respectively; and a horizontal inner and outer radius of 0.6 mm and 2.4 mm, respectively) centered at the fovea;
  •  
    The mean GCL+IPL thickness within six sectors of the elliptical annulus (where SN = superior-nasal; S = superior, ST = superior-temporal; IT = inferior-temporal; I = inferior; and IN = inferior-nasal).
These parameters were also acquired using the Ganglion Cell Analysis module of version 6 of Cirrus software. 
Statistical Analyses
All data analyses were conducted using SAS version 9.3 software (SAS, Cary, NC). For each method (Cirrus and Iowa) and each region (including the overall elliptical region), the following intervisit measures were computed based on fitting general linear models: (1) the intraclass correlation coefficient (ICC), a measure of the reproducibility of the measures from the two visits within eyes, with 95% confidence intervals (CI) estimated using a SAS macro written by Lu and Shara 16 ; (2) the intervisit SD, a measure of the variation between visit 1 and visit 2 measurements within eyes; and (3) the coefficient of variation (CV), a measure of relative variation. CVs for the paired-by-visit Cirrus measures and paired-by-visit Iowa measures for each region (including the overall elliptical region) were compared to test the null hypothesis of no difference in the relative variation for the two methods. 17 Method-specific Bland-Altman plots of the GCL+IPL thickness values were constructed to compare differences between visit 1 and visit 2 values with the means of visit 1 and visit 2 values and include 95% limits of agreement. 18  
Results
Repeated SD-OCT imaging data were obtained from 116 eyes of 58 recruited patients. Three eyes were excluded from all GCL+IPL thickness analyses because the SD-OCT signal strength was <6 on at least one of the two visits; six eyes were excluded because the SD-OCT volume did not contain imaging information within the elliptical annulus centered at the fovea (as defined in Methods); and three additional eyes were excluded due to problems in running the Zeiss software module. This left 104 eyes with which to perform GCL+IPL thickness analyses from 56 patients. The mean age of the 56 participants was 62.9 (±12.7) years old. There were 37 females. The study sample included 52 Caucasians, 3 African-Americans, and 1 Hispanic patient. The mean time between visits was 75.6 (±19.8) days. The mean Humphrey 24-2 mean deviation (MD) was −1.75 (±2.83) dB with a range of −13.38 to 1.79 dB, and the mean pattern SD (PSD) was 3.33 (±2.97) dB with a range of 1.13 to 15.95 dB. The overall mean GCL+IPL thicknesses were 70.0 (±11.4) μm (Iowa) and 69.2 (±11.2) μm (Cirrus). The regional mean GCL+IPL thicknesses using each method are shown in Figure 2
Figure 2
 
Mean GCL+IPL thickness measurements for Cirrus (left) and Iowa (right) across all eyes and visits (n = 2 [104] = 208; shown in right-eye orientation).
Figure 2
 
Mean GCL+IPL thickness measurements for Cirrus (left) and Iowa (right) across all eyes and visits (n = 2 [104] = 208; shown in right-eye orientation).
Based upon the repeated intervisit measurements of these 104 eyes, the resulting ICC, intervisit SD, and CV values for each method and region are summarized in the Table and Figure 3. The overall ICC was 0.98 (95% CI: 0.97–0.99) for the Iowa algorithm and 0.95 (95% CI: 0.93–0.97) for the Cirrus algorithm; the overall intervisit SDs were 1.55 μm (Iowa) and 2.45 μm (Cirrus); and the CVs were 2.2% (Iowa) and 3.5% (Cirrus), P < 0.0001. Sector-based regional ICCs ranged from 0.94 to 0.99 (Iowa) and 0.64 to 0.95 (Cirrus). Regional ICCs were significantly better using the Iowa approach except for the superior region. Sector-based regional intervisit SDs ranged from 1.53 to 2.74 μm (Iowa) and 2.96 to 7.51 μm (Cirrus); and sector-based regional CVs ranged from 2.2% to 4.0% (Iowa) and 4.2% to 10.7% (Cirrus), P < 0.0001 for all comparisons. The mean ± 1.96 SD differences in overall GCL+IPL thicknesses across visits were 0.0 ± 4.3 μm (Iowa) and 0.5 ± 6.8 μm (Cirrus), as shown in the Bland-Altman plots of Figure 4. The mean ± 1.96 SD differences in sectoral GCL+IPL thicknesses across visits were 0.0 ± 7.6 μm (Iowa, SN sector), 0.1 ± 12.5 μm (Cirrus, SN sector), −0.2 ± 7.6 μm (Iowa, S sector), −0.1 ± 8.2 μm (Cirrus, S sector), −0.3 ± 7.4 μm (Iowa, ST sector), 0.4 ± 14.1 μm (Cirrus, ST sector), 0.1 ± 4.4 μm (Iowa, IT sector), 1.0 ± 12.6 μm (Cirrus, IT sector), 0.5 ± 4.2 μm (Iowa, I sector), 0.9 ± 12.4 μm (Cirrus, I sector), 0.2 ± 4.8 μm (Iowa, IN sector), and 0.8 ± 20.8 μm (Cirrus, IN sector), as shown in the Bland-Altman plots of Figure 5
Figure 3
 
Color-coded plots of the ICC (top row), intervisit SD (middle row), and CV (bottom row) for the Cirrus (left) and Iowa (right) algorithms. Plots are shown in a right-eye orientation. For each parameter, colors are assigned so that blue corresponds to the better reproducibility values and red corresponds to worse reproducibility values. The Iowa ICC values indicated with an asterisk were significantly higher than the corresponding Cirrus ICC values, and the Iowa CVs were significantly lower.
Figure 3
 
Color-coded plots of the ICC (top row), intervisit SD (middle row), and CV (bottom row) for the Cirrus (left) and Iowa (right) algorithms. Plots are shown in a right-eye orientation. For each parameter, colors are assigned so that blue corresponds to the better reproducibility values and red corresponds to worse reproducibility values. The Iowa ICC values indicated with an asterisk were significantly higher than the corresponding Cirrus ICC values, and the Iowa CVs were significantly lower.
Figure 4
 
Bland-Altman plots of overall GCL+IPL thickness, using Cirrus and Iowa algorithms. Each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 4
 
Bland-Altman plots of overall GCL+IPL thickness, using Cirrus and Iowa algorithms. Each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 5
 
Bland-Altman plots of sectoral (superior-nasal, superior, superior-temporal, inferior-temporal, inferior, and inferior-nasal) GCL+IPL thicknesses using Cirrus and Iowa algorithms. As described in the legend to Figure 4, each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 5
 
Bland-Altman plots of sectoral (superior-nasal, superior, superior-temporal, inferior-temporal, inferior, and inferior-nasal) GCL+IPL thicknesses using Cirrus and Iowa algorithms. As described in the legend to Figure 4, each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Table
 
Regional ICC, Intervisit SD, and CV of GCL+IPL Thickness Measurements*
Table
 
Regional ICC, Intervisit SD, and CV of GCL+IPL Thickness Measurements*
Region Method ICC ICC 95% CI Intervisit SD CV, %
SN Cirrus 0.86 0.80, 0.90 4.50 6.2
Iowa 0.94 0.91, 0.96 2.73 3.7
S Cirrus 0.95 0.93, 0.97 2.96 4.2
Iowa 0.94 0.91, 0.96 2.74 3.7
ST Cirrus 0.84 0.77, 0.89 5.08 7.3
Iowa 0.95 0.93, 0.97 2.68 4.0
IT Cirrus 0.88 0.83, 0.92 4.57 6.8
Iowa 0.99 0.99, 0.99 1.58 2.5
I Cirrus 0.87 0.82, 0.91 4.52 6.7
Iowa 0.99 0.99, 0.99 1.53 2.2
IN Cirrus 0.64 0.51, 0.74 7.51 10.7
Iowa 0.98 0.97, 0.99 1.74 2.4
Overall Cirrus 0.95 0.93, 0.97 2.45 3.5
Iowa 0.98 0.97, 0.99 1.55 2.2
Discussion
Our reported ICC (0.98, Iowa; 0.95, Cirrus), intervisit SD (1.6 μm, Iowa; 2.5 μm, Cirrus) and CV (2.2%, Iowa; 3.5%, Cirrus) results indicate that the GCL+IPL thickness has a high reproducibility in this population. The Iowa Reference Algorithm reproducibility was comparable to that reported by Mwanza et al., 8 in which an overall GCL+IPL thickness ICC of 0.98 and a CV of 1.8% were reported. However, our Cirrus algorithm reproducibility results appear to be worse than those reported by Mwanza et al. 8 Since Mwanza et al. 8 used a “pre-release version” of the algorithm (not the actual version 6.0) to analyze the images, we have no way of directly comparing our current results to those of that study. Our Cirrus intervisit reproducibility results also appear to be worse than the intravisit reproducibility results of Choi et al., 9 in which an overall GCL+IPL thickness ICC of 0.98 and a CV of 1.0% were reported. This may be due in part to the fact that we were computing intervisit reproducibility results rather than intravisit reproducibility results. In addition, differences in exclusion criteria based on segmentation quality (discussed below) may have also contributed to our lower Cirrus reproducibility results than those previously reported by Mwanza et al. 8 and Choi et al. 9  
As determined from the Bland-Altman plots of the overall GCL+IPL thicknesses (Fig. 4), three eyes from the Iowa algorithm and seven eyes from the Cirrus algorithm had a mean overall intervisit GCL+IPL thickness difference of ≥5 μm. In examining the three Iowa eyes, the thickness differences of two of the eyes (with mean differences of 17.4 μm and 5.3 μm) were due to an incorrect automated determination of the fovea center from one of the visits rather than layer segmentation inaccuracies. In fact, manually correcting the placement of the automatically determined fovea center (with a single click) results in mean thickness differences of only 0.3 μm (from 17.4 μm) and 0.5 μm (from 5.3 μm) across the two visits for these two eyes. The remaining eye, with a mean difference greater than 5 μm, had GCL+IPL boundaries that would also be difficult to delineate manually. In examining the analysis reports of the 7 Cirrus eyes with a mean overall intervisit GCL+IPL thickness difference of ≥5 μm, some type of layer segmentation inaccuracy seemed to be at least part of the cause (due to a boundary appearing incorrectly in the central cross-sectional view and/or the thickness map having an unusual pattern of thickness). Because of its proprietary nature, we do know the exact cause of the increased number of layer segmentation inaccuracies with the Cirrus algorithm; however, in our experience, the true 3D segmentation algorithms, such as the Iowa Reference Algorithm, which simultaneously finds multiple surfaces in 3D in one optimization step rather than using a 2D slice-by-slice approach, are often able to better overcome local ambiguous image information (such as that due to local signal dropout) that can cause errors with a slice-by-slice approach. 19 Two of these seven Cirrus eyes (with mean differences of 25.0 μm and 10.0 μm) had differences that were likely also caused by an incorrect placement of the fovea (data not shown). Because such data are included in the analysis, our reported glaucoma intervisit reproducibility estimates are likely more conservative than estimates reported in studies where scans with layer segmentation inaccuracies (which are hard to consistently define a priori as such inaccuracies may be very subtle and/or not be visible in the central slice available on the Cirrus analysis report) are excluded. 
Sectorally (see Bland-Altman plots of Fig. 5), 10 eyes from the Iowa algorithm and 22 eyes from the Cirrus algorithm had at least one sector with a GCL+IPL intervisit thickness difference of ≥5 μm (including the three Iowa eyes and seven Cirrus eyes discussed above, where the overall GCL+IPL intervisit thickness difference was also ≥5 μm). Of the seven Iowa eyes and 15 Cirrus eyes with a sectoral intervisit thickness difference of ≥5 μm but an overall intervisit thickness difference <5 μm, all sectoral intervisit thickness differences were ≤9 μm in six of seven of the Iowa eyes and nine of 15 of the Cirrus eyes. Of these cases with sectoral thickness differences ≤9 μm, no obvious segmentation inaccuracies were noted (based on visual examination of the boundaries on central slices of the Cirrus cases and all slices of the Iowa cases), except for one Cirrus case with a very localized segmentation inaccuracy. In addition to the obvious fact that fewer A-scans are used to compute the sectoral measurements, the worse reproducibility in the sectors (versus that in the overall elliptical region) in these cases may also have been caused by imaging differences (e.g., subtle motion artifacts and/or rotational differences of the eye) that would affect the sectoral thicknesses more than the overall thickness (e.g., in cases where the patient had thinning near a sectoral boundary). In the remaining cases (six Cirrus eyes and one Iowa eye) with a sectoral intervisit thickness difference >9 μm (yet an overall invervisit thickness difference of <5 μm), a potential segmentation inaccuracy was noted in five of seven cases. Clinically, because of the worse reproducibility in the sectors, it is thus important to recognize that larger differences over time are needed to indicate true change in a sector when compared to the overall elliptical region. 
The particular GCL+IPL sectors used in this work were chosen for consistency with those provided by the Zeiss commercial software. However, other options exist for reporting GCL thickness. For example, a number of studies have used the ganglion cell complex (GCC) within the macula, which is frequently defined as the RNFL+GCL+IPL. Examples of studies reporting reproducibility of GCC thicknesses in patients with glaucoma include the work of Tan et al. 6 who reported an ICC of 0.99 and a CV of 1.25% of the GCC (segmented using a customized algorithm) from RTVue SD-OCT scans acquired over a 7- × 6-mm region (Optovue, Inc., Fremont, CA). Zhang et al. 20 reported an ICC of 0.91 and CV of 0.93 for the average macular inner retinal layer using RTVue of normal rhesus monkeys (which had a much thicker inner retinal layer). In addition to choice of layers, it is also important to note the different definitions of region (e.g., sectors versus the entire macular region). It is also possible that other region definitions, such as bundle-based regions 12,13 or patient-adjusted regions, may prove to be more appropriate for assessment of glaucoma. 
The GCL+IPL thickness from macula-center SD-OCT scans is complementary to other available structural parameters, such as the peripapillary RNFL thickness, whose reproducibility is also important to consider. For example, Mwanza et al. 21 reported an ICC of 0.97 and a CV of 2.7%, and Lueng et al. 22 reported an ICC of 0.96 and a CV of 1.8%, using the Cirrus RNFL thickness algorithm. We have also recently reported a peripapillary RNFL thickness ICC of 0.96 and a CV of 2.8% by using the Cirrus algorithm, an ICC of 0.96 and a CV of 3.0% by using the Iowa Reference Algorithm, and an ICC of 0.99 and CV of 1.6% by using a newly reported angle-corrected Iowa Reference Algorithm. 23  
In the current study, the GCL+IPL thickness reproducibility results were computed using only the imaging data from the device of a single manufacturer (Cirrus), and correspondingly, we cannot directly make any claims regarding the reproducibility of the GCL+IPL thickness of the Iowa Reference Algorithm from the SD-OCT volumes obtained from other devices. While the use of proprietary algorithms of SD-OCT manufacturers (such as the Zeiss software studied here) is limited to a particular device, custom-designed third-party software tools, 10,11,24,25 including the Iowa Reference Algorithm, can be used with imaging data from multiple devices. Thus, a potential advantage of using a third-party algorithm is the ability to compare results across devices. Formal evaluation of the reproducibility of structural parameters based on imaging data from multiple devices will be the subject of our future work. 
The reproducibility was computed based on SD-OCT data (with signal strength ≥6) obtained from (primarily early) open-angle glaucoma or patients suspected of having glaucoma. It is possible that the reproducibility of each algorithm may be worse when tested with data from SD-OCT volumes from patients outside of our study population or those with lower signal strengths. Thus, extra care must be taken when applying any GCL+IPL algorithm (whether from a commercial manufacturer or third-party algorithm) to evaluate data from patients beyond the study population (including patients with more advanced glaucoma) or to data with signal strength <6. Visually examining the resulting boundaries for potential accuracy problems is especially important in these cases. 
In summary, we demonstrate excellent reproducibility of the macular GCL+IPL thickness in glaucoma patients, using the Iowa Reference Algorithm. It is comparable to commercially available software (Cirrus version 6), with most sectors demonstrating a significantly better reproducibility. One advantage of the Iowa Reference Algorithm compared to proprietary software is that it is hardware-independent and can be used across different OCT devices. Future work will focus on establishing the reproducibility across multiple devices and diseases. 
Acknowledgments
The authors thank Wallace L.M. Alward, MD, and John H. Fingert, MD, PhD, for permission to recruit study patients from their clinics; Marilyn E. Long for assistance in acquiring and organizing the image data; and Andreas Wahle, PhD, for extension of XNAT for the storage of ophthalmic data. 
A portion of this work was presented at the annual meeting of the Association for Research in Vision and Ophthalmology, Seattle, Washington, May 2013. 
Supported by National Institutes of Health Grants R01 EY018853 (MDA, YHK, MS), R01 EY019112 (MDA, MS), R01 EB004640 (MS), and R01 EY023279 (MKG); Department of Veterans Affairs Rehabilitation Research and Development Division (Iowa City Center for the Prevention and Treatment of Visual Loss and Career Development Award 1IK2RX000728 [MKG]); Research to Prevent Blindness, New York, NY; American Glaucoma Society Mid-Career Physician Scientist Award (YHK); and the Marlene S. and Leonard A. Hadley Glaucoma Research Fund. 
Disclosure: M.K. Garvin, P; K. Lee, None; T.L. Burns, None; M.D. Abràmoff, P; M. Sonka, P; Y.H. Kwon, None 
References
Leung CKS Chan W Ed F Comparison of macular and peripapillary measurements for the detection of glaucoma: an optical coherence tomography study. Ophthalmology . 2005; 112: 391–400. [CrossRef] [PubMed]
Leite MT Rao HL Zangwill LM Weinreb RN Medeiros FA. Comparison of the diagnostic accuracies of the Spectralis, Cirrus, and RTVue optical coherence tomography devices in glaucoma. Ophthalmology . 2011; 118 : 1334–1339. [PubMed]
Hood DC Anderson SC Wall M Kardon RH. Structure versus function in glaucoma: an application of a linear model. Invest Ophthalmol Vis Sci . 2007; 48 : 3662–3668. [CrossRef] [PubMed]
Mwanza J-C Durbin MK Budenz DL Glaucoma diagnostic accuracy of ganglion cell-inner plexiform layer thickness: comparison with nerve fiber layer and optic nerve head. Ophthalmology . 2012; 119: 1151–8. [CrossRef] [PubMed]
Hood DC Raza AS De Moraes CGV Liebmann JM Ritch R. Glaucomatous damage of the macula. Prog Retin Eye Res . 2013; 32: 1–21. [CrossRef] [PubMed]
Tan O Chopra V Lu AT-H Detection of macular ganglion cell loss in glaucoma by Fourier-domain optical coherence tomography. Ophthalmology . 2009; 116: 2305–2314. [CrossRef] [PubMed]
Kotera Y Hangai M Hirose F Mori S Yoshimura N. Three-dimensional imaging of macular inner structures in glaucoma by using spectral-domain optical coherence tomography. Invest Ophthalmol Vis Sci . 2011; 52: 1412–1421. [CrossRef] [PubMed]
Mwanza J-C Oakley JD Budenz DL Chang RT Knight OJ Feuer WJ. Macular ganglion cell-inner plexiform layer: automated detection and thickness reproducibility with spectral domain-optical coherence tomography in glaucoma. Invest Ophthalmol Vis Sci . 2011; 52: 8323–8329. [CrossRef] [PubMed]
Choi YJ Jeoung JW Park KH Kim DM. Glaucoma detection ability of ganglion cell-inner plexiform layer thickness by spectral-domain optical coherence tomography in high myopia. Invest Ophthalmol Vis Sci . 2013; 54: 2296–2304. [CrossRef] [PubMed]
Garvin MK Abràmoff MD Wu X Russell SR Burns TL Sonka M. Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE Trans Med Imaging . 2009; 28: 1436–1447. [CrossRef] [PubMed]
Quellec G Lee K Dolejsi M Garvin MK Abràmoff MD Sonka M. Three-dimensional analysis of retinal layer texture: identification of fluid-filled regions in SD-OCT of the macula. IEEE Trans Med Imaging . 2010; 29: 1321–1330. [CrossRef] [PubMed]
Lee K Kwon YH Garvin MK Niemeijer M Sonka M Abràmoff MD. Distribution of damage to the entire retinal ganglion cell pathway quantified using spectral-domain optical coherence tomography analysis in patients with glaucoma. Arch Ophthalmol . 2012; 130: 1118–1126. [CrossRef] [PubMed]
Garvin MK Abràmoff MD Lee K Niemeijer M Sonka M Kwon YH. 2-D pattern of nerve fiber bundles in glaucoma emerging from spectral-domain optical coherence tomography. Invest Ophthalmol Vis Sci . 2012; 53: 483–489. [CrossRef] [PubMed]
The Iowa Institute for Biomedical Imaging. Iowa Institute for Biomedical Imaging downloads. 2013. Available at: http://www.biomed-imaging.uiowa.edu/downloads/. Accessed January 28, 2013.
Kwon YH Adix M Zimmerman MB Variance owing to observer, repeat imaging, and fundus camera type on cup-to-disc ratio estimates by stereo planimetry. J Glaucoma . 2009; 18: 305–310. [CrossRef] [PubMed]
Lu L Nawar S. Reliability analysis: calculate and compare intra-class correlation coefficients (ICC) in SAS. In: Proceedings of NESUG (NorthEast SAS Users Group). 2007. Available at: http://www.nesug.org/Proceedings/nesug07/sa/sa13.pdf. Accessed March 1, 2013.
Zar JH. Biostatistical Analysis . Englewood Cliffs, NJ: Prentice-Hall, Inc.; 1974: 103–105.
Bland JM Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res . 1999; 8: 135–160. [CrossRef] [PubMed]
Abràmoff MD Garvin MK Sonka M. Retinal imaging and image analysis. IEEE Rev Biomed Eng . 2010; 3: 169–208. [CrossRef] [PubMed]
Zhang Z Yang D Sang J Reproducibility of macular, retinal nerve fiber layer, and ONH measurements by OCT in rhesus monkeys: the Beijing Intracranial and Intraocular Pressure (iCOP) Study. Invest Ophthalmol Vis Sci . 2012; 53: 4505–4509. [CrossRef] [PubMed]
Mwanza J-C Chang RT Budenz DL Reproducibility of peripapillary retinal nerve fiber layer thickness and optic nerve head parameters measured with Cirrus HD-OCT in glaucomatous eyes. Invest Ophthalmol Vis Sci . 2010; 51: 5724–5730. [CrossRef] [PubMed]
Leung CK-S Cheung CY-L Weinreb RN Retinal nerve fiber layer imaging with spectral-domain optical coherence tomography: a variability and diagnostic performance study. Ophthalmology . 2009; 116: 1257–63. [CrossRef] [PubMed]
Lee K Sonka M Kwon YH Garvin MK Abràmoff MD. Adjustment of the retinal angle in SD-OCT of glaucomatous eyes provides better intervisit reproducibility of peripapillary RNFL thickness. Invest Ophthalmol Vis Sci . 2013; 54: 4808–4812. [CrossRef] [PubMed]
Hood DC Cho J Raza AS Dale EA Wang M. Reliability of a computer-aided manual procedure for segmenting optical coherence tomography scans. Optometry Vis Sci . 2011; 88: 113–123. [CrossRef]
Chiu SJ Li XT Nicholas P Toth CA Izatt JA Farsiu S. Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation. Optics Express . 2010; 18 : 19413–19428. [CrossRef] [PubMed]
Figure 1
 
Computation of macular GCL+IPL parameters. (A) Central slice of macula-centered SD-OCT volume of a left eye from a patient. (B) Automated 3D layer segmentation results shown on a central slice of macula-centered volume. The GCL+IPL is between the second (yellow) and third (orange) surfaces. (C) Projection image of macula-centered SD-OCT volume with elliptical annular sectors, used for computing regional GCL+IPL thicknesses. For each sector (ST, S, SN, IN, I, and IT) and the combined elliptical annular region, the mean GCL+IPL thickness is measured. SN, superior nasal; S, superior; ST, superior temporal; IT, inferior temporal; I, inferior; IN, inferior nasal. (D) Color-coded thickness map with overlaid sectoral thickness measurements.
Figure 1
 
Computation of macular GCL+IPL parameters. (A) Central slice of macula-centered SD-OCT volume of a left eye from a patient. (B) Automated 3D layer segmentation results shown on a central slice of macula-centered volume. The GCL+IPL is between the second (yellow) and third (orange) surfaces. (C) Projection image of macula-centered SD-OCT volume with elliptical annular sectors, used for computing regional GCL+IPL thicknesses. For each sector (ST, S, SN, IN, I, and IT) and the combined elliptical annular region, the mean GCL+IPL thickness is measured. SN, superior nasal; S, superior; ST, superior temporal; IT, inferior temporal; I, inferior; IN, inferior nasal. (D) Color-coded thickness map with overlaid sectoral thickness measurements.
Figure 2
 
Mean GCL+IPL thickness measurements for Cirrus (left) and Iowa (right) across all eyes and visits (n = 2 [104] = 208; shown in right-eye orientation).
Figure 2
 
Mean GCL+IPL thickness measurements for Cirrus (left) and Iowa (right) across all eyes and visits (n = 2 [104] = 208; shown in right-eye orientation).
Figure 3
 
Color-coded plots of the ICC (top row), intervisit SD (middle row), and CV (bottom row) for the Cirrus (left) and Iowa (right) algorithms. Plots are shown in a right-eye orientation. For each parameter, colors are assigned so that blue corresponds to the better reproducibility values and red corresponds to worse reproducibility values. The Iowa ICC values indicated with an asterisk were significantly higher than the corresponding Cirrus ICC values, and the Iowa CVs were significantly lower.
Figure 3
 
Color-coded plots of the ICC (top row), intervisit SD (middle row), and CV (bottom row) for the Cirrus (left) and Iowa (right) algorithms. Plots are shown in a right-eye orientation. For each parameter, colors are assigned so that blue corresponds to the better reproducibility values and red corresponds to worse reproducibility values. The Iowa ICC values indicated with an asterisk were significantly higher than the corresponding Cirrus ICC values, and the Iowa CVs were significantly lower.
Figure 4
 
Bland-Altman plots of overall GCL+IPL thickness, using Cirrus and Iowa algorithms. Each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 4
 
Bland-Altman plots of overall GCL+IPL thickness, using Cirrus and Iowa algorithms. Each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 5
 
Bland-Altman plots of sectoral (superior-nasal, superior, superior-temporal, inferior-temporal, inferior, and inferior-nasal) GCL+IPL thicknesses using Cirrus and Iowa algorithms. As described in the legend to Figure 4, each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Figure 5
 
Bland-Altman plots of sectoral (superior-nasal, superior, superior-temporal, inferior-temporal, inferior, and inferior-nasal) GCL+IPL thicknesses using Cirrus and Iowa algorithms. As described in the legend to Figure 4, each point (corresponding to one eye) is plotted as the difference in the overall GCL+IPL thickness value between the two visits versus the average of the GCL+IPL thickness values across the two visits using the indicated method. V1, visit 1 value; V2, visit 2 value. Dashed lines are drawn at the overall mean of the visit differences and at ±1.96 of the SD of the visit differences.
Table
 
Regional ICC, Intervisit SD, and CV of GCL+IPL Thickness Measurements*
Table
 
Regional ICC, Intervisit SD, and CV of GCL+IPL Thickness Measurements*
Region Method ICC ICC 95% CI Intervisit SD CV, %
SN Cirrus 0.86 0.80, 0.90 4.50 6.2
Iowa 0.94 0.91, 0.96 2.73 3.7
S Cirrus 0.95 0.93, 0.97 2.96 4.2
Iowa 0.94 0.91, 0.96 2.74 3.7
ST Cirrus 0.84 0.77, 0.89 5.08 7.3
Iowa 0.95 0.93, 0.97 2.68 4.0
IT Cirrus 0.88 0.83, 0.92 4.57 6.8
Iowa 0.99 0.99, 0.99 1.58 2.5
I Cirrus 0.87 0.82, 0.91 4.52 6.7
Iowa 0.99 0.99, 0.99 1.53 2.2
IN Cirrus 0.64 0.51, 0.74 7.51 10.7
Iowa 0.98 0.97, 0.99 1.74 2.4
Overall Cirrus 0.95 0.93, 0.97 2.45 3.5
Iowa 0.98 0.97, 0.99 1.55 2.2
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×