Reproducibility of Quantitative Optical Coherence Tomography Subanalysis in Neovascular Age-Related Macular Degeneration

Abstract

purpose. To determine the intergrader reproducibility for computer-assisted grading of optical coherence tomography (OCT) images in eyes with neovascular age-related macular degeneration (AMD), by using a standardized grading procedure.

methods. Sixty OCT image sets (of six radial lines each) were independently analyzed by two graders using validated custom software (OCTOR) to draw boundaries manually on OCT B-scans. Spaces delineated by these boundaries included retina, subretinal fluid, subretinal tissue, and pigment epithelial detachments (PEDs). Volume measurements for the nine Early Treatment of Diabetic Retinopathy Study (ETDRS) subfields and the mean foveal center point (FCP) thickness were calculated by the software and compared by using weighted κ statistics and intraclass correlation coefficients (ICCs).

results. Intergrader comparison of the foveal central subfield (FCS) volume, total volume, and mean FCP thickness showed a high level of agreement and strong correlation between measurements for all spaces (κ_weighted = 0.72–0.97; ICC = 0.92–0.99). The best agreement was observed for total volume of the combination of all four graded spaces (κ_weighted = 0.97, mean difference = 0.31 mm³, or 2.51%). The highest ICCs were seen for FCP thickness measurements. The poorest agreement was found for grading of subretinal tissue. Eyes with advanced choroidal neovascularization (CNV) and poor visibility of the retinal pigment epithelium (RPE) band appeared to show the greatest intergrader discrepancies.

conclusions. Analysis of OCT images by trained graders using computer-assisted grading software allows for highly reproducible quantitative measurements, even in eyes with complex diseases such as neovascular AMD. Quantitative subanalysis may be useful in studying the differential morphologic effect of therapies on various anatomic components.

Over the past 15 years, optical coherence tomography (OCT) imaging has had a dramatic impact on the diagnosis and management of vitreoretinal disease. ^{1 2 3 4 5 6 7 8 9} As a noninvasive technique that provides high-resolution cross-sectional images of the fundus, OCT has become a critical tool for assessing the morphologic response of the retina to therapeutic intervention. OCT-derived measurements of retinal thickening have become important secondary outcome parameters in clinical trials for studies of macular edema ^{4 8 9} and choroidal neovascularization (CNV). ^{3 5} In addition to quantitative metrics, OCT is widely used for qualitative assessment to establish the presence of retinal cysts, vitreomacular interface abnormalities, subretinal fluid, and pigment epithelial detachments (PEDs). However, many of these additional morphologic features visible on OCT cannot be quantified by existing StratusOCT software algorithms (Carl Zeiss Meditec, Inc., Dublin, CA). Moreover, the limited retinal quantification that is currently available is frequently inaccurate because of erroneous detection of the inner and outer boundaries of the retina, particularly in patients with neovascular age-related macular degeneration (AMD). ^{10 11 12} As a result, a considerable amount of potentially valuable quantitative information is not extracted by the current commercial algorithms. In an effort to use these additional data, we developed a software tool (OCTOR) that allows the user to draw the boundaries of all structures of interest manually and we demonstrated excellent reproducibility and comparability of this method with the StratusOCT software in normal eyes. ¹³ For this approach to be truly useful, however, reliability must be demonstrated in eyes with significant disease such as those with neovascular AMD. To achieve valid and reproducible results with a manual grading approach, the OCT grading procedure requires standardization. In this report, we describe grading rules and conventions for delineating and quantifying OCT morphologic features in neovascular AMD, and demonstrate the reproducibility of this approach. 

Methods

Data Collection

For this retrospective study, the baseline and month-3 follow-up visits of 30 consecutive patients who were about to begin anti-VEGF therapy for neovascular AMD and had StratusOCT (Carl Zeiss Meditec, Inc.) imaging at the Doheny Eye Institute were selected for analysis (total of 60 visits or cases). Approval for the collection and analysis of image data was obtained from the Institutional Review Board of the University of Southern California. The research adhered to the tenets set forth in the Declaration of Helsinki. 

In 33 cases, high-resolution 6-mm B-scans (512 A-scans per B-scan) obtained by using the Radial Lines protocol were available for analysis. In 27 cases (mainly patients with unstable fixation or poor cooperation) for whom good high-resolution images could not be obtained by the photographer, only Fast Macular Thickness protocol (128 A-scans per B-scan) B-scans were available for use in the analysis. Although Fast Macular Thickness scans have been used in many published OCT studies, the high-resolution scanning protocols are the preferred technique in the Doheny Imaging Unit, due to the greater morphologic detail provided. 

Computer-Assisted Grading Software

The software used for OCT analysis (OCTOR; publicly accessible at www.driamd.org/ Doheny Retina Institute Advanced Macular Diagnostics, Los Angeles, CA) was written by Doheny Image Reading Center (DIRC) software engineers, to facilitate manual grading. ¹³ Raw OCT scan data are exported from the StratusOCT and loaded into the OCTOR software. As previously described, ^{12 13} the OCTOR software allows the grader to use a computer mouse to draw various boundaries manually in each of the six radial line OCT B-scans (Fig. 1) . After the required layers are drawn by the grader, the software calculates the distance in pixels between the relevant manually drawn boundary lines. This measurement is converted into micrometers to yield thicknesses at each location, to allow quantification of the corresponding spaces (Table 1) , each space being defined by an inner and outer boundary line. The thickness at all unsampled locations between the radial lines is then interpolated based on a polar approximation, to produce a thickness map analogous to the StratusOCT output data. Thicknesses are converted into volume data (cubic millimeters) by multiplying the average thickness by the sampled area. The OCTOR software provides the volume and thicknesses for each graded space in each of the nine Early Treatment of Diabetic Retinopathy Study (ETDRS) macular subfields. The means and SDs for the foveal center point (FCP) thickness are also calculated. The OCTOR software does not realign the scans based on manual identification of the foveal center on each scan, because reliable identification of the foveal center may be difficult in eyes with extensive disruption of the retinal morphology. The interpolation algorithm, intragrader reproducibility, and intergrader reliability have been validated for quantification of the retina in normal eyes without disease. ¹³  

Grading Procedures

Sixty OCT image sets were analyzed by two independent certified DIRC graders (SJ, JWT). The procedure for grader certification for OCT analysis at the DIRC is disease specific and consists of (1) core lectures and instruction provided by the reading center principal investigator (SRS), (2) reading materials regarding OCT, ¹⁴ (3) practice cases, and (4) a certification examination reviewed and scored by the reading center principal investigator. Each grader followed the DIRC standard grading protocol for OCT assessment described in this report. 

Boundaries drawn in each of the six OCT B-scans for each case included the internal limiting membrane (ILM), the outer border of the photoreceptor layer, the inner and outer borders of subretinal tissue (if present), the retinal pigment epithelium (RPE), and the estimated normal position of the RPE in eyes with PEDs (Fig. 1) . 

For each of the resultant spaces (retina, subretinal fluid, subretinal tissue, and PED) (Table 1) , the OCTOR software was then used to generate volume measurements for ETDRS subfield 9 (foveal central subfield, FCS) and the total of subfields 1 to 9, as well as the FCP thicknesses. In addition, all four of these spaces were combined, to allow calculations of volumes for a new space termed the inner retinal surface height from the choroid (IHC). 

Grading Protocol

The following sections describe the standard DIRC grading rules and conventions that were applied in this study. In general, in normal areas without pathologic subretinal or sub-RPE spaces, the outer border of the photoreceptors (boundary 2), the RPE boundary (boundary 5), and the estimated normal position of the RPE boundary (boundary 6) were drawn as one line located at the inner surface of the hyperreflective RPE band (Fig. 1) . In good-quality OCT scans of healthy eyes, two hyperreflective lines are apparent at the retina–RPE interface. The upper band represents the inner–outer photoreceptor junction. ^{15 16 17} The lower, often thicker band is believed to be the RPE layer. In high-quality scans, a third line may appear between (Fig. 1)these two bands and may represent the interdigitation between the photoreceptor outer segments and the RPE. ¹⁵  

Boundary 1: Internal Limiting Membrane.

The ILM appears as a hyperreflective line separating the retina and the vitreous and is identified to determine the inner border of the neurosensory retina (Fig. 1) . In eyes with epiretinal membrane (ERM) or vitreous adhesion anterior to the ILM, the ILM boundary is drawn just below the hyperreflective band corresponding to the ERM or cortical vitreous. Areas of detachment of the ERM from the ILM (due to tractional gathering of the retina) are useful for reliable identification of the retinal surface (Fig. 2) . 

Boundary 2: Outer Border of the Photoreceptors.

The outer border of the photoreceptors is drawn to indicate the outer border of the neurosensory retina (Fig. 1) . In healthy eyes, the photoreceptors interdigitate with the RPE, so that only the inner segments and the inner segment–outer segment junction are clearly visible anterior to the RPE band. In areas in which the retina is detached from the RPE, the outer segments of the photoreceptors become more easily visible (Fig. 1Ainset, asterisk). In these areas of neurosensory detachment, the outer segments may appear elongated, presumably due to reduced RPE-mediated phagocytosis of the shed outer segment discs as a result of physical separation from the RPE. Elongated photoreceptor outer segments may be distinguished from subretinal tissue by the identification of subretinal fluid separating the photoreceptors from the underlying subretinal tissue or RPE. 

Boundaries 3 and 4: Inner and Outer Border of Subretinal Tissue.

The subretinal space is a potential space and is not visible on OCT in normal eyes due to the close apposition of the photoreceptor outer segments and the RPE. In cases in which a photoreceptor–RPE separation occurs, a subretinal space may be quantified. This subretinal space may be occupied by fluid (hyporeflective) or other material (hyperreflective; Fig. 2 ). In patients with CNV, this hyperreflective material may include fibrovascular tissue (as in type 2 CNV, a histologic definition used to denote neovascular tissue above the RPE, but below the neurosensory retina), ¹⁸ hemorrhage, lipid, or thick fibrin. Because it is frequently not possible to distinguish among these entities based on their OCT signal alone, the generic label subretinal tissue is assigned to any hyperreflective material in the subretinal space. Any remaining homogenous hyporeflective areas are considered to be subretinal fluid. The inner border of subretinal tissue is drawn at the upper surface of the hyperreflective material in the subretinal space. This border also represents the outer boundary of the subretinal fluid. The outer border of the subretinal tissue is drawn coincidental with the boundary outlining the inner border of the RPE. 

Boundary 5: Inner Border of the RPE.

The RPE boundary is drawn at the superior (internal) border of the hyperreflective band corresponding to the RPE layer (Fig. 1) . In eyes with no subretinal space, the RPE boundary is coincident with the outer border of the photoreceptors (boundary 2). In a diseased eye, the RPE layer may appear less reflective due to atrophy or depigmentation of the RPE. The RPE layer may also become more difficult to discern as a result of hyperreflective signals from surrounding fibrovascular tissues (especially in mixed type 1 [sub-RPE] and type 2 [subretinal] CNV membranes ¹⁸ ; Fig. 2 ). In such cases, the RPE boundary is first drawn in all areas where the RPE band is clearly visible, including along the surface of areas of RPE elevation. When the gap in the drawn boundary is small (<300 μm), the RPE boundary is interpolated across the gap. When the area of uncertainty is larger, the grader applies the convention of drawing the RPE boundary at the estimated normal position of the RPE. In other words, it is assumed no RPE elevation exists when the RPE band is not visible (Fig. 2) . 

Boundary 6: Estimated Normal Position of the RPE.

It is critical to define the estimated normal position of the RPE in cases with PEDs (i.e., the outer border of the PED) to allow for PED quantification. In eyes without PEDs, this boundary is coincident with the inner border of the RPE (boundary 5). In eyes with PEDs, the estimated normal position of the RPE is easy to distinguish if the presumed Bruch's membrane–choriocapillaris complex is visible as a thin medium-reflective line (Fig. 1) . In those cases, the presumed Bruch's membrane–choriocapillaris complex is traced to define the estimated normal RPE position. If this complex cannot be identified due to shadowing from overlying structures, the grader interpolates this boundary between the adjacent portions of the scan where the normal RPE location is clearly identifiable (Fig. 2) . When identifying the presence of PEDs and the normal RPE position, the grader must be cognizant of sharp discontinuities or elevations of the RPE that are sometimes observed on OCT B-scans as a result of eye movements and A-scan misalignment errors by the StratusOCT software. These areas are not considered PEDs and the normal RPE position is drawn along the RPE band in these areas. Also, an area of RPE elevation with a basal diameter <250 μm (as estimated by the grader with reference to the total length of the B-scan of 6 mm) is not considered a PED for this grading protocol. These small areas of elevation probably represent drusen, in many cases. All definite PEDs in the B-scan exceeding this size criteria are drawn and included for quantification. 

Statistical Methods

Results of both graders were compared for qualitative data (identification of the presence of subretinal tissue, subretinal fluid and PEDs) and for quantitative measurements of all graded spaces. The volume (cubic millimeters) of the FCS, the volume of subfields 1 to 9 (total volume), and the mean FCP thickness (micrometers) and standard deviation, were calculated for each case and each space (including the newly defined combined IHC space). The mean, median, and maximum differences between graders were calculated for each parameter. For retina and IHC, the mean percentage differences were also calculated for each case, and the mean value was averaged across all 60 cases: percent difference = absolute difference/(mean OCTOR measurement of both graders)/2. 

Bland-Altman plots were generated with commercially available software (SigmaPlot 2004 for Windows, ver. 9.01; Systat Software Inc., Erkrath, Germany) to illustrate the level of agreement between the graders. Results measured by the graders were also compared by using intraclass correlation coefficients (ICCs), a measure of the correlation between graders that also takes into account the differences in individual ratings, and by using κ statistics, a measure of intergrader concordance on categorical scales that adjusts for chance agreement. For weighted κ (κ_weighted) calculations, the distribution for each variable was divided into 10 intervals of equal size: (maximum value–minimum value)/10 = size of each interval. The weighted κ's are linear weights, calculated using the Cicchetti-Allison method: weight = 1 − distance between column, row values /maximum distance. The column values were numbered from 1 to 10, where each column number represents one tenth of the range of numbers for each measure. The maximum possible distance was 9. The κ statistics were interpreted using the ranges suggested by Landis and Koch ¹⁹ : 0 to 0.2, slight agreement; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and >0.80, almost perfect. Previous reports have suggested that κ statistics and ICCs present two distinct types of information regarding agreement. ²⁰ Both κ's and ICCs were generated to increase confidence in these assessments. 

Finally, ICC statistics were also calculated separately for cases obtained with different scanning protocols (high-resolution Radial Lines versus the Fast Macular Thickness protocol). 

Results

The manual grading process using the OCTOR software required approximately 8 to 10 minutes for each set of six OCT B-scans. The results of qualitative comparison between graders regarding the presence (or absence) of disease are shown in Table 2 . 

Quantitative intergrader comparison of volume and thickness measurements for the various spaces (retina, subretinal fluid, subretinal tissue, PED, and IHC) showed substantial to almost perfect agreement (according to Landis and Koch ¹⁹ guidelines) and strong correlation (measured using ICCs) for all parameters (κ_weighted = 0.72–0.97and ICC = 0.92–0.99; Table 3 ). The best agreement was observed for total volume of the combination of all four graded spaces (IHC) (κ_weighted = 0.97, mean difference = 0.31 mm ³ or 2.51%). The highest ICCs were seen for FCP thickness measurements. The poorest agreement was found for the grading of subretinal tissue. Bland-Altman plots showed mean differences close to zero with narrow confidence intervals, indicating good agreement between graders (Figs. 3 4 5 6 7) . 

Retina

Total retinal volumes ranged from 5.71 to 13.30 mm³, as measured by grader 1, and from 5.97 to 13.41 mm³, as calculated by grader 2. The mean (median) percent differences between graders were 2.42% (2.05%) for total retinal volume, 5.75% (5.41%) for FCS volume, and 6.39% (3.71%) for mean FCP thickness calculations and weighted kappa statistics ranged from 0.84 to 0.88. Grader results correlated highly (ICC = 0.98–0.99). The maximum difference between graders was 0.61 mm³ total volume or a 7.68% mean percentage difference (Fig. 3) . 

Subretinal Tissue

Grader 1 identified subretinal tissue in 47 of the 60 cases. In seven (11.67%) cases, there was a disagreement between graders regarding the presence of subretinal tissue. Of all calculated spaces, subretinal tissue revealed the lowest agreement between graders (κ_weighted = 0.72–0.78; ICC = 0.92–0.96) The total volume of subretinal tissue ranged from 0 to 2.58 mm³ for grader 1 and from 0 to 2.15 mm³ for grader 2. The mean absolute difference was 0.13 mm³ for total volume, 0.01 mm³ for FCS volume, and 17.08 μm for FCP thickness (Fig. 4) . 

Subretinal Fluid

Subretinal fluid was identified in 37 cases by grader 1. In 5 (8.33%) of 60 cases, there was a disagreement regarding the presence of subretinal fluid (κ_weighted = 0.85–0.94; ICC = 0.95–0.99). The mean difference between graders was 0.04 mm³ for total volume, 0.003 mm³ for the FCS volume, and 4.43 μm for FCP thickness calculations (Fig. 5) . 

Pigment Epithelial Detachment

Forty-eight of 60 cases demonstrated a PED according to grader 1. Only 2 (3.33%) cases yielded a disagreement regarding the presence of a PED between graders. The total PED volume ranged from 0 to 18.13 mm³ for grader 1 and from 0 to 12.95 mm³ for grader 2 (κ_weighted = 0.89–0.92; ICC = 0.92–0.97). The mean difference between gradings was 0.25 mm³ for total volume, 0.01 mm³ for FCS volume, and 18.87 μm for FCP thickness. The maximum difference for total volume was 7.64 mm³ in one case with a massive PED that occupied nearly the entire OCT B-scan. The lack of adjacent flat areas of RPE in this case, led to apparent variable assessments of the estimated normal position of the RPE by the graders, resulting in discrepant PED volume values. The second largest difference was 0.86 mm³ (Fig. 6) . 

Inner Retinal Surface Height from Choroid

The best agreement out of all calculated parameters was achieved for the combined space IHC (κ_weighted = 0.92–0.97; ICCs = 0.96–0.99). The mean percentage difference between graders was 2.51% for total volume and 3.71% for FCS volume. The maximum difference for total volume was 7.61 mm³ (27.17%) and occurred in the same case with the large discrepancy in PED volume. The next largest difference between graders was 0.87 mm³ (7.81%; Fig. 7 ). 

Comparison between Scan Types

ICCs for grading reproducibility calculated separately for Radial Line scans and Fast Macular Thickness scans are shown in Table 4 . ICCs for Radial Line scans ranged from 0.90 to 1.00 and for Fast Macular Thickness scans, from 0.85 to 0.99. Overall Radial Line scans showed the stronger correlation between graders. The largest discrepancies between the protocols were seen for total subretinal fluid volume (ICC = 0.89 for Fast Macular Thickness scans, but 1.00 for Radial Line scans) and FCP thickness of subretinal tissue (ICC = 0.85 for Fast Macular Thickness scans, but 0.98 for Radial Line scans). 

Discussion

In this study, excellent agreement was observed between two independent DIRC graders using computer-assisted manual OCT grading software (OCTOR) for quantitative measurements of the retina, subretinal tissue, subretinal fluid, and PEDs in eyes with neovascular AMD. 

OCTOR has been demonstrated to yield high intergrader and intragrader reproducibility for retinal volume and thickness measurements in normal eyes after manual grading of the retinal boundaries. ¹³ Diseased eyes, particularly eyes with complex pathology such as neovascular AMD, can have significant disruption of the normal boundaries due to growth of fibrovascular tissue and associated exudation. These disruptions—for example, in the RPE layer—may make identification of these boundaries challenging. Similarity in reflectivity between adjacent pathologic structures may also create difficulties in accurate boundary identification. For example, subretinal tissue may be difficult to distinguish from turbid subretinal fluid (e.g., containing fibrin or blood cells) or from the photoreceptor layer in areas where the outer segments of the photoreceptors appear more highly reflective or elongated. These concerns were borne out in the present study, as subretinal tissue showed the poorest agreement among the characterized spaces (albeit, κ_weighted > 0.72; ICC = 0.92–0.96). 

Fast Macular Thickness protocol B-scans have been the OCT scan type used in most published clinical studies and clinical trials. An important reason cited for the use of these scans has been the faster acquisition speed (2 seconds), which may attenuate the deleterious effects of eye movements. Unfortunately, the eyes of patients with poor fixation, such as those with neovascular AMD, may still move in this interval. Although one can attempt to correct for movement errors by manually identifying the location of the foveal center on each B-scan, we have observed that manual localization is difficult to perform reliably in patients with marked disruption of the normal retinal morphology, a frequent occurrence in patients with neovascular AMD. A better solution, in our experience, has been to reduce the reliance on OCT parameters that are the most sensitive to even small fixation errors (such as the FCP, which only averages a single point on each B-scan), and consider parameters (such as the total macular volume) that average more points and may be more resistant to these effects. 

Despite our concern that reproducible morphologic grading of CNV lesions using lower-resolution Fast Macular Thickness scans would be difficult, the ICCs were relatively good, with the exception of subretinal tissue (Table 4) . Nonetheless, high-resolution Radial Line scans appeared to be associated with more reproducible measurements than were Fast Macular Thickness scans (Tables 2 4) . Computer-assisted manual grading is dependent on precise visualization and identification of the various retinal and subretinal boundaries, and these boundaries are better visualized on high-resolution scans. Of the 14 instances (Table 2)showing disagreement between graders regarding the presence of subretinal tissue, fluid, or PED, 10 were from Fast Macular Thickness scans and 4 were from Radial Line scans. There was no definite trend for one grader to overcall findings compared with the other, though grader 1 (SJ) more frequently identified subretinal tissue and less frequently identified PEDs. 

For quantitative comparisons, ICCs were also generally higher for Radial Line scans than for Fast Macular Thickness scans (Table 4) . Thus, we would generally recommend the use of high-resolution Radial Line scans for quantitative manual grading of eyes with CNV. It should be noted, however, that the reproducibility of other parameters that could affect the outcome of eyes with CNV, such as the presence of vitreomacular traction or retinal cysts, was not included in this analysis. This report only focused on OCT structures that could be quantified by the OCTOR software and for which a reading center grading protocol was available. Although retinal cysts may be quantified using the manual grading software, we have not yet developed a method for quantifying vitreomacular traction. 

Despite the high intergrader reproducibility observed in this study, it should be noted that fixation instability and the inability to precisely localize the area of scan acquisition may ultimately limit the intervisit reproducibility. Consideration of parameters which sample greater areas of the macula (e.g., total macular volume) may ameliorate this problem to some extent. For example, if there is a pool of subretinal fluid in the central macula, even if the fixation changes slightly at the next visit, this fluid will still likely be included in the total macula assessment. There will still be problems, however, from interpolation of the radial line data from time-domain OCT which will ultimately limit the accuracy of these measurements. New OCT technologies, such as spectral domain OCT, ^{21 22 23} are likely to afford the best solutions to these problems by providing a dense map of the retina with precise registration and localization. These technologies, however, are not yet broadly used and may still be subject to misidentification of retinal boundaries in eyes with CNV. Regardless, we suggest that manual grading with the OCTOR may improve the quality of the data currently being obtained, until a better solution is available. 

Investigators in several studies from Puliafito's laboratory ^{1 2 3 4 5 6 14} have described the appearance of normal eyes and various disease states that may be visible on OCT images. Recent improvements in OCT imaging resolution (spectral domain, ^{21 22 23} ultra–high-resolution, ²⁴ and adaptive optics based ¹⁵ OCT technology) have further improved the accuracy of OCT interpretation. The various retinal signal bands visible on OCT have been correlated with various retinal sublayers in histologic studies in monkey eyes. ^{24 25 26} However, because of a lack of histologic correlation in diseased human eyes, it is not yet possible to identify definitely all hypo- and hyperreflective structures manifest in an OCT B-scan. Therefore, we developed conventions based on the available literature and previous reading center experience in OCT interpretation to establish grading rules to facilitate reproducible measurements in eyes with complex diseases. Applying the DIRC standard grading rules, we were able to achieve acceptable levels of agreement between graders and highly correlated quantitative measurements in eyes with neovascular AMD. 

One of the grading conventions adopted for this analysis was the localization of the inner surface of the RPE (boundary 5) at the surface of the outermost band of the hyperreflective bands visible within the normal retina-RPE interface. This convention was based on previously published reports by Pons and Garcia-Valenzuela ¹⁷ and Costa et al. ¹⁶ In eyes without subretinal or sub-RPE spaces, the outer border of the photoreceptors (boundary 2) and the estimated normal position of the RPE (boundary 6) were drawn at the same position as the RPE boundary, to avoid calculation of pixels between those boundaries as pathologic spaces. 

Another grading rule was to consider all the hyperreflective material between the retina and RPE to be part of the subretinal tissue space. Thus, when assessing the results of this analysis, it is important to note that subretinal tissue volume is not equivalent to type 2 (subretinal) CNV volume, ¹⁸ as it may not be possible to distinguish CNV from other lesion components of similar reflectivity (e.g., heme, pigment, or lipid) based on OCT characteristics alone. 

Another important convention was the estimation of the original RPE location (boundary 6) in PEDs by identification of the Bruch's membrane–choriocapillaris complex or by interpolation between the areas adjacent to the PED in which the RPE remained in its physiological position. This convention, as illustrated by one case in this study, may fail when the transverse extent of the PED is extremely large and extends for the entire span of the B-scan; but, fortunately, this was an uncommon finding. 

A final useful grading convention for quantitative subanalysis was the technique for localization of the presumed RPE in areas of the scan where the RPE is not clearly visible. In patients with more advanced or complex CNV lesions, the RPE may become depigmented, undergo metaplasia, or die. In other cases, the hyperreflective signals from surrounding fibrovascular tissue in the subretinal space or sub-RPE space may obscure the RPE band. In these areas of the scan where the location of the RPE was uncertain, we presumed that there was no RPE elevation and drew the inner RPE boundary (boundary 5) coincident with the estimated normal position of the RPE (boundary 6). Although this convention could underestimate the true PED volume in some cases, it appeared to provide good intergrader agreement. When this rule is applied, graders disagreed in only two cases regarding the presence of PEDs, and measurements for PED volume and thicknesses correlated highly. 

One approach to minimize the impact of variability in the positioning of the borders of the subretinal tissue and the RPE layer is to use measurements of the combined IHC space (which is the total distance between the inner retinal surface and the estimated normal position of the RPE) instead of the individual spaces. Indeed, the IHC measurements in this series demonstrated the best agreement between graders. The clinical relevance of IHC measurements and their correlation with other anatomic and functional outcomes, however, still needs further evaluation. 

Finally, it should be noted that the high level of reproducibility observed in this study was obtained by graders who had undergone a formal certification program in the reading center. The accuracy of grading by new users who undertake a less extensive training program is uncertain, but could be assessed in the future if other researchers use the OCTOR software or a similar grading tool. 

In summary, satisfactory reproducibility of quantitative OCT measurements in eyes with neovascular AMD can be achieved using manual grading software and applying the DIRC standard grading rules. This type of analysis may be useful in studying the morphologic changes in CNV lesions over time and in assessing response to therapy. 

 Supported by National Eye Institute Grants EY03040 and R01 EY014375.

 Submitted for publication February 12, 2007; revised April 11, 2007; accepted June 13, 2007.

 Disclosure: S. Joeres, None; J.W. Tsong, None; P.G. Updike, P; A.T. Collins, None; L. Dustin, None; A.C. Walsh, P; P.W. Romano, None; S.R. Sadda, P

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

OCT B-scan graded using computer-assisted grading software (OCTOR). (A) Raw scan image, (B) manually graded boundaries, (C) calculated spaces. In this high-resolution scan, three hyperreflective bands may be discerned at the retina–retinal pigment epithelium (RPE) interface (1, 2, and 3, inset). The outermost hyperreflective line (3) represents the outer surface of the RPE layer, whereas the innermost hyperreflective line (1) corresponds to the inner–outer photoreceptor junction. The middle hyperreflective line (2) visible only on high-quality OCT scans is presumed to represent the interdigitation between the RPE and photoreceptors. In areas without pathologic separation, the outer border of the photoreceptors, the RPE boundary and the estimated normal position of the RPE are drawn as one combined line at the inner surface of the outermost hyperreflective band (3). In areas in which the photoreceptors become separated from the RPE, the outer segments may appear elongated (*, inset). Underlying the areas of RPE elevation, the presumed Bruch's membrane–choriocapillaris interface (representing the normal location of the RPE) can sometimes be identified as a thin medium-reflective line (A).

Table 1.

 

Manually Graded Boundaries and Corresponding Spaces Calculated by the OCTOR Software

Space	Inner Boundary	Outer Boundary
Retina	Internal limiting membrane	Outer border of photoreceptors
SRF	Outer border of photoreceptors	Inner border of SRT*
SRT	Inner border of SRT	Outer border of SRT
PED	Inner border of the RPE	Estimated normal position of the RPE
IHC	Internal limiting membrane	Estimated normal position of the RPE

 SRF, subretinal fluid; SRT, subretinal tissue; PED, pigment epithelial detachment; RPE, retinal pigment epithelium; IHC, inner retinal surface height from choroid.

*  As SRF typically appears on top of SRT, the OCTOR software was programmed to count all pixels between the photoreceptors and the inner border of SRT as SRF. In OCT scans showing SRF without the presence of SRT, the boundary SRT inner border is drawn coincident with the boundary inner border of the RPE.

Figure 2.

 

(A–C) Grading of an OCT B-scan in an eye with an epiretinal membrane (ERM), subretinal hyporeflective space (presumed fluid), subretinal tissue, and pigment epithelial detachment (PED) (A). Separation of the subretinal tissue and PED in this case is challenging due to disruption of the retinal pigment epithelium (RPE) layer. The grader first draws the RPE boundary peripherally where the boundary is more certain. Additional discontinuous areas where the RPE can be identified are then drawn (B). Small gaps (B, yellow arrows) within the RPE layer are then interpolated, to complete the inner surface of the RPE (C). The Bruch's membrane–choriocapillaris complex is not visible below the PED in this case, so the estimated normal RPE location is drawn by interpolation between the adjacent areas without RPE elevation. (D–F) B-scan image from a case with subretinal tissue and PED with both a large gap (red arrow, E) and a small gap (E, yellow arrow) between areas of clearly identifiable RPE. (F) In the area of the small gap, the grader interpolates across the gap when drawing the RPE boundary (boundary 5). For the large gap, no RPE elevation is presumed to exist in an area without definite RPE, and the grader lowers the RPE boundary (boundary 5) in this area so that it is coincidental with the estimated normal position of the RPE (boundary 6). (A, D) Raw B-scan images; (B, E) B-scan images after partial drawing of boundaries using OCTOR software; (C, F) B-scan images after all relevant boundaries have been drawn.

Table 2.

 

Intergrader Comparison: Qualitative Data for Identification of the Presence of a Pathologic Space

N = 60 (33 RL, 27 FM)	Identified by Grader 1 Only	Identified by Grader 2 Only	Identified by Both Graders	Identified by None of the Graders
Subretinal tissue	5 (1 RL, 4 FM)	2 (0 RL, 2 FM)	42 (22 RL, 20 FM)	11 (10 RL, 1 FM)
Subretinal fluid	2 (1 RL, 1 FM)	3 (1 RL, 2 FM)	35 (17 RL, 18 FM)	20 (14 RL, 6 FM)
Pigment epithelial detachment	0	2 (1 RL, 1 FM)	48 (29 RL, 19 FM)	10 (3 RL, 7 FM)
	Disagreement *			Agreement

 RL, Radial Line scan; FM, Fast Macular Thickness scan.

*  Disagreement was observed in 10 Fast Macular Thickness scans but in only 4 Radial Line scans.

Table 3.

 

Intergrader Comparison: Quantitative Data

	Grader 1 Mean	Grader 2 Mean	Mean Absolute Difference (Median, Maximum)	ICC	Weighted κ
Retina
FCP thickness (μm)	290.67	302.15	16.55 (11, 68)	0.99	0.88
FCS volume (mm3) (subfield 9)	0.24	0.25	0.01 (0.01, 0.05)	0.98	0.84
Total volume (mm3) (subfield 1–9)	7.75	7.87	0.19 (0.17, 0.61)	0.98	0.85
Subretinal tissue
FCP thickness (μm)	65.73	51.55	17.08 (8.50, 106.00)	0.95	0.72
FCS volume (mm3) (subfield 9)	0.05	0.04	0.01 (0.01, 0.07)	0.96	0.78
Total volume (mm3) (subfield 1–9)	0.37	0.27	0.13 (0.07, 0.62)	0.92	0.73
Subretinal fluid
FCP thickness (μm)	23.77	24.13	4.43 (0.00, 29.00)	0.99	0.92
FCS volume (mm3) (subfield 9)	0.01	0.01	0.00 (0.00, 0.03)	0.97	0.85
Total volume (mm3) (subfield 1–9)	0.19	0.18	0.04 (0.01, 0.58)	0.95	0.94
Pigment epithelial detachment
FCP thickness (μm)	83.73	82.20	18.87 (11.00, 265.00)	0.97	0.89
FCS volume (mm3) (subfield 9)	0.07	0.07	0.01 (0.01, 0.21)	0.97	0.92
Total volume (mm3) (subfield 1–9)	1.12	0.96	0.25 (0.05, 7.64)	0.92	0.89
Inner retinal surface height from choroid
FCP thickness (μm)	463.57	460.00	19.23 (11.00, 262.00)	0.99	0.92
FCS volume (mm3) (subfield 9)	0.37	0.36	0.02 (0.01, 0.20)	0.99	0.96
Total volume (mm3) (subfield 1–9)	9.43	9.28	0.31 (0.15, 7.61)	0.96	0.97

 FCP, foveal center point; FCS, foveal central subfield; ICC, intraclass correlation coefficients.

Figure 3.

 

Bland-Altman Plot demonstrating the intergrader reproducibility for retinal volumes.

Figure 4.

 

Bland-Altman Plot demonstrating the intergrader reproducibility for volumes of subretinal tissue.

Figure 5.

 

Bland-Altman plot demonstrating the intergrader reproducibility for volumes of subretinal fluid.

Figure 6.

 

Bland-Altman plot demonstrating the intergrader reproducibility for volumes of PEDs.

Figure 7.

 

Bland-Altman plot demonstrating the intergrader reproducibility for volumes of the combination of all four spaces, including the distance from the inner retinal surface to the estimated normal position of the retinal pigment epithelium (RPE).

Table 4.

 

Comparison between Radial Line and Fast Macular Thickness Scans

	ICC Radial Line Scans	ICC Fast Macular Thickness Scans
Retina
FCP thickness (μm)	0.99	0.97
FCS volume (mm3) (subfield 9)	0.99	0.97
Total volume (mm3) (subfield 1–9)	0.99	0.95
Subretinal tissue
FCP thickness (μm)	0.98	0.85
FCS volume (mm3) (subfield 9)	0.98	0.88
Total volume (mm3) (subfield 1–9)	0.90	0.93
Subretinal fluid
FCP thickness (μm)	0.99	0.98
FCS volume (mm3) (subfield 9)	0.98	0.97
Total volume (mm3) (subfield 1–9)	1.0	0.89
Pigment epithelial detachment
FCP thickness (μm)	0.97	0.92
FCS volume (mm3) (subfield 9)	0.97	0.95
Total volume (mm3) (subfield 1–9)	0.92	0.97
Inner retinal surface height from choroid
FCP thickness (μm)	0.99	0.99
FCS volume (mm3) (subfield 9)	0.99	0.99
Total volume (mm3) (subfield 1–9)	0.96	0.99

 FCP, foveal center point; FCS, foveal central subfield; ICC, intraclass correlation coefficients.