Repeatability of Cone Spacing Measures in Eyes With Inherited Retinal Degenerations

Abstract

Purpose: To determine short-term variability of adaptive optics scanning laser ophthalmoscopy (AOSLO)–derived cone spacing measures in eyes with inherited retinal degenerations (IRD) and in normal eyes.

Methods: Twenty IRD patients and 10 visually normal subjects underwent AOSLO imaging at two visits separated by no more than 1 month (NCT00254605). Cone spacing was measured in multiple macular regions in each image by three independent graders. Variability of cone spacing measures between visits, between graders, and between eyes was determined and correlated with standard clinical measures.

Results: Cone spacing was measured in 2905 regions. Interobserver agreement was high both in normal eyes and eyes with IRD (mean intraclass correlation coefficient [ICC] = 0.838 for normal and 0.892 for eyes with IRD). Cone spacing measures were closely correlated between visits (ICC > 0.869 for both study groups). Mean relative intervisit spacing difference (absolute difference in measures divided by the mean at each region) was 4.0% for normal eyes and 4.9% for eyes with IRD. Cone spacing measures from fellow eyes of the same subject showed strong agreement for all subjects (ICC > 0.85 for both study groups).

Conclusions: Adaptive optics scanning laser ophthalmoscopy–derived macular cone spacing measures were correlated between observers, visits, and fellow eyes of the same subject in normal eyes and in eyes with IRD. This information may help establish the role of cone spacing measures derived from images of the cone mosaic obtained with AOSLO as a sensitive biomarker for longitudinal tracking of photoreceptor loss during disease progression and in response to treatment. (ClinicalTrials.gov number, NCT00254605.)

Inherited retinal degenerations (IRD) constitute a genetically heterogeneous group of diseases characterized by slowly progressive loss of cone and rod photoreceptors due to genetic mutations in these cells or in the closely interacting and supportive RPE cells, with resulting loss of vision.¹ Several modalities are currently in clinical use for monitoring eyes with retinal degenerations. Structural measures include fundus photography, fundus autofluorescence, and spectral-domain optical coherence tomography (SD-OCT), and functional measures include visual field sensitivity, full-field ERG and multifocal ERG responses. Natural history studies of retinal degenerations have shown that significant change in visual function may be observed only after several years; because measures of visual function demonstrate high intervisit variability, thresholds for significant change range from 20% to 50% for standard clinical measures including full-field ERG cone flicker amplitudes, kinetic and static perimetry, suggesting that significant photoreceptor loss is necessary before visual function change can be measured reliably using standard methods.^2–12 Spectral-domain OCT–based measures of the annual change in inner-segment/outer-segment (IS/OS) junction or ellipsoid zone (EZ)¹³ band width in patients with X-linked RP predicted 13% loss of functioning retina per year, consistent with measures of disease progression reported for ERG and visual field, thus establishing SD-OCT as a valid measure of structural disease progression in eyes with RP.¹⁴ However, EZ width provides only a binary measure of photoreceptor cell survival, indicating the retinal region at which cells are no longer visible with confocal imaging, although photoreceptors with altered waveguiding properties may persist,¹⁵ and does not assess the integrity of surviving photoreceptors. An imaging modality capable of imaging photoreceptors with greater sensitivity than current standard measures may facilitate accurate assessment of surviving photoreceptors, disease progression, and response to treatment in eyes with IRD. 

Adaptive optics scanning laser ophthalmoscopy (AOSLO) can noninvasively and reliably generate images of individual cone photoreceptors in vivo.^16–19 Adaptive optics scanning laser ophthalmoscopy has been used to characterize retinal photoreceptor structure in healthy eyes and in eyes with IRD,^17,20–25 and AOSLO-derived quantitative measures of cone spacing obtained from IRD patients in a clinical trial provided an objective measure of structural disease progression and response to experimental treatment.²⁶ Changes in cone spacing may provide a more sensitive measure of photoreceptor loss than standard clinical measures, such as visual acuity and foveal sensitivity, because cone spacing measures more than 50% below the normal mean have been observed in patients with IRD who retain normal visual acuity and foveal sensitivity.²⁷ 

For any measure used in clinical practice or research, accurate interpretation of change requires characterization of the variability of measurements between visits. In IRD patients, the intervisit variability of rod-mediated responses, including dark-adapted thresholds and mean dark-adapted sensitivity, was reported to be higher than that found for cone-mediated responses.⁴ The intervisit variability of ERG amplitude in IRD patients was similar to controls in some studies,^2,4,5,28 but in another study, patients with lower baseline amplitudes manifested higher intervisit variability of ERG amplitudes.²⁹ Moreover, ERG amplitudes are often reduced below measurable levels in patients with more advanced stages of disease, thus complicating their use as a clinical outcome measure. Similarly, thresholds for significant change in visual field sensitivity range from 20% to 40% for both kinetic perimetry^11,28,30 and static perimetry.¹² Birch et al.¹⁴ reported a test-retest variability of 3.6% for central EZ band width in eyes with RP, a value considerably lower than values reported for ERG and visual field sensitivity testing. 

There are few reports of repeatability of AOSLO-derived structural cone measures. In a previous AOSLO study of cone structure, the estimated cone density measurement error in three IRD patients treated with ciliary neurotrophic factor (CNTF)²⁶ was 6.3%, mostly arising from cone selection errors. Cone density estimates measured at the same location in a single healthy subject were within only 2%³¹; in another study, the parafoveal cone density repeatability ranged between 2.7% to 17.1%, depending on the method of cone identification^32,33; these data were obtained from healthy, visually normal subjects, and little is known about reliability of AOSLO-derived cone measures among subjects with retinal diseases. With the potential to use AOSLO-derived quantitative measures of macular photoreceptors as an objective and sensitive outcome measure in clinical trials for IRD patients, their repeatability and reproducibility in this patient population must be determined. 

This study aims to determine the short-term variability of cone spacing measures derived from in vivo AOSLO images of visually normal subjects and of IRD patients. We measured intervisit, intergrader, and interocular short-term variability of AOSLO-derived cone spacing measures and assessed their correlation with measures of macular cone function in normal eyes and in IRD eyes of variable disease severity. This information is essential to establish and validate the reliability of cone spacing measures derived from images of the cone mosaic using AOSLO as a sensitive, objective, noninvasive biomarker that can be used for longitudinal tracking of disease progression and response to treatment in eyes with IRD. 

Methods

This prospective study included participants enrolled in a longitudinal study of normal eyes and eyes with inherited retinal degeneration (NCT00254605). In addition, the analysis included baseline data from 10 participants with RP and Usher syndrome type 3 enrolled in a phase 2 clinical trial assessing the effect of sustained-release CNTF on cone photoreceptor structure and function in IRD patients over 36 months (NCT01530659). The study was conducted at a single center where AOSLO images were acquired. All subjects gave written informed consent before participating in the study. All research procedures were performed in accordance with the Declaration of Helsinki. The study protocol was approved by the institutional review board of the University of California, San Francisco. 

Inclusion criteria were as follows: Subjects were enrolled if baseline AOSLO images contained unambiguous cone mosaics in at least seven different retinal locations within 4° of the fovea at each visit. 

Subjects with significant cataract, high myopia, or extensive cystoid macular edema that precluded acquisition of quantifiable AOSLO images at baseline were excluded. 

The study population consisted of two study groups: 

1.  
   Inherited retinal degenerations group, including 20 patients aged 18 to 65 years with RP, Usher syndrome type 3, or choroideremia; and
2.  
   Control group, including 10 visually normal, age-similar subjects.

A similar experimental approach was used for each of the study groups. High-resolution AOSLO images of macular cones were obtained from each subject at two baseline visits separated by no more than 1 month. All subjects underwent complete eye examination and determination of best-corrected visual acuity according to the early treatment of diabetic retinopathy study (ETDRS) protocol. Additional assessments at baseline included full-field ERG according to International Society for Clinical Electrophysiology of Vision standards³⁴ and visual field testing, including automated perimetry using Humphrey Swedish Interactive Threshold Analysis 10-2 protocol (Carl Zeiss Meditec, Inc., Dublin, CA, USA) and manual kinetic perimetry with a Goldmann perimeter using V4e and I4e targets. The Goldmann V4e target was used as a measure of peripheral visual field, whereas the I4e target was chosen because it correlates with extent of the IS/OS junction or ellipsoid zone band of SD-OCT scans³⁵; both of these were used as measures of disease severity. Imaging studies included SD-OCT (Spectralis HRA-OCT; Heidelberg Engineering, Vista, CA, USA), color and infrared fundus photography. Axial length was measured noninvasively using partial coherent interferometry with short-coherence infrared light (780 μm wavelength; Carl Zeiss Meditec, Inc.), as described previously.^26,36 

Genetic Testing

Whole blood was obtained for mutation analysis in IRD patients who had forms of RP for which genetic testing was available; genetic testing was not performed on RP cases without a history of Ashkenazi Jewish descent or a family history of consanguinity (patients 40023, 30015, 40031, 40043, 40046, 40041, 40037, 40039, 40047, 40058, 40026, and 40060). Five patients were tested through the eyeGENE consortium for molecular analysis of genes associated with x-linked RP (patients 40015 and 40049), with autosomal dominant RP (patients 40073 and 10048), and with choroideremia (patient 40028); no disease-causing mutations were identified in 10048. Genetic testing for mutations associated with Ashkenazi heritage for patient 40030 was performed through the Carver Nonprofit Genetic Testing Laboratory (Iowa City, IA, USA); no disease-causing mutations were identified in the DHDDS, LCA5, MAK, PCDH15, or CLRN1 genes. Genetic testing was performed through a research protocol (Radha Ayyagari, PhD, Project #081869, University of California San Diego, San Diego, CA, USA) using whole-exome sequencing^36,37 in families with autosomal recessive RP from a consanguineous pedigree (patient 40032). Mutation analysis of the CLRN1 gene in 30007 was carried out by sequencing the coding region. Patients' genomic DNA was extracted from whole blood samples with Puregene Genomic DNA PurificationKit (Gentra Systems, Minneapolis, MN, USA) or from saliva with Oragene kits (DNA Genotek, Inc., ON, Canada). The three exons and the exon-intron boundaries of the CLRN1 main splice variant (GenBank accession NM_174878) were screened for mutations by genomic sequencing, as previously described.^38,39 

Adaptive Optics Scanning Laser Ophthalmoscopy Image Acquisition and Processing

At each baseline visit, all patients underwent AOSLO imaging as previously described.¹⁷ Briefly, the AOSLO system uses a low-coherence, 840-nm light source, a Shack-Hartmann wavefront sensor, and a 140-actuator microelectromechanical (MEMS) deformable mirror (Boston Micromachines Corporation, Watertown, MA, USA). Digital videos were recorded throughout the central macular area of 5.7° in diameter, centered on the fovea. Each video subtended an area of 1.2° squared. 

Custom software was used to minimize distortions in images caused by eye movements.^40,41 After correction, static frames were averaged to increase the signal-to-noise ratio. These images were then arranged (Adobe Photoshop; Adobe Systems, Inc., San Jose, CA, USA) by aligning landmarks on overlapping images to create a continuous montage of the central macular cones. Image scales were computed from calibration images recorded before each imaging session to achieve a ratio of 420 × 420 pixels per degree in the final AOSLO montage. 

Selection of Regions of Interest

For each eye, the two baseline AOSLO montages were aligned with high-quality baseline clinical images, including color and infrared fundus photographs (Adobe Illustrator; Adobe Systems, Inc.). The resulting image overlays were used to correlate morphologic features noted in AOSLO images with other morphologic retinal measures (Fig. 1). For each study eye, a single author (AR) selected retinal regions of interest (ROIs) by identifying regions in which an unambiguous cone mosaic containing at least 20 cones was clearly visible at both baseline visits, with a goal to identify at least 1 ROI per degree within 2.85° radius of the anatomic fovea. Thus, among the entire study group, ROIs were selected in similar eccentricity ranges from the foveal center, determined by the location of the minimum height at the foveal depression on OCT. 

Figure 1

 

Baseline images from the right eye of IRD subject 40032. Left: Infrared SLO image obtained at baseline is shown with SD-OCT scan superimposed and area imaged with AOSLO outlined in black. White horizontal line represents location of SD-OCT scan. Right: Adaptive optics scanning laser ophthalmoscopy image obtained at one of the baseline visits. White boxes denote location of ROIs where cone spacing was analyzed in each image. White scale bar: 200 μm. Regions of interest are labeled with initials and numbers to indicate their relative position from the fovea on the montage: N, nasal; T, temporal; S, superior; I, inferior; R, right, L, left. When there is a second letter: U, upper; L, lower. The numbers are added sequentially as identifiers by the ROI selector (AR) but do not indicate eccentricity.

Cone Spacing Analysis

All AOSLO images from all study eyes were independently evaluated by three different graders from a pool of six graders, with each grader masked to the other graders. Each grader measured cone spacing in each ROI at each baseline AOSLO montage using customized software and methods previously described.^17,19,42,43 Briefly, each image was interpreted for the presence of features consistent with cone mosaics, including an orderly array of uniformly sized bright round or oval profiles. Individual cones within the identified mosaic were manually denoted. Although the exact area and location of each ROI were determined at the time of ROI selection, graders were allowed to shift their selections by no more than 0.2 arcminutes toward immediately neighboring regions of the mosaic if those regions contained better visualized, unambiguous arrays of cones. Average nearest-neighbor cone spacing was determined for each ROI from the first peak in the density recovery profile (DRP), a method devised by Rodieck⁴⁴ to quantify the spatial arrangement of cells.⁴² Density recovery profile plots the average histogram of density of all cells surrounding each cell in the mosaic as a function of distance of each cone from the central cone. Manual adjustment of the parameters (bin widths for the histogram, and the extent over which to make the Gaussian fit) was performed to best identify the location of the first peak in the histogram, as illustrated in Figure 2. Manually setting the bin width for the histogram depends on the actual cone spacing (locations with smaller cone spacing require denser bins) and the number of cones in the dataset (fewer cones generally require larger bins to increase the signal-to-noise ratio). Cone spacing was chosen as the primary outcome because it provides a robust and conservative measure for comparison among eyes. Cone spacing measures obtained from normal eyes at baseline were used to derive the range of normal eccentricity-dependent cone spacing values that served as control data. A double exponential function was fit to the spacing of the normal data:  

 

where A, B, and C are constants. Confidence intervals (95%) were estimated using the Matlab curve fitting toolbox (The MathWorks, Natick, MA, USA). Cone spacing measures calculated for each ROI in the IRD group were compared with the control range at similar retinal eccentricities.  

Figure 2

 

Method of cone spacing analysis. An ROI in a control subject (40053, right eye) (A) is shown with manually labeled cone photoreceptors (B). The polygonal packing pattern of cones in the mosaic can be seen. (C, D) Density recovery profile analysis for measuring cone spacing. (C) Average cone density in expanding annular rings around each selected cone. The first peak in the histogram indicates the nearest-neighbor-distance, or average cone spacing. Gaussian fit to the first peak in the DRP histogram is used to compute the average nearest-neighbor cone spacing.

Analysis of OCT Images

Line scans through the horizontal midline were obtained from each eye with SD-OCT (Spectralis HRA-OCT; Heidelberg Engineering). Images of a scan through the fovea were exported to data analysis software (Igor Pro; WaveMetrics, Inc., Portland, OR, USA) and manually segmented using subroutines^45–49 to identify boundaries between the different retinal layers. Average retinal thickness measures were computed at the foveal center and at the central 3°. For the purpose of measuring EZ band width, the three outermost segmentation lines were used: Bruch's membrane and the RPE junction, the photoreceptor outer segment (OS) and the RPE junction or interdigitation zone (IZ),¹³ and the inner-segment EZ, or inner-segment/outer-segment (IS/OS) junction. For each scan, the nasal and temporal borders of the EZ were defined as the locations where the thickness of the OS layer had declined to zero. The width of the EZ was defined as the horizontal distance between these two locations. The EZ band width was chosen because it was reported to be directly correlated to the visual field boundary in eyes with RP.^35,47 

Analysis of Goldmann Visual Field

Goldmann perimetry was performed with test targets V4e and I4e. For each study eye, the dimension from the center point to the radial boundary of each isopter was measured in degrees and areas not seen were subtracted. Measurements were taken every 30°, and the 12 measurements on each visual field were summed to determine the total area of each isopter. 

Statistical Analysis

Short-term variability of AOSLO-derived cone spacing measures was assessed for the following outcome measures: 

1.  
   Interobserver agreement: For each baseline AOSLO image from each study eye, cone spacing measures at each ROI were compared among all graders. Intraclass correlation coefficient (ICC)⁵⁰ values were separately calculated for each image to assess interobserver agreement on cone spacing measures at each ROI. For analysis of variance among graders, we used the clustered bootstrap⁵¹ and simple linear regression to summarize the difference in cone spacing measures among graders at each ROI to compute the SD. Polynomial regression was used to determine the SD among spacing measures obtained at different retinal eccentricities for each study group;
2.  
   Intervisit variability: For each study eye, cone spacing measures at each ROI in AOSLO images obtained at each baseline visit were analyzed separately by three independent graders. The ICC values were computed for measures of cone spacing in each study group. The mean absolute difference between cone spacing measures at each ROI obtained from each baseline visit (expressed in arcminutes) was calculated for each grader. We also determined the mean relative difference (a percentage calculated by dividing the absolute value of the difference between the two baseline measures by the mean of the two measurements and multiplying by 100) for each study eye by grader;
3.  
   Interocular variability: For each study subject, cone spacing measures were compared between fellow eyes. We used a regression model for each eye to estimate cone spacing at 1°, adjusting for grader, and predicted the spacing at 1° for the grader with the minimum variance. Using this model we predicted cone spacing at 1° for each eye. From this, we calculated ICCs for cone spacing measures from fellow eyes; and
4.  
   Correlation between variability of AOSLO-derived cone spacing measures and macular measures of disease severity: For each study eye from the IRD group, we studied the Spearman correlation between SD of cone spacing measures obtained by all graders at each ROI and SD-OCT–derived EZ bandwidth. Subjects whose EZ band extended beyond the edge of the scan were excluded from this analysis. Similarly, intergrader SD of spacing measures was correlated with clinical macular measures including the area of I4e and V4e isopters tested with Goldmann kinetic perimetry for all study eyes.

Results

Participant Characteristics

The characteristics of study participants are presented in the Table. To enroll the 20 patients included in this study, 34 patients with IRD were imaged using AOSLO; 10 IRD subjects showed fewer than seven ROIs in each eye on each baseline visit and were excluded, whereas 4 patients refused to return for a second imaging session despite the presence of at least seven ROIs in each eye after the first baseline session. Eleven of the 20 subjects included in the IRD group and 5 of the 10 control subjects were male. Ages ranged from 18 to 63 years (median 33 years) in the IRD group and 25 to 57 years (median 48.5 years) in the control group; ages were similar in both groups (P = 0.04, Wilcoxon rank sum test). Visual acuity ranged from 20/32 to 20/12.5 (ETDRS letters: 73–94) in the 40 eyes with IRD; visual acuity was significantly lower in the IRD compared with the control group, which ranged from 20/20 to 20/12.5 (ETDRS letters: 87–99) (P < 0.001, clustered bootstrap). Foveal sensitivities ranged between 27 and 40 dB in the IRD group and 34 and 43 dB in the control group; sensitivity was −3.56 dB lower in the IRD compared with the control group (95% CI: −4.90 to −1.20, P < 0.05, clustered bootstrap). One control subject (10017) underwent AOSLO imaging on two separate baseline sessions, but AOSLO images were obtained from the left eye only at baseline 2 visit; thus, only a single baseline AOSLO image was available for analysis from the right eye of this participant. Full-field ERG was not obtained from two participants in the control group (10017 and 30016) due to technical issues with the ERG system at the time of screening. All other clinical testing performed according to the study protocol on these two subjects yielded normal results. Another subject from the IRD group (40049) withdrew consent from the study during the screening phase. The ERG testing was not done on this subject but previous ERG tests had resulted in unmeasurable responses to all stimuli. All other clinical images and AOSLO-derived cone spacing measures obtained from this subject before withdrawal of consent are included in the current report. 

Table

 

Demographic Data and Clinical Measures of Study Participants at Baseline

 

Adaptive Optics Scanning Laser Ophthalmoscopy–Derived Cone Spacing Measures

Overall, 1676 ROIs were analyzed in the 80 images obtained from the IRD group, with an average of 21 ROIs analyzed per image (range, 7–31). In the control group, cone spacing was measured in 1229 ROIs in the 39 images with an average number of 31.50 ROIs measured per image (range, 23–35). Figure 3 plots cone spacing measures calculated for all ROIs in the study compared with retinal eccentricity. Ten (0.81%) cone spacing measures in the normal group were greater than the 95% CI range and 6 (0.49%) were below, whereas 241 (14.38%) cone spacing measures in the IRD group were greater than and 15 (0.89%) were below the normal 95% CI range. 

Figure 3

 

Adaptive optics scanning laser ophthalmoscopy–derived cone spacing measures from control subjects and patients with IRD, plotted as a function of retinal eccentricity. Left: Data from control subjects were used to derive best fit to the normal mean (middle dashed line). Outer dashed lines: 95% confidence limits of the best fit. Right: Cone spacing data from the group of patients with IRD are shown in reference to the normal range (red dashed lines).

Interobserver Agreement on AOSLO-Derived Cone Spacing Measures

For each baseline AOSLO image, we computed cone spacing measures from all graders at each ROI. Single-score ICC values were 0.84 (95% CI 0.59–0.95) for the control group and 0.89 (95% CI 0.71–0.96) in the IRD group. Different guidelines exist for the interpretation of ICC, but one reasonable scale is that an ICC value of less than 0.40 indicates poor reproducibility, ICC values in the range 0.40 to 0.75 indicate fair to good reproducibility, and an ICC value of greater than 0.75 shows excellent reproducibility.^50,52 Thus, our results indicate overall strong agreement among graders for cone spacing measures in both study groups. 

We summarized the intergrader variability by computing the SD of grader disagreement in cone spacing measures obtained at each ROI. The median of the resulting SD was 0.03 (95% CI 0.026–0.034) arcminutes for control eyes and 0.037 (95% CI 0.032–0.044) arcminutes for eyes with IRD (P < 0.01). 

Intervisit Variability of AOSLO-Derived Cone Spacing Measures

Figure 4 shows representative images of cone mosaics at corresponding ROIs obtained at each baseline visit from a control subject and a subject from the IRD group. In both examples, AOSLO imaging showed continuous cone mosaics for which spacing measures differed by 1.1% to 1.2% between baseline visits. 

Figure 4

 

Intervisit agreement of cone mosaic measures in repeated AOSLO images. Left: Representative images of cone mosaics at corresponding ROIs obtained at each baseline visit are shown from one control subject (top) and one patient from the IRD group (bottom) in which agreement was high. In both subjects, AOSLO imaging showed continuous cone mosaics with similar appearance for which cone spacing measures compared favorably between baseline visits. Right: Representative images of cone mosaics at corresponding ROIs at each baseline visit from one control subject (top) and one patient from the IRD group (bottom) show examples in which cone spacing measures were more discrepant than the reported mean.

We computed the ICC for assessing intervisit agreement on spacing measures obtained from each study group at each visit for each of our graders. For healthy subjects, the ICC ranged from 0.93 to 0.95, whereas for patients with IRD, the ICC ranged from 0.87 to 0.98. We found that the mean absolute difference in cone spacing measures for each grader at each ROI was 0.054 arcminutes (95% CI 0.049–0.061) in the control group and 0.066 arcminutes (95% CI 0.058–0.075) in the IRD group. The mean relative difference (in percentage terms) in intervisit cone spacing measures for each grader at each ROI was 4.03% (95% CI 3.73–4.54) in the control group and 4.9% (95% CI 4.41–5.49) in the IRD group. 

Interocular Variability of AOSLO-Derived Cone Spacing Measures

For each subject, we calculated interocular ICC values based on predictions of cone spacing at 1° of retinal eccentricity, controlling for grader. We found that eyes from the same subject had higher ICC values for cone spacing measures than any two randomly selected eyes. This was true for the subgroup of control eyes (ICC 0.86, 95% CI 0.55–0.96) as well as for eyes with IRD (ICC 0.96, 95% CI 0.90–0.98). 

The adjusted mean absolute difference in spacing measures between fellow eyes was 0.015 arcminutes (95% CI 0.0090–0.020) for control eyes and 0.038 arcminutes (95% CI 0.024–0.054) for eyes with IRD. 

Correlation of Variability of AOSLO-Derived Cone Spacing With Standard Macular Measures

We correlated SD of cone spacing measures with standard macular measures of IRD disease severity to determine whether clinical macular findings affected variability of AOSLO-derived cone spacing measures. The Spearman correlation between SD for cone spacing measures among graders and the SD-OCT–derived EZ band width was −0.45 (95% CI −0.69 to −0.095) for eyes with IRD (P < 0.01). The Spearman correlation between estimated SDs among graders and the Goldmann I4e area was significant at −0.61 (95% CI −0.80 to −0.32, P < 0.001) for eyes with IRD, but was not significant at 0.41 (95% CI −0.033 to 0.68, P = 0.062) in healthy subjects. Similarly, the Spearman correlation between the estimated SD among graders and the Goldmann V4e area was −0.42 (95% CI −0.67 to −0.063, P = 0.029) for eyes with IRD, but was not significant for healthy subjects at 0.22 (95% CI −0.35 to 0.70, P = 0.44). Further, the Spearman correlation between estimated SD of the graders and SD-OCT–derived total retinal thickness at the foveal center was −0.67 (95% CI −0.83 to −0.32, P < 0.005) for eyes with IRD and not significant at −0.10 (95% CI −0.55 to 0.36, P = 0.67) for healthy subjects. The Spearman correlation between estimated SD of the graders and SD-OCT–derived retinal thickness at the central 3° was −0.66 (95% CI −0.84 to −0.29, P < 0.01) for eyes with IRD, but was not significant in the healthy group at −0.26 (95% CI −0.63 to 0.16, P = 0.23). There was no evidence that any of these clinical measures of disease severity was significantly correlated with SD for cone spacing measures among graders in normal eyes, possibly due to a roof effect in the healthy subjects. The results indicate that with increasing disease severity, interobserver agreement on cone spacing tended to decrease, as measured by increasing SD of cone spacing measures. 

Discussion

Examination by high-resolution AOSLO images enables in vivo visualization of cone photoreceptors and allows quantitative measurement of cone structure in eyes with IRD. Using AOSLO, we studied 40 eyes with IRD manifesting different levels of disease severity and 20 normal eyes. We found substantial intergrader, intervisit, and interocular agreement of macular cone spacing measures in normal eyes as well as in eyes with IRD. Further, standard macular measures including SD-OCT–derived EZ band width and retinal thickness, as well as Goldmann visual field area, were inversely correlated with intergrader variation in eyes with IRD, suggesting that AOSLO images from patients with more severe disease stages were associated with higher intergrader variability. This information is important for validating the clinical utility of AOSLO-derived cone measurements, and specifically for distinguishing between short-term variability of measures and longitudinal change of measures. Moreover, the data are valuable for evaluating change in AOSLO cone spacing measures during disease progression and between treatment groups in future clinical trials. 

Our comparison of AOSLO-derived cone spacing measures among three graders showed strong agreement in normal eyes as well as in eyes with IRD. We have previously reported that in eyes with RP, cone spacing increases near the edge of the remaining macular photoreceptor mosaic, beyond which RPE cell mosaics may be seen in some patients.^17,43 It is possible that abnormal cone structure along the transition zone between the healthier center and more severely affected peripheral macula^53,54 contributed to increased intergrader variability in IRD cone spacing measures found at regions of retinal eccentricity greater than 3°. In addition, errors in cone selection can affect variability in cone spacing measures³² and may be more likely when cones have abnormal imaging properties. Nonetheless, the overall substantial agreement on cone spacing measures found among three graders for control eyes as well as for eyes with IRD highlights the feasibility of objectively and reliably measuring structural cone mosaic parameters by independent, trained AOSLO image graders. 

We found a strong correlation of cone spacing measures obtained at two visits separated by no more than 1 month in eyes with IRD as well as in normal eyes (ICC > 0.87 for both study groups). The variance of measures obtained between visits was similar in control eyes and in eyes with IRD, and the mean relative difference in measures found between visits in each study group was small (<5%). Intervisit repeatability of spacing measures may be affected by differences in image quality obtained at each visit. Even when the images are obtained within a short interval during which no observable photoreceptor degeneration is expected to have occurred, the visibility of cone mosaics can be affected by small changes between testing sessions. Nonetheless, we found substantial intervisit agreement among all graders, suggesting that image artifact or analysis bias likely did not contribute significantly to variability of cone spacing measurements between testing sessions. High test–retest repeatability of cone spacing measures based on AOSLO images suggests cone spacing is a robust biomarker that can be used for longitudinal tracking of disease progression and response to treatment in eyes with IRD. These data may also aid in design of future clinical trials in terms of estimating disease progression based on change in AOSLO-derived cone spacing measures and in terms of comparing AOSLO-derived outcome measures between treatment groups. Future studies that correlate longitudinal measures of cone spacing with measures of visual function will elucidate whether statistically significant changes in AOSLO-derived cone spacing measures are clinically meaningful. 

We evaluated the similarity of macular cone spacing between fellow eyes in the healthy subjects as well as in patients with IRD manifesting different levels of disease severity. The correlation between predicted cone spacing at 1° of eccentricity from the foveal center between fellow eyes of either healthy subjects or subjects with IRD was stronger than that observed for any other randomly selected pairs of eyes (ICC > 0.85 for both study groups). The interocular ICC was lower for healthy subjects, likely because the range of measures obtained by all graders in healthy subjects was lower than in subjects with IRD, although the difference between the ICCs for normal and IRD groups did not reach statistical significance. The difference in mean adjusted cone spacing between eyes was smaller in healthy subjects compared with patients with IRD. A histologic study of a single healthy subject by Curcio et al.⁵⁵ showed an overall mean interocular rod and cone density difference of 8% within the central 6-mm retina. In agreement with our results, a previous study of interocular similarity of cone density in healthy subjects reported overall high agreement at corresponding retinal eccentricities along the nasal and temporal regions of fellow eyes.⁵⁶ The literature about symmetry of histopathological retinal damage in eyes with IRD is limited, but RP is broadly considered a bilateral condition that affects both eyes in a highly symmetrical fashion.^57–59 The substantial interocular similarity of cone spacing measures found in the present study in eyes with IRD may be valuable for design of future clinical trials investigating the effect of new therapies on photoreceptor loss in eyes with IRD, suggesting a rationale that fellow nontreated eyes might provide a control reference for the outcome measured in the treated eyes. 

We found correlations of varying strength between intergrader variability of cone spacing measures and standard macular measures, including SD-OCT–derived EZ band width and area of I4e and V4e Goldmann perimetry area of visual field seen. Greater intergrader variability was seen in eyes with more advanced disease severity. Eyes with advanced stages of retinal degeneration have been reported to show significant retinal structural damage due to degeneration of photoreceptors.^60,61 Further, in retinal regions in which photoreceptors are severely diminished, RPE cells may become visible in AOSLO images.⁴³ Such structural disruptions may make identification of unambiguous cone mosaics challenging and thus affect reproducibility of AOSLO-derived cone spacing measures; however, newer imaging techniques including the use of split detection images will likely help distinguish ambiguous features in confocal images of degenerating retinal tissue.⁶² Last, image resolution quality may be more decreased in eyes with IRD due to increased prevalence of cataract, vitreous floaters, and cystoid macular edema. These factors can affect interobserver agreement on cone spacing measures obtained from AOSLO images. 

Cone spacing was selected as the AOSLO image metric in our study because it provides a conservative and robust measure of the structural integrity of the cone mosaic, and permits measurement even when image quality does not allow visualization of every cone in the mosaic. However, cone spacing measures may overestimate the global health of the photoreceptor mosaic, because cone spacing is measured only in regions in which cones are clearly seen within a mosaic. Future studies determining the variability of additional photoreceptor mosaic metrics such as AOSLO-derived cone packing or cone density in eyes with IRD may further contribute to understanding of the structural integrity of the mosaic and may facilitate accurate longitudinal tracking of disease progression. 

A potential limitation to our study is that only subjects with at least seven ROIs per eye were included in the study. Ocular comorbidities, including cataract, cystoid macular edema, and vitreous floaters, are frequently found in eyes with IRD and may limit AOSLO image resolution quality. Image resolution artifacts may falsely affect cone spacing measures and variability of cone spacing measures. Because we included only subjects with image quality that permitted quantitative analysis, this likely improved our variability measures. Determination of short-term variability of AOSLO-derived cone spacing measures in eyes with poor AOSLO image quality merits further investigation. Another potential limitation is the use of a single author to select ROIs, which may have introduced bias into the cones selected. We felt that a single, experienced, masked observer was necessary to select the ROIs with the most unambiguous cones, which may have led to an overestimation of true repeatability, if repeatability increases with better cone visualization, or perhaps underestimation of true repeatability, if the single author's ROI selections were biased in some way. Finally, the cone locations were identified manually by three independent, trained graders because automated cone-counting methods have not been shown to reliably identify cones in eyes with retinal degenerations. The current article describes the variability in cone spacing measures between graders and visits using manual measurement of cone spacing with trained graders. Comparison between cone spacing measures derived from trained graders and automated cone-counting algorithms would be valuable, but is beyond the scope of the current article. 

Strengths of this study include the large number of images and ROIs analyzed in each study group, allowing us to establish a strong database for the range of AOSLO-derived cone spacing measures found in normal eyes as well as in eyes with IRD. 

In summary, we observed excellent intergrader, intervisit, and interocular agreement of cone spacing measures derived from high-resolution AOSLO imaging in eyes with IRD. We found strong agreement among graders on cone spacing measures in control eyes as well as in eyes with IRD, and we also found good test–retest agreement of measures obtained at baseline in control eyes and in eyes with IRD with different levels of disease severity. Repeatability of measures may differ between imaging systems, based on the properties and methods used. Nonetheless, our data suggest that AOSLO-derived cone spacing is an appropriate measure for patients at different stages of disease severity with preserved cones within 3° of the fovea, provided that high-resolution images can be obtained. Further, these results may facilitate design of outcome measures in future clinical trials investigating new therapies for eyes with IRD. 

Acknowledgments

The authors gratefully acknowledge the efforts and expertise of Radha Ayyagari, PhD, for whole exome sequencing to identify the mutation in the patient 40032. 

Supported by an Alan Laties Career Development Award (SZS), Foundation Fighting Blindness (SZS, JLD, TCP, AR), National Institutes of Health Grants EY002162 (JLD) and EY014375 (JLD, AR), Research to Prevent Blindness (JLD), US Food and Drug Administration Office of Orphan Product Development FDA R01 FD004100 (JLD, TCP, AR), The Bernard A. Newcomb Macular Degeneration Fund (JLD), That Man May See, Inc. (JLD), Hope for Vision (JLD), University of California San Francisco (UCSF)-CTSI Grant TL1 TR000144 (SKL), UCSF Research Allocation Program Novel Clinical/Translational Methods Award (RS), and The George and Rosalie Hearst Foundation (MM). 

Disclosure: S. Zayit-Soudry, None; N. Sippl-Swezey, None; T.C. Porco, None; S.K. Lynch, None; R. Syed, None; K. Ratnam, None; M. Menghini, None; A.J. Roorda, P; J.L. Duncan, Foundation Fighting Blindness (S)