Deep Learning–Facilitated Study of the Rate of Change in Photoreceptor Outer Segment Metrics in RPGR-Related X-Linked Retinitis Pigmentosa

Abstract

Purpose: The aim of this retrospective cohort study was to obtain three-dimensional (3D) photoreceptor outer segment (OS) metrics measurements with the assistance of a deep learning model (DLM) and to evaluate the longitudinal change in OS metrics and associated factors in retinitis pigmentosa GTPase regulator (RPGR) X-linked retinitis pigmentosa (XLRP).

Methods: The study included 34 male patients with RPGR-associated XLRP who had preserved ellipsoid zone (EZ) within their spectral-domain optical coherence tomography volume scans and an approximate 2-year or longer follow-up. Volume scans were segmented using a DLM with manual correction for EZ and apical retinal pigment epithelium (RPE). OS metrics were measured from 3D EZ–RPE layers of volume scans. Linear mixed-effects models were used to calculate the rate of change in OS metrics and the associated factors, including baseline age, baseline OS metrics, and follow-up duration.

Results: The mean (standard deviation) of progression rates were −0.28 (0.43) µm/y, −0.73 (0.61) mm²/y, and −0.014 (0.012) mm³/y for OS thickness, EZ area, and OS volume, respectively. In multivariable analysis, the progression rates of EZ area and OS volume were strongly associated with their baseline values, with faster decline in eyes with larger baseline values (P ≤ 0.003), and nonlinearly associated with the baseline age (P ≤ 0.003). OS thickness decline was not associated with its baseline value (P = 0.32).

Conclusions: These results provide evidence to support using OS metrics as biomarkers to assess the progression of XLRP and as the outcome measures of clinical trials. Given that their progression rates are dependent on their baseline values, the baseline EZ area and OS volume should be considered in the design and statistical analysis of future clinical trials. Deep learning may provide a useful tool to reduce the burden of human graders to analyze OCT scan images and to facilitate the assessment of disease progression and treatment trials for retinitis pigmentosa.

Retinitis pigmentosa (RP) is a class of inherited retinal degenerations involving progressive loss of photoreceptors. Inheritance can be categorized as isolated (simplex), autosomal dominant, autosomal recessive, or X-linked recessive based on family history.^1–4 The proportion of X-linked RP (XLRP) is estimated to be between 6%⁵ and 16%⁶ of RP prevalence in the United States. Patients with XLRP showed earlier onset and more rapid progression to blindness when compared to other forms of RP.^7–10 Hence, XLRP imposes a comparatively greater lifetime burden of the disease.¹¹ With potential new and emerging treatments on the horizon for RP in general and XLRP in particular,^12–15 it is essential to understand the natural history of the disease and determine the most efficient and sensitive biomarkers for disease progression for use as outcome measures in future treatment trials. 

Studies using optical coherence tomography (OCT) scan images have demonstrated that the structural defects in XLRP mainly occur in the outer retina (e.g., photoreceptors) as the disease progresses.^16–18 The disruption of the ellipsoid zone (EZ) is clearly evident in an OCT B-scan image when the change of photoreceptor outer segment (OS) from visible to disappearing is within the scan area. It has been documented that patients with retinitis pigmentosa GTPase regulator (RPGR)-associated XLRP exhibit shortening of OS in the early stages of the disease.¹⁹ Hence, measurement of the surviving OS thickness or volume, as well as EZ width or area, may provide effective biomarkers for RP. It has been suggested that both preserved EZ width²⁰ and EZ area²¹ could serve as robust biomarkers to be used as outcome measures in treatment trials.²² 

To our knowledge, no published studies have yet examined changes in OS volume in RP. One of the challenges is that obtaining reliable OS volume measurement requires accurate segmentation of OS boundaries, including both the EZ band and apical retinal pigment epithelium (RPE). Current conventional automatic OCT segmentation algorithms often incorrectly identify EZ transition zones; thus, manual segmentation by human graders is required for accurate delineation of the OS, which is time consuming and costly.²³ Recent advances in deep learning (DL)–based neural networks have provided new techniques for automatic segmentation of retinal layers from OCT scan images.^23–27 In the past few years, we have developed several deep machine learning models to quantify OS metrics.^23,26,27 The purpose of this study was to obtain three-dimensional (3D) OS metrics measurements with the assistance of our deep learning models (DLMs) and to evaluate the rate of change in OS metrics and factors associated with disease progression in RPGR XLRP through a retrospective longitudinal study. 

Methods

Study Patients

This retrospective longitudinal study reviewed male patients with XLRP in the Southwest Eye Registry at the Retina Foundation of the Southwest who typically were followed up at 1- to 2-year intervals. The inclusion criteria consisted of (1) volume scans having the limits of the preserved EZ band within the scan limit and having detectable EZ in at least three B-scans within a single volume scan; (2) a minimum of approximately a 2-year follow-up; and (3) having mutations in the RPGR gene located on the X-chromosome. For patients who participated in treatment trials, volume scans obtained after the treatment were excluded for the treated eyes. Among 115 male patients with RPGR-associated XLRP who had their OCT scan visits reviewed, 47 had less than 2 years of follow-up, 28 had minimal or no detectable EZ, and six had preserved EZ extended beyond the volume scan window. The remaining 34 male patients met all of the inclusion criteria and were selected for this study. Among these 34 patients, two eyes from two patients did not have 2-year follow-up before treatment and were excluded from the study, leaving 66 eyes eligible for the study. A subset of these patients (n = 26) was reported in a previous study that examined the rate of progression of EZ width in XLRP,²⁰ and additional data were collected from 10 of them after the last report. 

Informed consent was obtained from the subjects before the data were collected. The study was in accordance with the tenets of the Declaration of Helsinki and was approved by the Institutional Review Board at UT Southwestern Medical Center. 

Automatic Retinal Layer Segmentation and Manual Correction

A total of 342 OCT volume scans composed of 186 low-density scans (mean B-scan separation, 0.24 mm) and 156 high-density scans (mean B-scan separation, 0.066 mm) obtained between January 14, 2008, and July 25, 2022, using SPECTRALIS spectral-domain optical coherence tomography (Heidelberg Engineering, Heidelberg, Germany), were identified for the data analysis of this study. All volume scans were exported as XML files, which were then imported into MATLAB (MathWorks, Natick, MA, USA) to create a structured data type containing B-scan images and scan parameters for each OCT scan to be processed by DLMs. 

All B-scan images in a volume scan were first segmented automatically using a DLM for retinal layer boundaries of inner limiting membrane, distal (basal) inner nuclear layer, EZ, proximal (apical) retinal pigment epithelium (pRPE), and Bruch's membrane. The details of the DLM have been reported previously.^26,27 In short, the DLM employed was a hybrid model²⁷ composed of first a U-Net convolutional neural network (CNN)²⁸ for initial, fast semantic segmentation of OCT B-scan images to obtain single-pixel boundary lines, then a sliding-window CNN model^23,29 for refinement by correcting potential errors made in the initial segmentation. The specific model used in this study was the RP340 model as reported in Wang and Birch.²⁶ The model was implemented in MATLAB and trained on a Mac Pro desktop computer (3.2-GHz 16-Core Intel Xeon W with 192-GB 2933-MHz DDR4 RAM; Apple, Cupertino, CA, USA). 

Automatically segmented B-scan images in all volume scans were then checked and manually corrected by two experienced human graders for any errors made by the DLM for the boundary lines of the EZ and pRPE. Manual correction was performed using the software of the Manual Segmentation Utility created by the Hood Visual Science Laboratory (New York, NY, USA).³⁰ 

Photoreceptor OS Metric Measurements

To obtain 3D OS metric measurements (OS thickness, EZ area, and OS volume), EZ and pRPE in each B-scan image of a volume scan were extracted to obtain a two-dimensional (2D) OS layer. The 3D OS map from each volume scan was reconstructed by interpolating the discrete 2D OS layers from individual B-scans over the grid of scan area. Figure 1 illustrates examples of OS thickness on mid-line B-scans from two volume scans (red areas in Figs. 1A, 1D), the preserved EZ area overlapped with infrared fundus images (Figs. 1B, 1E), and the 3D OS volume maps (Figs. 1C, 1F). OS metrics measurements were then obtained from the reconstructed 3D OS maps. Specifically, mean OS thickness was the average of all non-zero single-pixel OS thicknesses over the grid; EZ area was estimated by multiplying the area of a single grid pixel by the number of pixels with measurable OSs; and OS volume was the sum of the products of the OS thickness at each pixel and the pixel area. 

Figure 1.

 

Examples of OS metric measurements. The data were obtained from the left eye of patient P22. (Top row) Volume scan obtained at the baseline. (Bottom row) Volume scan obtained 10.6 years after the baseline visit. (A, D) OS thickness (red area) on the mid-line B-scans from the volume scans. (B, E) Preserved EZ area overlapped with infrared fundus images; (C, F) OS volume maps. Note that, in the volume maps, the z-axis is in micrometers, different from the x-axis and y-axis, which are in millimeters.

Average OS metric measurements obtained from manual correction of the DLM segmentation by two human graders were used for longitudinal statistical analysis. To compare the performance of the DLM with manual correction to that of the DLM only, the averages of three OS metric measurements were obtained.²⁶ 

Statistical Analysis

For each OS metric (OS thickness, EZ area, or OS volume), multivariable linear mixed-effects models that adjusted by age, its baseline value, follow-up duration, and OCT scan density and accounted for both inter-eye correlation and longitudinal correlation³¹ were used to calculate the progression rate in each eye. The regression models were then applied to determine the factors associated with the progression rate of each OS metric, including baseline age and baseline OS metrics (grouped based on quartiles). Their associations with progression rate were assessed using linear-trend P values and R². In addition, EZ area progression was modeled with and without square-root transformation to evaluate whether the square-root transformation eliminates the dependency of progression rate on baseline values. Similarly, OS volume progression was modeled with and without cube-root transformation to evaluate whether the cube-root transformation eliminates the dependency of progression rate on baseline values. Bland–Altman plots³² were employed to assess agreement of the OS metrics measurements from the DLM with versus without manual correction. All of the statistical comparisons were made in SAS 9.4 (SAS Institute, Cary, NC, USA), and two-sided P < 0.05 was considered statistically significant. 

Data Availability

The datasets generated during the current study and the algorithm employed to obtain automatic segmentation and OS metrics measurements are available at https://github.com/yzwang23/RFSW_RP_DLM/. 

Results

Rate of OS Metrics Progression and Associated Factors

Table 1 shows the demographics, follow-up length, number of follow-up visits, and baseline characteristics of the participants. Age at the first visit for OCT volume scans ranged from 8.4 to 45.6 years. Race/ethnicity was reported as white for 30 participants (88%), black for one participant (3%), Hispanic for no participants (0%), and other for three participants (9%). The length of follow-up ranged from 1.9 to 12.5 years. The number of follow-up visits ranged from two to 16. Baseline visual acuity and OS metrics measurements are also reported in this table. 

Table 1.

 

Demographics, Follow-Up Length and Number of Visits, and Baseline Characteristics of Participants

 

Linear mixed-effects models were used to calculate progression rates of OS metrics for each eye. The overall mean (SD) progression rates were −0.28 (0.43) µm/y for OS thickness, −0.73 (0.61) mm²/y for preserved EZ area, and −0.0142 (0.0118) mm³/y for OS volume. Relative to their mean at baseline, the estimated annual percent progression rates were −1.5% for OS thickness, −9.6% for EZ area, and −9.2% for OS volume. 

Figure 2 shows OS metrics as a function of the age at the time when OCT images were obtained for the right eye of each participant. Both preserved EZ area (Fig. 2B) and OS volume (Fig. 2C) had a clear decline over time, and the decline rate was smaller in eyes with a lower baseline value. In comparison, OS thickness (Fig. 2A) showed an overall trend of decline over time with much larger intersubject and intergrader variabilities. The results obtained from the left eyes of these patients were very similar to those of the right eyes. The inter-eye correlation coefficients in the progression rate of OS metrics were 0.37 (95% confidence interval [CI], 0.03–0.64) for OS thickness, 0.90 (95% CI, 0.80–0.95) for EZ area, and 0.88 (95% CI, 0.77–0.94) for OS volume. 

Figure 2.

 

OS metric measurements obtained using the DLM with manual correction as a function of the age at the time when the OCT image was obtained. (A) Mean OS thickness; (B) EZ or OS area; and (C) OS volume. Only the data obtained from the right eyes are shown in this figure (n = 33). One patient had only the left eye included in the study because the right eye did not have 2-year follow-up before treatment. Error bars indicate ±1 SD of two OS metric measurements from the manual correction of DLM segmentation by two human graders. To reduce the cluster introduced by the error bars and to improve the visualization, (A) shows only the representative error bars for a few selected patients. “P” in the figure refers to individual patient.

Table 2 shows the multivariable analysis results for progression rates of OS metrics by the levels of the associated factors. The grouping of eyes was based on quartiles of baseline OS metrics measurements to ensure that subgroups had similar sample sizes. The progression rates of preserved EZ areas and OS volumes were strongly associated with their baseline values, with faster declines in eyes with larger baseline values (P ≤ 0.003), but OS thickness was not associated with its baseline value (P = 0.32). In addition, progression rates in preserved EZ area and OS volume were nonlinearly associated with the baseline age (P ≤ 0.003), with the largest decline in age at 10 to 15 years. Progression rates were not associated with the length of follow-up (P ≥ 0.20) or OCT scan density (P ≥ 0.18). 

Table 2.

 

Rate of OS Metrics Progression by Levels of the Associated Factors

 

Figure 3 plots mean OS metrics measurements as a function of follow-up time in years. All visit dates were rounded up to the nearest integer years for calculating their mean values at annual intervals. Due to the varied follow-up lengths for the participants, the number of patients having OCT scans decreased with time. To ensure a sufficient sample size, Figure 3 only shows the average OS metrics measurements from the first 3 years of follow-up visits where the number of patients was 34 (66 eyes) for the baseline visit, 27 (51 eyes) for the year 1 visit, 25 (47 eyes) for the year 2 visit, and 18 (34 eyes) for the year 3 visit. The linear regression slopes from Figure 3 are −0.239 µm/y for OS thickness, −1.065 mm²/y for preserved EZ area, and −0.023 mm³/y for OS volume, in close agreement with the rates of change of OS metrics determined by the linear mixed-effects model analysis. 

Figure 3.

 

Mean OS metric measurements versus follow-up time in years. (A) Mean OS thickness; (B) mean EZ area; and (C) mean OS volume. Zero on the horizontal axis represents the baseline visit. The vertical axis scale is about a third of that in Figure 2 for the corresponding OS metric. The dashed lines and formulas are linear regressions fit to the data. The number of patients in each follow-up year was 34 (66 eyes) for the baseline visit, 27 (51 eyes) for the year 1 visit, 25 (47 eyes) for the year 2 visit, and 18 (34 eyes) for the year 3 visit. Error bars indicate ±1 SE of mean OS metric measurements. R² is the coefficient of determination for the association between the OS metrics and follow-up time.

Square Root Transformation of Preserved EZ Area and Cube Root Transformation of OS Volume

Table 2 also shows the progression rates calculated from the square roots of preserved EZ areas and the associated factors. The overall mean (SD) progression rate was −0.157 (0.111) mm/y for the square root of preserved EZ area. Relative to their mean baseline value (Table 1), the estimated annual percent rate of progression was −6.4% for the square root of preserved EZ area, which was strongly associated with its baseline value (P = 0.008) and the baseline age (P < 0.001). As illustrated in Table 2 and Figure 4, R² for the association between the preserved EZ area progression rate and its baseline value was 0.51, double that between the square root of the preserved EZ area progression rate and its baseline value. 

Figure 4.

 

Progression rate of EZ area (A) and its square root (B), as well as OS volume (C) and its cube root (D) as a function of their baseline values for all eyes (n = 66). The rates of progression were obtained from the multivariate linear mixed-effects model analysis conducted on the average of two measurements from the DLM segmentation with manual correction by two graders. Solid lines represent the linear fit to the data.

In addition, Table 2 includes the progression rates of the cube root of OS volume and the associated factors. The overall mean (SD) progression rate was −0.023 (0.015) mm/y for the cube root of OS volume. Relative to their mean baseline values (Table 1), the estimated annual percent rate of progression was −4.9%. Unlike the square root of preserved EZ area but similar to OS thickness, the change of cube root of OS volume was not associated with its baseline value (P = 0.11). Figures 4C and 4D plot the progression rates of OS volume and cube roots of OS volume against their baseline values, respectively. R² was 0.42 between the progression rate of OS volume and its baseline value. In comparison, R² was 0.11 between the progression rate of the cube root of OS volume and its baseline value. 

Agreement in OS Metrics Measurements Between the DLM Without Versus With Manual Correction

Figure 5 shows Bland–Altman plots comparing OS metrics determined by the DLM without versus with manual correction. Here the average measurements of two graders were used to compare with the DLM only to reduce the variability, as there was a performance difference between grader 1 and grader 2. The DLM alone overestimated OS thickness by a mean (SD) of 2.5 (1.37) µm, underestimated the preserved EZ area by a mean (SD) of −0.87 (0.94) mm², and underestimated the OS volume by a mean (SD) of −0.008 (0.013) mm³. The coefficients of repeatability (CoRs) calculated as 1.96 times the SDs of the difference³² were 2.68 µm for OS thickness, 1.83 mm² for preserved EZ area, and 0.025 mm³ for OS volume, respectively. 

Figure 5.

 

Bland–Altman plots of differences in the OS metric measurements between the DLM only and the DLM with manual correction (vertical axis, DLM only minus DLM with manual correction) versus their mean (horizontal axis) for mean OS thickness (A), EZ area (B), and OS volume (C). Because different OS metrics have different units, the scale (or the range) of the vertical axis in each plot was set to equal to that of its horizontal axis, so that the difference was “normalized” to the range of each OS metric measurement for easy comparison. The text inside each part of the figure lists the values of mean difference (Mean Diff), standard deviation of the difference (SD), and coefficient of repeatability (CoR, defined as 1.96 times SD of the difference). Dashed horizontal lines represent ±95% limits of agreement (mean ± CoR). Dotted horizontal lines represent the mean difference.

As a comparison, Bland–Altman analysis of the OS metrics measurements between two human graders revealed a mean difference (SD) of 3.15 (1.68) µm for OS thickness, −0.88 (0.93) mm² for preserved EZ area, and −0.004 (0.006) mm³ for OS volume. CoRs were 3.29 µm for OS thickness, 1.82 mm² for preserved EZ area, and 0.012 mm³ for OS volume. The agreement in OS metric measurements between the DLM with versus without manual correction of human graders was comparable to that between two human graders. 

The performance of the DLM only was also compared with that of individual graders. For the DLM only versus grader 1, the mean differences (SD) were 0.97 (1.36) µm for OS thickness measurement, −0.43 (0.63) mm² for preserved EZ area measurement, and −0.006 (0.011) mm³ for OS volume measurement. The CoRs were 1.17 µm for OS thickness, 1.23 mm² for preserved EZ area, and 0.022 mm³ for OS volume. For the DLM only versus grader 2, the mean differences (SD) were 4.12 (1.81) µm for OS thickness, −1.31 (1.34) mm² for preserved EZ area, and −0.010 (0.014) mm³ for OS volume. The CoRs were 4.12 µm for OS thickness, 2.62 mm² for preserved EZ area, and 0.028 mm³ for OS volume. 

Discussion

This study evaluated the progression rate of OS metrics in male patients with XLRP associated with an RPGR mutation. The results of this study demonstrated for the first time that, with the assistance of DLMs, it is possible to analyze volumetric measurements of photoreceptor OSs in a large OCT image dataset of XLRP. This allowed quantitative measurement of longitudinal changes of OS volume, as well as preserved EZ area and OS thickness, to assess disease progression. The linear mixed-effects models revealed that the progression rates of preserved EZ area and OS volume were dependent on their baseline values, with faster declines in eyes with larger baseline values. Relative to their means at baseline, the overall annual decreases for preserved EZ area and OS volume were around 9.5%. A previous study predicted a 13% reduction in EZ area based on a 7% annual decrease in EZ width,²⁰ and this predicted rate of decrease is comparable to the 14% to 16% annual percent decrease of EZ area estimated from Table 2 for the subgroups with middle-level baseline EZ area. 

Among the three OS metrics, 3D OS volume had a percent rate of progression comparable to that of the 2D EZ area but it was more than 6 times faster than one-dimensional (1D) OS thickness, which had an annual reduction of 1.5%. Although a 1D metric, such as EZ width as discussed in the previous paragraph, may show a smaller percent rate of change, the much slower percent progression rate for OS thickness is likely due to the limited range of OS thickness, as well as the axial resolution of OCT A-scans. Given an A-scan resolution of 3.87 µm/pixel for SPECTRALIS OCT and a mean baseline OS thickness around 20 µm among the patients with XLRP in this study, the mean OS thickness was only represented by 5 pixels. Thus, a shift of 1 pixel in the retinal layer boundary segmentation of EZ/pRPE would lead to a 20% change of OS thickness, which would result in a larger measurement variability to mask the actual change of OS thickness.²⁷ Hence, mean OS thickness alone may not be a good candidate as a biomarker. On the other hand, the EZ area contributes to OS volume the most and dictates the variability of OS volume measurements. Because of the large range of EZ areas (>60 mm² from 9-mm volume scans) and high-resolution of B-scans (∼5.8 µm/pixel), the impact of segmentation errors on the measurement of EZ area, and hence OS volume, is much smaller than that on OS thickness. Therefore, both EZ area and OS volume are better and more effective candidates than OS thickness as biomarkers to monitor disease progression in XLRP. The question is whether OS volume could offer something that EZ area could not, if, for example, a treatment increased the sensitivity of surviving photoreceptors with the increase of their outer segment length but not the area of preserved EZ. Although OS volume could be a potential new biomarker for assessing disease progression in XLRP, further studies are required to determine whether OS volume has any advantages over EZ area. 

From the imaging and mathematical points of view, the progressive loss of the EZ band and hence EZ area over time in RP is the opposite of the progressive increase of geographic atrophy (GA) area over time in age-related macular degeneration. It has been shown that the growth rate of the square root of GA area is independent of baseline GA size,³³ leading to the suggestion that using the square root transformation of GA area measurements could potentially eliminate the dependence of growth rates on baseline lesion.³⁴ In contrast, our results showed that the progression rate of the preserved EZ area in XLRP was dependent on its baseline value even after the square root transformation. Our finding is consistent with a previous study showing a weak relationship between loss of EZ width and the initial width of the EZ band in XLRP, where the greater the initial EZ band width, the greater annual loss of EZ band (R² = 0.16, P = 0.03).²⁰ A possible explanation for this difference may be due to the different patterns of disease progression for RP and GA. Another reason could be that, in our study, we had more cases of large EZ area (>10 mm²) when compared to the atrophy sizes reported in the GA studies.^33,34 As Feuer et al.³⁴ noted, very small and very large lesions are expected to grow more slowly in GA, as demonstrated in a more recent study that included more cases of large-size GA (up to 30 mm²) which showed that the square root transformation did not eliminate the dependence of the rate of progression of GA on baseline lesion size.³⁵ Hence, the square root transformation may not be applicable to datasets with an extended range of baseline values. 

On the other hand, the progression rate of 1D cube roots of OS volume was not associated with its baseline value, similar to the result that the change of OS thickness was not associated with its baseline value. This finding could be due to the reduced range of 1D measures after cube root transformation of OS volume (Fig. 4D), as well as the increased intersubject variability because of the larger variability of OS thickness measurements as shown in Figure 2A. It appears that the 1D cube root of OS volume behaves closer to OS thickness than the 1D square root of EZ area. Further studies may be needed with increased sample sizes and improved OS thickness measurements (for example, by employing ultra-high-resolution OCT scans) to address the apparent difference in dependence on baseline values between 1D measures from square root transformation of EZ area and from cube root transformation of OS volume. 

In this study, manual correction by human graders was performed on the automatic segmentation results of a deep learning model. Preliminary timing analysis revealed that the average times (SD) of two graders to examine and correct DLM segmentations of EZ and pRPE were 4.10 (2.04) minutes for a low-density (31-line) volume scan and 9.33 (1.76) minutes for a high-density (121-line) volume scan, suggesting that DL segmentation with manual correction can potentially be more efficient than traditional manual grading. On the other hand, the DL with manual correction segmentation method could be different from the conventional gold standard of manual grading. It has been demonstrated in our previous study, employing the same DLM as the one in this study, that there was a close agreement between the DLM only and the manual grading by a reading center for the EZ area measurements. Bland–Altman analyses showed similar findings for EZ area measurements in two studies, with a CoR of 1.83 mm² from this study compared to 1.62 mm² from the previous study. In addition, the performance comparison between the DLM only and the DLM with manual correction of human graders was comparable to that between two human graders. The manual correction method in this study may be different from the conventional method of human grading, and additional studies are needed to compare OS metrics measurement by the DLM and by different human grading methods, including the traditional manual segmentation approaches, to establish DLMs with manual correction as a potential new gold standard. However, our results suggest that automatic OS metric measurements by DLM are comparable to those obtained by manual grading. 

The comparison of the performance between human graders as well as between individual graders and the DLM revealed that intergrader variability as determined by CoR was higher than that for the DLM versus grader 1 for OS thickness measurement and to some degree for EZ area measurement, whereas intergrader variability was smaller than that between the DLM versus grader 2. It is worth pointing out that grader 2 was one of the main contributors to the generation of the training data (the ground truth) for the DLM, but grader 1 had no input to the training data for the DLM. Nevertheless, it appears that grader 1 and the DLM are closer to each other than grader 2 and the DLM. These results suggest that intragrader variability, as well as intergrader variability, may contribute to the difference between individual graders and the DLM. Because the DLM was trained with the data generated from human graders with intra- and intergrader variability, it may reflect an “average” performance of graders. 

In summary, the results from this study provide evidence to support using OS volume and EZ area as biomarkers to assess the progression of XLRP and as the outcome measures to evaluate treatment effects in clinical trials. Because the primary outcome measure of RP clinical trials is vision, it is also important to demonstrate the relationship between OS metrics and visual function, such as visual field sensitivity. Our preliminary results have shown that mean visual field sensitivity had much higher correlation coefficients with EZ area (r = 0.84) and OS volume (r = 0.84) than OS thickness (r = 0.45) in RP.³⁶ Given that their progression rates are dependent on their baseline values, the baseline values of OS metrics should be considered in the design and statistical analysis of future clinical trials for XLRP. Our results also suggest that DL may provide effective tools to significantly reduce the burden of human graders when analyzing OCT scan images and to facilitate the study of structure and function relationships,³⁶ in addition to the use of DL to assess disease progression, particularly with respect to future treatment trials for RP in general and XLRP in particular. 

Acknowledgments

Supported by a Foundation Fighting Blindness Individual Investigator Research Award and by a grant from the National Eye Institute, National Institutes of Health (EY09076). 

Disclosure: Y.-Z. Wang, None; K. Juroch, None; Y. Chen, None; G.-S. Ying, None; D.G. Birch, None