Test–Retest Variability of Fundus-Tracked Perimetry at the Peripapillary Region in Open Angle Glaucoma

Abstract

Purpose: To examine the association between the measured level and local gradient of visual sensitivity on the magnitude of test–retest variability of its measurements at the peripapillary region using fundus-tracked perimetry in eyes with glaucoma.

Methods: A total of 30 participants with open angle glaucoma underwent three examinations in one eye on fundus-tracked perimetry using a stimulus pattern that sampled the peripapillary region densely. Factors associated with the magnitude of test–retest variability at each location were examined.

Results: There was no significant change in average pointwise sensitivity (PWS) between tests 1 and 2 (P = 0.855), but a significant reduction between tests 2 and 3 (P < 0.001). Therefore, all subsequent analyses were performed only between tests 1 and 2. Multivariate analyses revealed that the magnitude of test–retest variability at a given location was significantly associated with its average sensitivity and gradient of sensitivity relative to the immediately adjacent locations (P ≤ 0.001), meaning that locations with low levels of sensitivity (4–18 dB) with low gradients of sensitivity (<2 dB/location) had a 90% test–retest limit of ±5.83 dB, compared to a limit of ±10.65 dB in areas of high gradients of sensitivity (>4 dB/location).

Conclusions: On a pointwise basis, the test–retest variability of visual sensitivity in glaucoma is not just related to its measured level, but also its local gradient when using fundus-tracked perimetry. Locations with low sensitivity do not necessarily demonstrate very high test–retest variability, depending on the local uniformity of visual field damage.

A major impediment to the clinical management and expeditious development of novel therapies for glaucoma is the lack of sufficiently sensitive clinical measures of visual function for this often slowly progressive disease. Based on current recommendations, up to six tests on static automated perimetry (SAP) over 2 years are required to reliably detect rapid disease progression,¹ because the large degree of test–retest variability with SAP means that the confidence interval can span up to the entire dynamic range in areas of reduced function for eyes with glaucoma.^2–4 This substantial variability poses difficulties when trying to determine the optimal management for patients with glaucoma in clinical settings. As a result, larger sample sizes and longer follow-up durations are required when evaluating new treatments for glaucoma. 

The large degree of test–retest variability with SAP, especially in areas of functional loss, has been attributed to disease-related changes in the response characteristics of the retinal ganglion cells (RGCs).^2,4–6 Whilst an earlier study suggested that fixation errors had a minimal influence on this degree of variability,⁷ other studies have suggested that the interaction between fixation shifts and the spatial variation in functional loss (such as the local gradient of sensitivity) contributes substantially to the variability observed.^8–12 Therefore, compensating for these fixational shifts could allow more precise sensitivity measurements to be obtained. Fundus-tracked perimetry (often called microperimetry) is a technique that can compensate for fixation shifts through real-time visualization of the fundus. Indeed, we have observed previously that the degree of test–retest variability was smaller using fundus-tracked perimetry¹³ compared to our previous study using static automated perimetry without fundus tracking in eyes with the early stages of age-related macular degeneration (AMD).¹⁴ 

Although fundus-tracked perimetry has been used increasingly in recent years to monitor disease progression to evaluate interventions in macular diseases,^15–18 its use in glaucoma has been limited to date. It has been used to evaluate the macular^19–21 and peripapillary region^22–24 in glaucomatous eyes, with the latter shown to detect functional losses that correlate with retinal nerve fiber layer (RNFL) losses measured with optical coherence tomography (OCT) imaging.^22,23 Some studies also have observed that such functional losses occurred in the absence of typical visual field defects measured using a more typical stimulus pattern that covered the central 30° radius with stimulus spacing of 6° (similar to the 30-2 stimulus pattern used on a Humphrey Field Analyzer; Carl Zeiss Meditec, Dublin, CA, USA).^24,25 Furthermore, visual field defects measured in the peripapillary region were also always present in eyes with established visual field defects from glaucoma.^24,25 Similar observations regarding the improved detection of functional losses near the peripapillary region compared to conventional visual field testing also have been made by other studies that used regionally condensed stimulus patterns for glaucoma eyes in areas of morphologic damage.^26–28 These observations point towards the potential clinical and research use of sampling the peripapillary region in eyes with glaucoma. 

Sampling the peripapillary region densely also provides the opportunity to account for the effect of local spatial variation in visual function when evaluating test–retest variability, since we have shown previously that measurement variability is increased at the border of deep scotomas,²⁹ and such deep scotomas are expected at the peripapillary region in eyes with glaucoma.^22–24 Sampling the peripapillary region is also advantageous since a similar region across all participants can be evaluated, which is difficult for regionally condensed stimulus patterns that target areas of known visual field losses, because the eccentricity of spatial loss can vary markedly between participants. 

Therefore, we sought to examine the test–retest variability of fundus-tracked perimetry at the peripapillary region in eyes with open angle glaucoma, to examine the association between the measured level of sensitivity and its local gradient on its magnitude of test–retest variability. 

Methods

This study was approved by the Human Research Ethics Committee of the Royal Victorian Eye and Ear Hospital (RVEEH) and was conducted in adherence with the Declaration of Helsinki, and informed consent was obtained from all participants. 

Participants

The inclusion criteria for an eye in this study included the diagnosis of glaucoma with open angles on gonioscopy examination, best-corrected visual acuity (BCVA) of 6/12 or better, spherical refractive error within ± 5.00 diopters (D) and cylindrical refractive error less than 3.00 D. All participants also were required to have performed SAP testing with a 24-2 Swedish interactive threshold algorithm (Carl Zeiss, Meditec, Inc.) in the study eye within 6 months, with ≤33% fixation losses and false-negative errors and ≤15% false-positive errors. All participants were required to be over 18 years of age, and diabetic participants without evidence of any retinopathy were included. The exclusion criteria for study eyes include a history of intraocular surgery (except uncomplicated cataract or glaucoma surgery performed ≥3 months ago) or glaucoma due to secondary causes (e.g., trauma). Participants also were excluded if they had any systemic or ocular disease that affected visual function (e.g., multiple sclerosis or AMD) or had any conditions that affected cognition (e.g., dementia, stroke). Participants also were excluded from this study if they were taking any medication known to affect visual function (e.g., hydroxychloroquine), or had any physical or mental impairment preventing them from participating in this study and/or providing informed consent. If both eyes met the eligibility criteria, the eye with the worse visual field mean deviation (MD) was chosen as the study eye. 

Procedures

All participants firstly performed standard measurements of BCVA on a 4-meter Early Treatment for Diabetic Retinopathy (ETDRS) chart, followed by fundus-tracked perimetry examinations before standard ophthalmic examination by a glaucoma specialist. 

Fundus-Tracked Perimetry Examinations

Fundus-tracked perimetry examinations were performed using the Macular Integrity Assessment (MAIA; CenterVue, Padova, Italy) device before any tests that could influence the ocular surface (such as applanation tonometry). The MAIA fundus-tracked perimeter performs fundus tracking using a line-scanning laser ophthalmoscope (SLO) that uses a super-luminescent diode illumination with a central wavelength of 850 nm. It uses the entire fundus as a reference when performing fundus tracking, capturing the fundus images at 25 frames per second. A red cross of 1.2° × 1.2° in size located 4.8° nasally from the center was used as the fixation target, and Goldman Size III stimuli (0.43°) were presented against a background with a luminance of 1.27 cd/m² using a 4-2 staircase threshold strategy. The maximum and minimum luminance of the stimuli was 318 and 1.37 cd/m², respectively, creating a dynamic range of 36 dB of differential contrast. 

To sample the peripapillary region in eyes with glaucoma at high density, we specifically designed a customized stimulus pattern that sampled 36 locations 6.1° radially from the center of the optic nerve head, encompassing the entire peripapillary region except the nasal quadrant (Fig. 1). As a result, the central angle between two neighboring stimuli relative to the center of the optic nerve head was 7.5° (meaning that there were four stimuli within each 30° clock hour), and the distance between two neighboring stimuli subtended a visual angle of 0.8°. Test reliability was assessed by measuring the frequency of responses to suprathreshold stimuli (of 10 dB) presented at the optic nerve head, manually located on the fundus image before the start of the examination. Such responses were considered false-positive errors, and any participant with false-positive errors of >25% on any of the tests was excluded from this study. This cutoff was chosen since the suprathreshold stimulus is presented once approximately every 1 minute, meaning that there typically were four to six presentations per examination (that typically took 4–6 minutes to complete). In this study, three examinations were performed (Tests 1–3) and all participants were given a few minutes of rest in between each examination to minimize the effect of fatigue. 

Figure 1

 

The peripapillary stimulus pattern used for fundus-tracked perimetry examinations in this study is shown on the right eye of a participant (top), where the test locations used to illustrate the parameters analyzed are indicated with a white arrow and numbers. (Bottom) The average sensitivity of the two tests are shown on this graph, with the test location being analyzed indicated by “x.” The average sensitivity of the two test locations one (“x + 1” and “x − 1”) and two positions (“x + 2” and “x − 2”) from the test location being analyzed (“x”) were used for the calculation of the gradient of sensitivity. Note: this graph also highlights that the separation between two stimuli with this stimulus pattern is 0.8°.

In this study, several parameters from the fundus-tracked perimetry examinations were determined to examine their association with the degree of test–retest variability for each pair of tests. At each test location (x), the average sensitivity of the two tests was determined. The gradient of sensitivity relative to the two test locations one and two positions adjacent – considered as x ± 1 and x ± 2, respectively – were calculated as follows (using the mean of the absolute difference in average sensitivity [AvgSen] of the test locations [x, x + 1, x − 1, x + 2 and x − 2]; Fig. 1):  

 

 

 

These parameters could not be determined for the first two and last two test locations (numbers 1, 2, 35, and 36) of the stimulus pattern used in this study; therefore, these locations were excluded from the corresponding analyses in this study.  

Fixation stability was ascertained by calculating the area of an ellipse that contains 95% of the fixation locations during the test as described previously.³⁰ This is referred to as the bivariate contour ellipse area (BCEA), which was calculated automatically by the fundus-tracked perimeter. The BCEA was logarithmically transformed (logBCEA) before analysis to ensure that the data were normally distributed.³¹ 

Statistical Analysis

A linear mixed effect model was used to evaluate the change in pointwise sensitivity (PWS) between tests, while accounting for the hierarchical (test locations within an eye), repeated-measure nature of the data.³² A compound symmetry correlation structure (specifying homogenous correlations between test locations) was used as the type of covariance matrix to account for within-eye correlations when analyzing multiple test locations within each eye. The test number was considered as a fixed effect, while test locations nested within an eye were considered as a random effect. The 90% test–retest limits of PWS were calculated using the mean absolute test–retest difference obtained using generalized estimating equation (GEE) models, also to account for the hierarchical nature of the data. An exchangeable correlation structure (specifying homogenous correlations between test locations) was used as the type of covariance matrix to account for within-eye correlations because multiple test locations were examined within each eye, and analyses were performed using a Tweedie distribution to account for the distribution of the absolute test–retest difference values. The 90% test–retest limits of PWS can be determined by multiplying the standard deviation (SD) of the measurement (which is the standard error of measurement ×Display Formula , when two measurements are obtained) by 1.645. From probability theory, the SD of measurement differences that follow a normal distribution and with a mean of zero can be calculated by multiplying the mean absolute test–retest difference by 2 / Display Formula .  

Generalized estimating equation models using the same parameters above were used to examine the factors associated with the absolute test–retest difference (representing variability). Average sensitivity (at location x) was considered as a categorical covariate grouped into 4-dB bins (and with values >28 dB included in the 28-dB bin due to small sample sizes) due to its nonlinear association with the extent of test–retest variability. The remaining factors – the gradient of sensitivity relative to the two test locations at one and two positions adjacent, age, and logBCEA – were considered as continuous covariates. The Wald χ² test was used to test the significance of each of the parameters firstly in univariate analyses, before combining the statistically significant parameters (P < 0.05) in a multivariate analysis. 

Results

A total of 30 participants were included in this study, and were on average 71.3 ± 11.6 years old (range, 39–91 years). All study eyes were diagnosed with open angle glaucoma and were under treatment, and had a median visual field MD of −3.84 dB (interquartile range [IQR], 1.69 to −9.91 dB) and PSD of 4.34 dB (2.58–8.20 dB). 

Test–Retest Variability Across Sequential Examinations

The distribution of test and retest values of the fundus-tracked perimetry examinations are shown in Figure 2, demonstrating qualitatively a slightly greater degree of test–retest variability for the first pair of tests compared to the second pair of tests. 

Figure 2

 

Distribution of the PWS values are plotted for (A) test 1 versus test 2, and (B) test 2 versus test 3. Scale bar: the number of overlapping points at each location for the graph.

A significant reduction in the average PWS was observed between tests 2 and 3 (−0.61 dB, 95% confidence interval [CI] = −0.39 to −0.83 dB; P < 0.001), but not between test 1 and 2 (0.02 dB, 95% CI = −0.20–0.24 dB; P = 0.855; Fig. 3A). This meant that the 90% test–retest limits could not be calculated for the comparison between tests 2 and 3. For tests 1 and 2, the 90% test–retest limits were calculated for each 2-dB bin from the average sensitivity of the two tests, demonstrating a characteristic peak in variability for areas of low sensitivity (particularly between the 4- and 12-dB bins; Fig. 3B).^33,34 

Figure 3

 

(A) Changes in average PWS over the three tests are shown, demonstrating a significant decrease in average PWS between tests 2 and 3 (*P < 0.001). (B) The 90% test–retest limits of individual locations grouped into 2 dB bins based on their average sensitivity when comparing tests 1 and 2 are shown, illustrating a characteristic peak in variability for areas of low sensitivity (between the 4- and 12-dB bins). Error bars: 95% CI of the mean estimate for both graphs.

Factors Associated With Test–Retest Variability

To investigate which factors were associated with the magnitude of test–retest variability, the absolute test–retest difference of PWS between tests 1 and 2 was used. At each test location (x), the factors examined included the average sensitivity and gradient of sensitivity relative to the two test locations one and two positions adjacent (x ± 1 and x ± 2, respectively). Age and fixation stability (captured by the logBCEA of the fixation locations) also were included as patient-level factors. Univariate analyses revealed that average sensitivity, gradient of sensitivity relative to the two test locations one and two positions adjacent, and age were all significantly associated with the magnitude of test–retest variability (P ≤ 0.011), but that the logBCEA of fixation locations was not (P = 0.086). Using the statistically significant factors from the univariate analyses in a multivariate analysis, the only factors that remained associated with the magnitude of test–retest variability at a given location was its average sensitivity and gradient of sensitivity relative to the immediately adjacent locations (P ≤ 0.001; see Table). 

Table.

 

Analysis of Factors Associated With Test–Retest Variability Using Univariate and Multivariate Generalized Estimating Equation Models

 

To illustrate the impact of the gradient of sensitivity of the two immediately adjacent locations on test–retest variability, the 90% test–retest limits were calculated separately for locations with a gradient of <2 (n = 495), 2 to 4 (n = 377), and >4 (n = 148) dB/location. The 90% test–retest limits were calculated based on empirical cut-offs to provide two groups that contained locations with an average sensitivity between 4 and 18 dB (excluding locations <4 dB/location because of a floor effect) and locations >18 dB. These cut-offs were chosen to provide an approximate dichotomization into locations of low sensitivity (4–18 dB) and locations of near-normal sensitivity (>18 dB), ensuring that there were enough samples in each group analyzed for statistical expediency. For example, few locations with low sensitivity were present in areas of low gradients (Fig. 4). 

Figure 4

 

The 90% test–retest limits calculated separately for different gradients of sensitivity (from the immediately adjacent locations; x ± 1), and separately for locations of low sensitivity (4–18 dB) and of near-normal sensitivity (>18 dB); the number of locations included in each calculation is shown at the bottom of each bar. This graph illustrates how locations in areas with larger gradients of sensitivity exhibit a greater degree of test–retest variability, but that locations of low sensitivity with low gradients of sensitivity can exhibit 90% test–retest limits of ±5.83 dB.

These calculations showed, for instance, that the 90% test–retest limits were ±5.83 and ±3.38 dB for locations of low and near-normal sensitivity, respectively, in areas of low gradients (<2 dB/location), but up to ±10.65 and ±6.13 dB for locations of low and near-normal sensitivity, respectively, in areas of high gradients (>4 dB/location). Therefore, the test–retest limits were 83% and 82% higher in areas of high compared to low gradients for low and near-normal sensitivity, respectively. 

Discussion

This study demonstrated that the test–retest variability of fundus-tracked perimetry was associated with the local gradient of sensitivity when using a stimulus pattern that sampled locations only 0.8° apart, and with the measured level of sensitivity in eyes with glaucoma. Areas with low measured levels and gradients of sensitivity had a 90% test–retest limit of ±5.83 dB, compared to a limit of ±10.65 dB in areas of high gradients of sensitivity. These findings showed that fundus-tracked perimetry testing of the peripapillary region in eyes with glaucoma is subject to a large degree of variability, but highlights how areas with low levels of sensitivity not characterized by steep local gradients can have a magnitude of variability that does not span the entire dynamic range. 

In this study, we observed a significant decrease in perimetric sensitivity between tests 2 and 3, which prohibited analysis of the magnitude of test–retest variability. Previous studies also have observed this decline in sensitivity in normal and glaucoma eyes over different conditions, and have attributed this to fatigue.^35–37 However, we were surprised with this finding given the similarity of conditions – including testing duration, number of tests performed, amount of rest given between tests, and type of fundus-tracked perimeter used – with our previous study in eyes with AMD, where we observed a significant improvement in sensitivity between tests 1 and 2 (attributed to a learning effect), and no significant changes between tests 2 and 3.¹³ It is possible that a learning effect occurred over the three examinations performed, but was offset by a fatigue effect between tests 2 and 3 in the previous study (this could have been determined by retesting the participants in the same manner on another visit). If this were the case, this would account for the drop in sensitivity between tests 2 and 3 in this study, since the participants with glaucoma in this study were unlikely to exhibit a significant learning effect given that all participants had prior experience with perimetric tests. 

Furthermore, this study demonstrated that the local gradient of sensitivity also contributed significantly to the magnitude of variability observed, independent of the level of sensitivity itself. This meant that in locations with low measured levels of sensitivity, the magnitude of variability was up to 83% higher in locations of high compared to low local gradients of sensitivity. This is consistent with our previous observations of an increased magnitude of variability at the border of the optic nerve head (which is characterized by a steep gradient of sensitivity) when examining normal subjects with the same fundus-tracked perimeter.²⁹ This increase in variability at locations with steep local gradients of sensitivity is most likely the result of incorrectly sampling neighboring locations over the duration of the stimulus presentation, that can occur with small eye movements between each frame tracked when using fundus-tracked perimetry (rate of 25 frames per second for the device used in this study). 

In this study, the characteristic increase in variability at locations within the peripapillary region with low levels of sensitivity measured using fundus-tracked perimetry was qualitatively consistent with observations from previous studies using SAP,^33,34 and the suggestions that this increased level of variability is attributed to disease-related changes in the response characteristics of the RGCs.^2,4–6 However, while the 90% test–retest limits of ±5.83 dB for locations with low levels of measured sensitivity in areas with a relatively uniform local gradient are substantially higher than the limits of ±3.38 dB observed for locations of relatively normal function, they do not span the entire dynamic range of the instrument, unlike previously reported for SAP for visual field sensitivities in this range.^2–4 In areas of high gradient, however, we similarly report very high test–retest variability for locations with low sensitivity (±10.65 dB). A strictly direct comparison of the magnitude of variability with SAP is not possible because of the different background luminance level of the fundus-tracked perimeter in this study compared to those in previous studies, and because of the different region of the retina sampled. However, we presume that the magnitude of variability for fundus-tracked perimetry is closer to a “best case” scenario, since measurements in areas with steeper gradients of function should be less influenced by eye movements using this technique compared to SAP. We were unable to establish whether this was the case directly due to the inability to disable the fundus-tracking device on the device. 

A standard research method for determining estimates of the variability of response to perimetric stimuli is to measure frequency-of-seeing curves (FOS).^2,5,37,38 Estimates of perimetric test–retest variability from FOS are used widely to model patient response variability in computer simulation studies that aim to develop new perimetric test or analysis methods. Recently, a study that measured FOS curves has suggested that perimetric test–retest variability is so high for sensitivities below approximately 15 to 19 dB when using SAP, that there is perhaps limited use in attempting to measure sensitivity in this range.³⁸ Our data suggest that the local gradient of sensitivity should influence the steepness of the FOS curve (i.e., the variability, where a steep slope represents less variability). Although many stimulus patterns used for visual field testing are too widely spaced (≥6° apart) to estimate the local sensitivity gradient, it may be useful to estimate this for stimulus patterns, such as the 10-2 (used on a Humphrey Field Analyzer, used increasingly to evaluate the macular region in glaucoma³⁹) where the stimuli are spaced only by 2° apart. The measurement of full frequency-of-seeing curves in areas of steep and shallow local visual field gradients while fundus tracking is used could shed light on the extent to which current models of response variability may need refinement. 

Accurate functional testing at the peripapillary region enabled by fundus-tracked perimetry could be useful for glaucoma, since functional losses detected in this region have been observed to be missed on the coarse grid stimulus patterns typically used on SAP testing,^25,26 and because it also provides a unique opportunity to combine structural and functional information.^22,23 However, future studies sampling the peripapillary region may benefit from using a fundus-tracked perimeter with a faster tracking speed, and potentially by using shorter stimulus durations and larger stimuli as well. Combining these changes with an increased density of sampling locations, application of Gaussian filtering to the measurements,²⁶ and seeding the test procedure with structural information⁴⁰ could collectively reduce the magnitude of variability when sampling the peripapillary region. Future studies also are required to further explore the retinotopic mapping of the RGCs in the peripapillary region to better understand the relationship between structure and function in this region, and also whether the gradients in sensitivity observed in this study correlate with the gradients in structural parameters. 

In conclusion, this study revealed that the magnitude of test–retest variability of a given location at the peripapillary region of glaucoma eyes was associated with the measured level of sensitivity and its local gradient when assessed with fundus-tracked perimetry. We demonstrated that areas with low levels of sensitivity not characterized by steep local gradients have a magnitude of variability that does not span the entire dynamic range. In addition, our data suggest that models of test–retest variability used for the purposes of developing analytical methods for detecting progression or developing improvements to visual field procedures may benefit from consideration of local gradients within the visual field, in addition to pointwise level of sensitivity alone. 

Acknowledgments

Supported by The Ophthalmic Research Institute of Australia, The Menzies Foundation, the National Health & Medical Research Council (Early Career Fellowship #1104985, ZW), the Miller Foundation, and the Dorothy Adele Edols Charitable Trust. The Centre for Eye Research Australia receives support for operational infrastructure from the Victorian government. 

Disclosure: Z. Wu, None; A.M. McKendrick, Heidelberg Engineering GmBH (F), Haag-Streit AG (F, R), CenterVue SpA (F); Carl-Zeiss Meditec (R); X. Hadoux, None; J.C. Fan Gaskin, None; G.S. Ang, None; M.G. Sarossy, None; J.G. Crowston, None