Test–Retest Repeatability of Microperimetry at the Border of Deep Scotomas

Abstract

Purpose.: The purpose of this study was to examine the test–retest repeatability of microperimetric sensitivity at the border of deep scotomas.

Methods.: Thirty normal participants underwent two examinations, each on the Macular Integrity Assessment (MAIA) microperimeter and on the MP-1 microperimeter (four examinations in total). A customized stimulus pattern allowed microperimetric sensitivity to be measured at the border of the optic nerve head (ONH), which acted as a model for the border of a deep scotoma—and also at the macular and peripapillary region.

Results.: There were no significant changes in average point-wise sensitivity (PWS) values between the two examinations for all three regions using the MAIA microperimeter (P ≥ 0.262). The PWS coefficient of repeatability (CoR) was ±12.99 dB at the border of the ONH, which was significantly larger than points in the macular and peripapillary regions (P > 0.001). A significant decrease in average PWS, using the MP-1 microperimeter at the macular and peripapillary region (P < 0.001), meant that the PWS CoR could not be determined in these regions. No significant changes in average PWS were observed at the border of the ONH (P = 0.223), and the PWS CoR was ±7.52 dB in this region.

Conclusions.: Microperimetric test–retest repeatability at the border of a deep scotoma was worse than at other areas of normal retina, and this highlights the limitation of applying a single estimate of test–retest repeatability to determine whether significant functional decline has occurred at the border of a deep scotoma.

Fundus-tracked perimetry (often termed “microperimetry”) is a method of assessing sensitivity to luminance increments in the macular region, and it has been used increasingly in recent years due to advances in technology that have allowed modern instruments to accurately and automatically assess individual retinal locations with high precision. Microperimetry has been particularly useful for monitoring subtle longitudinal changes in disease severity and in response to treatment for slowly progressive conditions, including geographic atrophy caused by age-related macular degeneration (AMD),^1–4 macular telangiectasia,^5,6 and other hereditary diseases of retinal degeneration.^7–9 

For these slowly progressive conditions, many treatments being developed currently seek to prevent or slow the expansion of the degenerative or atrophic changes into adjacent areas of healthier retina. Thus, an expeditious evaluation of treatment efficacy might be achieved through assessing the healthier retina at the border of these pathological changes by using tools such as microperimetry, as this area is the first to be affected by disease progression. However, previous studies using standard automated perimetry have suggested that functional measurements in these areas (characterized by having a steep gradient in sensitivity) exhibit a large degree of variability.^10–12 More recently, a study using microperimetry suggested that the test–retest repeatability of its measurements were similar at the borders of degenerative changes, at the physiological blind spot (both of which are also characterized by having a steep gradient in sensitivity), and at other areas of retina.⁸ However, the spatial averaging applied to the data and limited dynamic range of the microperimeter used may have underestimated the differences in variability among these regions. Given the importance of measuring functional changes at the borders of these pathological changes, it is crucial to further examine microperimetry test–retest repeatability carefully. 

In this study, we sought to examine the test–retest repeatability at the border of the optic nerve head (ONH; the physiological blind spot) in normal participants as a model for the border of a deep scotoma, as the peripapillary retina is normal and is not characterized by a reduction in photoreceptor density,¹³ whereas there are no photoreceptors at the ONH. We performed this test by using two different microperimeters that differed in their spatial resolution of fundus tracking and stimulus dynamic range. 

Methods

This study was approved by the Human Research Ethics Committee of the Royal Victorian Eye and Ear Hospital (RVEEH) and was conducted in adherence with the Declaration of Helsinki. All participants provided written informed consent after an explanation of all test procedures. 

Participants

Normal participants over 18 years of age were recruited from the staff of the RVEEH and Centre for Eye Research Australia. Exclusion criteria for any participant included the presence of any ocular pathology (in the anterior or posterior segment), significant cataracts, amblyopia, or peripapillary atrophy caused by ocular pathology (such as glaucoma or pathological myopia); participants were allowed to have some peripapillary atrophy, as the absence of retinal pigment epithelium and photoreceptors in this region also represents an area of deep scotoma. Participants were also excluded if they had diabetes or any neurological or systemic disease affecting vision, if they were taking any medication known to affect retinal function (e.g., hydroxychloroquine), if they had any physical and/or mental impairment preventing them from participating in this study, or if they were unable to or did not provide written informed consent. The right eye was chosen as the study eye for all participants. 

Procedures

Spectral-domain optical coherence tomography (SD-OCT) line scans were first performed to exclude any ocular pathology, followed by microperimetric examinations. Spectral-domain OCT line scans were performed using a Spectralis HRA+OCT unit (Heidelberg Engineering, Heidelberg, Germany), using a setting involving 37 B-scans that covered a 30° × 15° area, set on high-resolution imaging mode and averaging 15 frames for each B-scan. 

Microperimetry Examination

Microperimetry examinations were performed using the Macular Integrity Assessment (MAIA; CenterVue, Padova, Italy) microperimeter and the MP-1 (Nidek Technologies, Inc., Padova, Italy) microperimeter without pupillary dilation. Identical verbal instructions were then given to all participants regarding how to perform in the microperimetry test. 

The MAIA microperimeter tracks the fundus at a rate of 25 frames per second, using the entire fundus as a reference. Visualization of the fundus was achieved using a line-scanning laser ophthalmoscope (SLO), with superluminescent diode illumination that has a central wavelength of 850 nm. The fixation target was a red ring of 1° diameter, and achromatic Goldman III stimuli were presented for 200 ms against a background of 1.27 cd/m², using a 4-2 staircase threshold strategy. The maximum and minimum luminance values of the stimulus were 318 cd/m² and 1.35 cd/m², respectively, creating a dynamic range of 36 dB of differential contrast. 

The MP-1 microperimeter also tracks the fundus at a rate of 25 frames per second, using a single reference landmark selected by the examiner on an infrared fundus image captured by a fundus camera with an image resolution of 768 × 576 pixels. The fixation target was a red ring of 1° diameter, and achromatic Goldman III stimuli were presented against a background of 1.27 cd/m² for 200 ms, using a 4-2 staircase threshold strategy. The maximum and minimum luminance values of the stimuli were 127 cd/m² and 2.54 cd/m², respectively, creating a dynamic range of 20 dB of differential contrast. 

In this study, we sought to design a stimulus pattern that would allow test–retest repeatability at the border of a deep scotoma to be carefully examined, choosing to use the physiological blind spot because it is a known area with a deep scotoma border that is not influenced by different disease states. First, we designed a movable customized stimulus pattern to be used for a practice examination, consisting of a total of 16 points, with nine points within the macular region to allow participants an opportunity to familiarize themselves with the testing procedure of microperimetry in a region that was easier to detect, and seven points along a horizontal axis near the ONH, spaced 0.5° apart to allow the border of the deep scotoma to be densely sampled (Fig. 1A). We then designed another moveable, customized stimulus grid for the study examinations, consisting of a total of 18 points, 14 points near the ONH (arranged as two horizontal rows of seven points, with a horizontal spacing of 0.5° between the points and a 0.5° spacing between the two rows in order to have a greater number of points in this region) and four points still within the macular region to encourage fixation and visual attention at another location (Fig. 1B). Note that all points within the stimulus pattern moved to the same degree when the stimulus pattern was moved, because the stimulus pattern was fixed. 

Figure 1

 

Microperimetry stimulus patterns used in this study, drawn to scale. (A) For the practice examination, a movable, customized stimulus pattern was designed and placed at a location where the series of horizontal points near the ONH had approximately two points within the ONH (white arrows) and one point at its border (black arrow). (B) For the formal study examination, another movable, customized stimulus pattern was designed and placed at a location where the two rows of horizontal points near the ONH had approximately four points within the ONH (white arrows) and two points at its border (black arrow).

In this study, participants were randomly allocated to perform examinations with either the MAIA or the MP-1 microperimeter before examinations with the other unit, in order to avoid the potential bias of starting on one of the microperimeters. A practice examination was first performed with the microperimeter allocated, and the movable practice stimulus pattern was placed, using the infrared image as a reference, into an area where the series of horizontal points were positioned approximately halfway between the vertical margins of the ONH and having approximately two points within the ONH (horizontally) and one point at its border (Fig. 1A). Two formal study examinations were then performed using the same microperimeter, using the results of the practice examination to assist the initial placement of the study stimulus pattern by observing where the border of the deep scotoma occurred. The study stimulus pattern was also placed approximately halfway between the vertical margins of the ONH, like the practice examination, with approximately four points within the ONH and two points at the border (Fig. 1B). Another two study examinations were then performed on the other microperimeter using the same procedure. The placement of the stimulus patterns is illustrated in Supplementary Clip S1 for ease of conceptualization. 

Due to the difficulty of having exactly four points within the ONH, examinations with ≥2 and ≤6 points within the ONH were accepted; if not, the protocol required a repeat examination (no repeat examinations were required in this cohort). The border of the ONH for each participant was then determined by averaging the sensitivity for points at the same eccentricity (horizontal distance from the points at the macula among the 14 points near the ONH) from the two examinations (total of four points, consisting of two points from each of the examinations) of each microperimeter. The border was then defined as the eccentricity with the greatest difference in average sensitivity between its two adjacent eccentricities, as illustrated in Figure 2 and Supplementary Clip S1. The six points immediately temporal to the border (within 1.5°) were considered to be in the peripapillary region. 

Figure 2

 

Determination of the border of the ONH. The average sensitivity of points at the ONH at each eccentricity (points of the same horizontal distance from the macular points) for the two microperimetry examinations (left) was first determined (plotted at top right). The eccentricity determined to have the greatest difference in average sensitivity between its two adjacent eccentricities (or the highest gradient of sensitivity, plotted at bottom right [arrow]) was considered the border of the ONH (top right [arrow]). The six points immediately temporal to the border were considered the peripapillary region.

The frequency of false-positive responses, indicated by the frequency of response to suprathreshold stimuli at the physiological blind spot (manually located before the presentation of the first stimuli) was used to provide an index of test reliability. Because a false-positive stimulus is presented approximately once every minute for the microperimetry examinations, only two to three false-positive stimuli were typically presented for the examinations in this study because they took approximately 3 minutes in duration, on average. Thus, any examination with greater than one false-positive response was discarded and repeated. 

Statistical Analysis

Changes in average point-wise sensitivity (PWS) in each region between the two microperimetry tests were determined using a linear mixed effects model,¹⁴ considering test number as the fixed effect and stimulus points nested within participants as the random effect. Bland-Altman plots were then used to inspect the test–retest characteristics for the points at the macular and peripapillary regions and the border of the ONH (but not within the ONH because the test–retest difference does not exhibit a normal distribution due to the floor effect of having zero sensitivity in this region); the test–retest coefficients of repeatability (CoR)¹⁵ were then calculated for regions that did not exhibit a significant systematic change in average PWS. The CoR for PWS were then compared between different regions using a Kruskal-Wallis test with post hoc Bonferroni corrections, because the distribution of the PWS CoR was not normally distributed (checked using a Shapiro-Wilk test). All statistical analyses were performed using commercially available statistical software (SPSS version 21 software; IBM, Armonk, New York, NY, USA). 

Results

A total of 30 normal participants were included in this study and were, on average, 38.4 ± 10.9 years of age (range, 22–67 years of age). 

Findings From the MAIA Microperimeter

Examining the changes in average PWS between the two examinations at different regions, we observed no significant changes at the macular and peripapillary region and at the border of the ONH (P ≥ 0.262) (Fig. 3). 

Figure 3

 

Changes in average PWS between the two microperimetry examinations within the same session, showing no significant changes for points overlying areas of normal retina at the macular and peripapillary region, and also for points at the border of the ONH.

Bland-Altman plots were used to inspect the test–retest characteristics of all microperimetric points, as shown in Figure 4, and revealed a large degree of test–retest variability for points at the border of the ONH; the remaining points appeared to exhibit a smaller degree of test–retest variability. 

Figure 4

 

Bland-Altman plot of PWS, using the MAIA microperimeter. The horizontal dashed lines representing upper (+2 SD) and lower (−2 SD) limits of 95% of the mean, from top to bottom, respectively, and the horizontal solid line represents the mean. Individual points are color coded to represent the location where they were measured.

The PWS CoR were then calculated for points at the macular and peripapillary regions and at the border of the ONH and were, on average, ±3.81 dB, ± 4.50 dB, and ±12.99 dB respectively. The PWS CoR for points at the border of the ONH were significantly larger than those for points at the macular and peripapillary regions (P > 0.001) but not significantly different between points at the macular and peripapillary regions (P = 1.000). The distribution of the PWS CoR for each region is shown in Figure 5. To account for the differences in number of points used to calculate the CoR in each region, the analysis was also performed for each eccentricity (with two points each) separately. The PWS CoR was still significantly higher at the border of the ONH than at all other eccentricities at the macular and peripapillary regions (P ≤ 0.01) but was not significantly different for all pairwise comparisons between all the other eccentricities at the macular and peripapillary regions (P ≥ 0.182) (Supplementary Fig. S1). 

Figure 5

 

Boxplots show PWS coefficients of repeatability for points at different locations using the MAIA microperimeter. Each boxplot includes the maximum (upper whisker, excluding outliers, represented by black dots), upper quartile (top of box), median (horizontal line in box), lower quartile (bottom of box), and minimum (lower whisker). *Statistically significant difference between locations at P < 0.05. These findings indicate that test–retest variability was much higher at the border of a deep scotoma (in this case, at the ONH) than in areas of normal retina.

Findings From the MP-1 Microperimeter

Examination of the changes in average PWS between the two examinations at different regions revealed there was a significant decrease in the PWS at the macular and peripapillary regions (P < 0.001) but no significant changes at the border of the ONH (P = 0.223) (Fig. 6). Due to this systematic change in PWS, the PWS CoR could only be calculated for the points at the border of the ONH (which exhibited a normal distribution, checked with the Shapiro-Wilk test; P = 0.234). The CoR at the border of the ONH was ±7.52 dB. 

Figure 6

 

Changes in average PWS between the two examinations within the same session, using the MP-1 microperimeter, showing a significant decrease for points overlying areas of normal retina at the macular and peripapillary regions (*P < 0.001) but no significant changes for points at the border of the ONH (P = 0.223).

Discussion

In this study, we found that the test–retest repeatability was worse at the border of deep scotomas (using the ONH in normal participants as a model for this) than at areas of normal retina, using the MAIA microperimeter with a larger stimulus dynamic range. These findings suggest that it would be inappropriate to use a single estimate of test–retest repeatability for both the border of a deep scotoma and the other retinal regions when attempting to determine whether a significant change in visual function has occurred with disease progression using microperimetry. 

Using the MP-1 microperimeter for the same participants in this study, we were unable to compare test–retest repeatability at the border of deep scotomas with the adjacent area of normal retina because there was a significant decrease in microperimetric sensitivity between the two examinations. This decrease is most likely attributed to the limited dynamic range of this microperimeter, where the sensitivity measured in these normal areas often reached a ceiling effect, and could thus only either stay the same or decrease on the repeated examination. In other areas, where the sensitivity measured was near the ceiling of 20 dB, a negative bias would also likely be present because the ceiling effect limited the full potential of a positive change on retest. For example, a point measured at 18 dB can only exhibit a positive improvement of +2 dB but has the potential to exhibit a change of ≤−2 dB. It is also unlikely that this decrease was due to a fatigue effect, because the examinations were short, and a fatigue effect should have had an influence on all regions tested, rather than on just the regions where a ceiling effect was present. Such a decline in sensitivity between examinations was not reported in a previous study using the MP-1 microperimeter.¹⁶ However, it is not known in that previous study whether the “follow-up” option was used, which would set the initial intensity of the stimulus at each point on retest based on the threshold obtained on the first test. When this option is not selected, the initial stimuli will be presented at the same intensity (set at a default of 16 dB for the MP-1) for both the first and the second examination, thus reducing the influence of the ceiling effect. Regardless, we found that the CoR of PWS at the border of the ONH was ±7.52 dB in this study (without excluding floor and ceiling effects), which is higher than those reported previously in the macular region (central 10° radius) of older participants with macular disease where the CoR of PWS was ±5.56 dB and ±4.94 dB, with and without excluding floor and ceiling effects, respectively.¹⁷ 

Our findings of poorer test–retest repeatability using the MAIA microperimeter are not consistent with those of a recent study that found the test–retest repeatability of microperimetry similar at both the border of degenerative changes or ONH and other areas of retina in eyes with hereditary retinal degeneration.⁸ Although there are several differences in the testing procedures used (including the chromaticity of the stimuli and background and stimulus pattern), a key difference was the use of a three-point spatial moving average to the microperimetric data, and the study reported a PWS CoR of ±4.21 dB that was applicable across all locations.⁸ Applying the same type of spatial averaging to our data, we found a similar PWS CoR of ±4.65 dB that was similar across all locations. It is important to note that such averaging can obscure the changes that occur with disease progression, especially at the border of a deep scotoma where spatial averaging will include both a point inside the atrophic or degenerative change and a point outside sampling an area of healthier retina, both of which are less likely to progress as rapidly as the border of these changes. 

The PWS CoR of ±3.81 dB in the macular region of the normal participants in this study, using the newer MAIA microperimeter, was also consistent with that of our previous study, where the PWS CoR was ±3.74 dB at the macular region for older normal participants.¹⁸ The PWS CoR at the border of deep scotomas was ±12.99 dB in this study, which was also higher than those with AMD in our previous study, where the PWS CoR for points at the macular region was ±4.37 dB or less.¹⁸ Although the PWS CoR was higher (indicating greater test–retest variability) using the MAIA unit than the MP-1 microperimeter in this study, it is likely that the smaller dynamic range of the MP-1 may have limited the full extent of the test–retest variability that could have occurred. 

The increased limits of the test–retest repeatability observed at the border of a deep scotoma is most likely due to a combination of the limits of the fundus-tracking systems used in the currently commercially available microperimeters and stimulus parameters such as its size and presentation duration. With the imaging systems of the microperimeter tracking the fundus at a rate of 25 frames per second, small degrees of eye movements that occur during the presentation of the stimulus (200 ms) can result in a neighboring retinal location being sampled during the gap between each frame tracked. Therefore, the test–retest repeatability may differ depending on the fixation stability of the participant and requires further detailed examination. However, this finding still highlights the potential limitation of applying the same limits of test–retest repeatability at the border of a deep scotoma. These findings have important implications for clinical studies using microperimetry for monitoring longitudinal changes and response to treatment for slowly progressive conditions, although care must be taken when generalizing these findings to such conditions where the gradient of sensitivity may differ from those examined in this study. 

In summary, this study showed that the test–retest repeatability at the border of deep scotomas was worse than that in other areas of normal retinas, highlighting the potential limitation of applying a single estimate of test–retest repeatability to determine whether a significant change has occurred in these regions. These findings are important to consider when measuring functional decline at the border of a deep scotoma. 

Acknowledgments

Supported by National Health and Medical Research Council Project Grant 1027624 and a University of Melbourne Early Career Grant. Centre for Eye Research Australia receives operational infrastructure support from the Victoria government and is supported by NHMRC Centre for Clinical Research Excellence Award 529923. 

Disclosure: Z. Wu, None; C.J. Jung, None; L.N. Ayton, None; C.D. Luu, None; R.H. Guymer, None