Test–Retest Reliability of Clinical Shack-Hartmann Measurements

Abstract

purpose. To evaluate the stability of clinical monochromatic aberrometry measurements over a wide range of time scales.

methods. Monochromatic aberrations in four normal eyes were measured with a clinical Shack-Hartmann aberrometer. A chin rest or a supplemental bite bar attachment was used to stabilize head and eye position. Five repeated measurements were taken within one test (5 frames, t < 1 second) without realignment. With realignment between each measurement, aberration measurements were repeated five times (t < 1 hour) on each day, at the same time of day on five consecutive days, and again on 5 days at monthly intervals. A control experiment studied the effect of systematically misaligning the eye to determine whether fixation errors can account for the variation in the repeated measurements.

results. Variability of wavefront root mean square (RMS) error (excluding defocus and astigmatism) was tracked across repeated measurements. Variances for different time scales were: 8.10 × 10⁻⁵ μm² (t < 1 second), 3.24 × 10⁻⁴ μm² (t < 1 hour), 4.41 × 10⁻⁴ μm² (t < 1 week), 9.73 × 10⁻⁴ μm² (t < 1 year). Bite bar and chin rest data were almost identical. Rotational fixation error up to 3° accounts for only part of the variability.

conclusions. Increased variability in aberration maps between days and months indicates biological fluctuations that are large enough to prevent achievement of “perfect vision,” even in the unlikely event that spherical and astigmatic refractive errors are corrected perfectly. However, lack of stability does not justify withholding treatment. A lasting benefit of aberration correction is expected despite temporal variability.

The ubiquity and historical success of optical corrections for the eye’s lower-order aberrations of defocus and astigmatism demonstrates that lower-order aberrations are reasonably stable over time. Otherwise, there would be little value in prescribing spectacle lenses if, for example, a given eye fluctuated between myopia and hyperopia on a daily basis. This experience suggests that for refractive correction to be effective, any variability in the eye’s refractive errors must be small relative to the mean. The same principle applies to the correction of higher-order aberrations with custom contact lenses ^{1 2 3 4} or wavefront-guided corneal ablation. ^{5 6 7 8 9 10} The success of these new technologies depends on the stability of the eye’s higher-order aberrations. By analogy with the example just given, correcting an eye with positive spherical aberration would be of little value and indeed would be detrimental, if the next day the eye had negative spherical aberration. 

Longitudinal studies in adults confirm that lower-order aberrations are stable over time. ¹¹ Even in eyes that are changing due to progression of myopia, the rate of change is low (e.g., mean change up to approximately −0.50 D per year, ^{12 13} with an SD of 0.25 D ^{14 15} ). In contrast to this picture of longitudinal stability, test–retest studies of refractive error indicate a substantial degree of variability in the measurement of refractive error. ^{16 17 18 19 20} For example, repeated measures of spherical equivalent on the same eye by different clinicians can differ by up to ±0.75 D (95% limit of agreement). ^{16 17} A similar degree of variability is found in repeated measures of astigmatism (e.g., 95% limit of agreement for cylinder power = ±0.80 D ¹⁷ ). Repeated subjective refractions by the same clinician can also vary by almost the same amount. ¹⁶ Objective refractions with autorefractors tend to be less variable, ^{16 17} but there is some question regarding their accuracy. ²⁰ This variability in the measured sphere and astigmatism is thought to reflect measurement error, rather than variability in the eye’s refraction, because stabilizing accommodation, which is the largest potential source of variability in spherical refractive error, increases test–retest variance. ^{16 21}  

Higher-order aberrations of the eye are generally much smaller than defocus and astigmatism. ^{22 23 24} For example, for small and large pupil sizes, the wavefront variance (WFV) generated by higher-order monochromatic aberrations (third order up to seventh order) is less than would be generated by 0.25 D of defocus. ²⁴ Consequently, effective correction of higher-order aberrations requires a higher level of measurement accuracy and repeatability than the current standard of care for lower-order aberrations. Efficacy also requires that biological variability in the eye’s higher-order aberrations be significantly smaller than the mean level of aberration, which is already very small. In short, the eye must be stable and measurements must be reliable to assure the success of higher-order corrections. 

In a companion study, ²⁵ we examined the accuracy and stability of a clinical aberrometer based on a Shack-Hartmann wavefront sensor for measuring higher-order monochromatic aberrations of a series of model eyes. We found that measurement accuracy exceeded the manufacturing tolerances of the model eyes we used to perform the tests. Furthermore, test–retest variance was extremely small (e.g., the 95% probability range of five repeated measurements of higher-order aberrations for a single alignment was ±0.002 D and ±0.005 D for multiple alignments). Using this same instrument, we examined test–retest variability of higher-order aberrations in human eyes. The purpose of this report is to describe the stability of these higher-order aberrations over time scales of seconds, minutes, days, and months. 

Methods

Apparatus

Aberrations were measured with a commercial aberrometer (Complete Ophthalmic Analysis System [COAS] manufactured in 2000; Wavefront Sciences, Inc., Albuquerque, NM). COAS measures the aberrations of the eye by the Shack-Hartmann wavefront sensing technique. The light source is a superluminescent diode that emits 850 nm near-infrared radiation. The wavefront sensor uses a 44 × 33 array of lenslets that are each 144 μm in diameter. Our instrument has a custom-designed magnification factor of 0.5, which means the lenslet array samples the exiting wavefront every 288 μm in the pupil plane. The purpose of this customized design is to allow us to measure aberrations in a pupil with a diameter as large as 9 mm, as needed in our other studies, rather than a maximum of 7.2 mm as do all the other COASs in the market. The aberrometer provides a detailed map of wavefront error over the extent of the eye’s pupil. The map is then subjected to Zernike analysis that yields a spectrum of aberration coefficients. ^{26 27} We have confirmed, in a series of single-surface aspherical model eyes, that the COAS provides extremely accurate measures of lower- and higher-order aberrations over a larger range than we encountered in the present study. ²⁵ All the measurements were taken in a temperature- and humidity-controlled environment at the Borish Center for Ophthalmic Research at the Indiana University School of Optometry. 

Experimental Procedure

Two clinicians with over 1 year of experience with the aberrometer performed all the measurements. Monochromatic aberrations were measured on the right eyes of four adults with normal optics (three with nearly emmetropic vision, and one with 3.50 D of myopia, all with low levels of astigmatism) ranging in age from 17 to 54 years. Alignment of the COAS measurement axis with the eye’s primary line of sight was achieved by asking the subject to fixate the center of the instrument’s fixation target (a bull’s eye pattern of rays and rings), whereas the operator aligned the pupil with the instrument axis. Axial conjugation between the eyes’ pupil plane and the instrument lenslet array was achieved by focusing the iris and first Purkinje images. After alignment, the room lights were dimmed and measurements were taken once the pupil had dilated to 6 mm. This delay between alignment and test varied from several seconds up to almost 1 minute and was necessary to obtain measurements over a moderately large pupil (6 mm in our case) without the use of mydriatic drugs. Subjects were instructed to remain stationary, by using the head and chin rest provided, and to continue to fixate the target while maintaining normal blinking. Immediately before each measurement, subjects were instructed to blink and then hold their eyes open. Aberrations were taken approximately 1 second after the final blink. COAS-generated wavefront error maps and Zernike coefficients were downloaded for off-line analysis using our own software written in a commercial program (MatLab; The MathWorks, Inc., Natick, MA). All aberration coefficients were calculated for a 6-mm diameter pupil and are reported according to Optical Society of America (OSA) recommended standards. ²⁷  

Five repeated measurements were taken over four different time scales—second, hour, week, and year—representing different orders of magnitude of temporal duration. For the time scale of a second, the instrument was configured to collect a burst of five measurements separated by 0.1 second to produce a five-frame movie. For the hour time scale, five-frame movies were taken five times during a single experimental session that lasted approximately 15 to 20 minutes. Between measurements the subject exited the instrument and on reentering was realigned. For the week time scale, five single measurements were taken at the same time of day (3–4 PM) 5 days in a row. For the year time scale, measurements were taken at the same time of day (3–4 PM) on five occasions separated by approximately 1 month over a 5-month period. One subject (PK) did not participate in this final experiment. Because there are 3 × 10⁷ seconds in 1 year, this experimental design allowed us to study temporal variability of individual eyes over seven orders of magnitude of time. Initial concerns about the effects of head movements prompted us to supplement some of the measurements made using the manufacturer’s head and chin rest with a second set of data obtained with a rigid bite bar for head restraint. 

In addition to the main experiment, we investigated the effect of systematically misaligning the eye by rotating the direction of gaze from the center of the fixation bull’s-eye pattern to each of the four surrounding rings (eccentricities of ±0.75°, ±1.5°, ±2.25°, and ±3°) along each of the eight primary meridians (0–360° in 45° steps). Initially, the subjects were asked to fixate the center of the fixation target while the operator aligned the instrument with the eye, as described. Once aligned, subjects were instructed to fixate one of the four rings along a certain meridian and the measurement was taken without realigning the instrument. Thus, along each meridian, nine positions were measured, with the center being measured twice. This procedure was repeated on all eight meridians to produce a total of 32 extrafoveal measurements within the central 6° of visual field. The whole process was repeated three times with each subject. Effects of axial and transverse misalignments are reported elsewhere. ²⁵  

All data were collected after informed consent was obtained from the subjects, and the study protocol was in accordance with the tenets of the Declaration of Helsinki. 

Data Analysis

One-way analysis of variance (ANOVA; P < 0.05 significance level) was used to compare the variances of aberration measurements for each subject at different time scales, and to establish whether variability increased over time. We also used ANOVA to compare aberrations measured with the two restraint methods (bite bar and chin rest). The overall variances at each time scale were calculated across subjects as follows. On the time scale of seconds, the between-frame variance was calculated for the five measurements taken within 1 second for each subject. We then averaged these variances across the four subjects and across the five repetitions made over 1 hour. On the hour time scale, the between-trial variance was calculated for five individual frames, one from each of five consecutive trials (e.g., the first frame in each trial). We then averaged these variances across the four subjects and across the five frames on each trial. On the week time scale, the between-day variances were calculated for individual trials across the 5 days for each subject (e.g., first trial on each day). We then averaged these variances across the four subjects and across the five trials on each day. On the year time scale, the between-month variance was calculated for individual trials across the 5 months for each subject (e.g., first trial in each month). We then averaged these variances across the three subjects and across the five trials in each month. 

The magnitude of aberrations of the eye was represented by wavefront variance (WFV) in square micrometers, which is calculated as the sum of squared Zernike coefficients. The root mean square (RMS) wavefront error is simply the square root of the wavefront variance. In this article, we refer to higher-order RMS wavefront errors as being the square root of the sum of squared third- and fourth-order Zernike coefficients. To provide a familiar metric for the magnitude of ocular aberrations, we converted wavefront RMS from micrometers to equivalent defocus (in diopters). Equivalent defocus is the amount of defocus needed to produce the amount of wavefront RMS obtained from higher-order aberrations. The relationship between RMS and equivalent defocus (Me) is calculated with the following equation: Me = 4√3·RMS/r ², in which r is the pupil radius in millimeters. ²⁴ As with RMS wavefront error, equivalent defocus quantifies only the amount of aberration in the eye and not its visual impact (Cheng X, et al. IOVS 2003;44:ARVO E-Abstract 2123; Applegate RA, et al. IOVS 2003;44:ARVO E-Abstract 2124). ^{28 29}  

Results

Variability of Aberration Maps over Different Time Scales

Figure 1 shows examples of five contour maps for higher-order aberrations measured at each of four different time scales in the same eye (CT). Even casual observation shows that the aberration maps were virtually identical when the five repeated measures were taken within 1 second. Although every map shows a noticeable similarity, as one might expect from repeated measurement of the same eye, the variability between repeated measures obviously increased as the time scale increases. One way to quantify this variability would be to compute the point-by-point mean and standard deviations of the five maps displayed in Figure 1 for each of the four time scales. We adopted this approach for the data collected over 1 second, and the results are shown by the two maps in the left-hand column of Figure 2 . However, for the other time scales we averaged across repeated measurements in an attempt to reduce the confounding effect of short-term variability on longer-term variability. For example, in the hourly time scale experiment, we averaged the first set of five measurements collected in less than 1 second to produce a mean map. We then averaged the second set of five measurements to produce a second mean map, and so on, until we had five mean maps. We then computed the point-by-point mean and standard deviation of these five mean maps as a way of quantifying the minute-to-minute variability. The results are shown in the second column of Figure 2 . A similar approach was used to determine the mean and standard deviation maps shown in Figure 2 for the week and year time scales. For example, the average of five measurements taken the same day produced a map showing the mean measurement. Over 1 week, we determined five such maps and from these we computed the mean and standard deviation maps shown in the third column of Figure 2 . 

The similarity of the maps displayed in Figure 2 suggests that long-term stability is greater than we surmised from inspection of Figure 1 . This observation confirms that averaging multiple measurements taken over relatively short time scales reduces the apparent variability over longer time scales. Nevertheless, it is still true that variability over the longer time scales is greater than variability over shorter time scales, as may be appreciated from inspection of the SD maps of Figure 2 . 

Variability of RMS Wavefront Error

To quantify the overall magnitude of the wavefront error generated by higher-order optical aberrations, we computed the RMS wavefront error for each of the 25 maps collected over 5 days. The variability of these 25 measurements is displayed in Figure 3 in the form of a frequency histogram compiled for each of the four eyes tested. Notice that each of the filled histograms is well removed from the origin, which indicates that the levels of RMS wavefront error observed in our four experimental eyes are significantly different from zero. However, this result does not necessarily imply that these eyes have significant levels of aberrations, because RMS wavefront error is a strictly positive quantity computed as the square root of the sum of squared Zernike coefficients on each map. Consequently, an optically perfect eye that is subject to measurement error may manifest significant levels of RMS wavefront error simply due to random fluctuations of the individual Zernike coefficients about zero. Thus, to use RMS to ascertain whether an eye exhibits significant levels of aberration, we must first establish that the measured RMS levels are higher than expected under the null hypothesis that the eye is aberration-free. To draw this comparison, we subtracted the mean aberration map shown in Figure 2 for the week time scale from each of the 25 individual aberrations maps measured that week to produce 25 residual aberration maps. Zernike analysis of these residual maps confirmed that the distribution of aberration coefficients for each Zernike mode has the same variance as for the original maps but has zero mean, as required by the null hypothesis. We then calculated the RMS of these residual maps and plotted the frequency distribution of these values as open histograms in Figure 3 . The filled histograms (representing the experimental data) and the open histograms (representing the null hypothesis) are widely separated without overlapping. This indicates that all four eyes in our study had higher-order RMS levels that were significantly greater than the levels predicted for eyes that were on average aberration free but that had the same level of variability. In summary, for every measurement of every eye, the RMS wavefront error exceeded the predictions of the null hypothesis, and therefore we concluded that our subjects’ eyes were significantly and consistently aberrant. 

Given this assurance that RMS measurements represent actual aberrations, rather than measurement noise, we used the RMS metric to quantify the instability of repeated aberration measurements over different time scales. Figure 4 shows the variation in five repeated measurements of RMS at each of four time scales for each of four eyes. The symbols in Figure 4A represent individual RMS measurements (single frames), and those in Figures 4B 4C 4D represent the mean of five repeated measurements of RMS. The fluctuations in these symbols represent the variability in mean RMS measurements over the four time scales (second, hour, week, and year). There is a clear indication of instability in the data, and the variability seems to increase with increasing time scales. To evaluate the statistical significance of these fluctuations, we used ANOVA to compare the variability on each time scale with the variability on the shorter time scale. For example, variability on the hour time scale (Fig. 4B) was evaluated by comparing trial-to-trial variability with frame-to-frame variability, and variability on the week and year time scales (Figs. 4C 4D) was evaluated by comparing day-to-day and month-to-month variability with trial-to-trial variability, respectively. (ANOVA was not possible for the second time scale, because there were no measurements on a time scale shorter than the second time scale.) In essence, ANOVA tells us whether the variability of means on any given time scale (the symbols) is significant compared with the variability over the shorter time scale (error bars). The results of this statistical test indicated that, with one exception (subject AB, Fig. 4C ), variability at each time scale was significantly larger than the shorter time scale (e.g., more variability over 1 week than over 1 hour). Restated, Figure 4 shows that in three of four eyes, the mean level of RMS changed significantly over each time scale (hour, week, and year). 

The increase in RMS fluctuations seen over the four time scales can be summarized for all subjects by the standard deviation of repeated measures over each time scale (see the Methods section), which we found to be 0.009 (second), 0.018 (hour), 0.021 (week), and 0.031(year) μm. To interpret these results in clinical terms, we converted micrometers of RMS error to diopters of equivalent defocus (see Methods section). The largest standard deviation of total higher-order aberrations (year, 0.031 μm) is equivalent to 0.02 D (95% probability range, ±0.04 D), which is much smaller than that reported for repeated measurements of spherical equivalent using autorefractors. ^{16 17}  

We searched for systematic changes in RMS wavefront error by performing regression analysis on each of the data sets shown in Figure 4 . The results gave no evidence of systematic decrease or increase in RMS for the data in Figures 4A 4B , or 4C. However, on the year time scale (Fig. 4D) one of the three eyes showed an increasing trend (subject AB, r = 0.799, P < 0.05) and one showed a decreasing trend (subject CT, r = 0.473, P < 0.05) in the levels of higher-order aberrations. 

Variability of Individual Zernike Aberration Coefficients

A second stage of quantitative analysis decomposes the aberration map into a weighted sum of Zernike radial polynomials (Z_n ^m). ^{26 27} Each of these aberration coefficients may be examined separately for test–retest stability in a manner analogous to the tests on RMS error described earlier. A summary of this analysis is shown in Figure 5 for one subject (LT). A pyramid of box-and-whisker plots show how the mean amplitude of each Zernike coefficient varies with time scale (each graph shows data for all four time scales: S, second; H, hour; W, week; Y, year). Each row in the pyramid represents a different radial order (n = 2–4), and the horizontal position in the pyramid indicates meridional frequency (m = −4 to +4). A short horizontal line indicates the mean coefficient amplitude, the confidence interval for this mean (±2 SEM) is a box centered on the mean, and the 95% probability range (±2 SD) is shown by the error bars (i.e., whiskers). Visual inspection of these graphs reveals whether the mean values of individual aberrations modes are significantly different from zero (i.e., the box crosses zero). Had additional data been collected, we would predict that 95% of all new data would lie within the range denoted by the whiskers. 

Figure 5 shows that, for this eye, most aberration modes (e.g., Z₂ ⁰, Z₂ ^±2, Z₃ ^±1, Z₃ ⁺³, Z₄ ⁰, and Z₄ ^±2) were statistically significantly different from zero at all four time scales. However, some modes were significantly different from zero only on some time scales (e.g., Z₃ ⁻³, Z₄ ^±4, and so forth). As we saw in the total RMS data (Fig. 4) , the variability of repeated measurements of each aberration mode generally increased with longer time scales. Even in cases in which the mean aberration level is significant, the 95% probability range sometimes crosses zero, indicating that some of the significant aberration coefficients changed sign from one measurement to the next. 

To assess the stability of individual aberration modes, we used the same ANOVA procedure that was used for RMS analysis. That is, we used ANOVA to determine whether the variability of the means over a given time scale was significant (again, ANOVA was not possible for the second time scale). Significant long-term instability is indicated by a filled circle immediately below or above each box-and-whisker symbol in Figure 5 . For this eye, every Zernike mode was found to be unstable on the year time scale and 7 of 12 modes were unstable on the week time scale. Although the precise pattern of significance varied from eye to eye, there was evidence of instability in all modes in all four eyes. 

Regression analysis of coefficient amplitude across repeated measures indicated that in most cases there were no systematic trends in individual Zernike modes. However, significant regressions were found for the year time scale, especially for the fourth-order modes, but the slopes were very shallow (mean slope = 0.0075 μm/mo). This mode-by-mode analysis showed that the overall changes in RMS observed over 5 months (Fig. 4) were not present in every higher-order Zernike mode. For example, subject CT, in whom the total higher-order RMS decreased over 5 months (see Fig. 4 ), showed a significant increase of Z₄ ⁻⁴, a significant decrease in Z₄ ⁰, and no significant change in Z₄ ⁻². These results demonstrate that using the overall RMS to determine the stability of aberrations can mask changes in individual Zernike coefficients. 

Sources of Variability

There are two general sources of variability in any set of aberration data: biological variability in the eye and measurement noise. To understand the impact of the measured variability in aberrations, we must first apportion the measured variability into these two categories. Although we have no independent measures of biological variability, we performed several control experiments to assess the level of experimental measurement noise. 

Eye translation during the experiment would alter the optical path of both the entering and measurement beams, and thus we were concerned that the standard chin and forehead rest would provide inadequate control of eye position for aberrometry measurements and would introduce variability into the data. Therefore, in addition to taking aberrometry measurements with the standard chin rest, we also took measurements using a bite bar arrangement to see whether this would reduce the measured variability in monochromatic aberrations. These two sets of data collected over 5 days are shown in Figure 6 for all four eyes. It is clear that the data obtained with each head stabilization technique are very similar. This was particularly true in subjects AB, CT, and PK. ANOVA confirmed that, with the exception of subject LT on days 3 and 4, there are no significant differences between the mean RMS levels measured with the two stabilization techniques. The error bars in Figure 6 indicate that the between-trial variability in RMS was almost identical for the two conditions (SD = 0.0147 μm with chin rest and 0.0150 μm with bite bar). The between-day variability (standard deviation across days) was also the same for chin rest (SD = 0.0201 μm) and bite bar (SD = 0.0210 μm). 

A mode-by-mode analysis confirmed that these two methods generated very similar levels of higher-order aberrations. From the nine third- and fourth-order modes measured on four eyes, we found no modes significantly different in two eyes, but three modes reached significance (P < 0.05) in each of the other two eyes. Although statistically significant, these six coefficients measured with the bite bar and chin rest were similar (mean absolute difference = 0.01 μm). We found no evidence of reduced variability in the bite bar data. In fact, the standard deviation of the measurements for each mode differed on average by only 0.0003 μm (slightly larger with the chin-rest than with the bite bar). These results clearly show that the increased head stability provided by the bite bar did not significantly reduce the variability in our data, and thus the increased variability in eye position observed with the chin rest ^{30 31} is not a significant source of variability in our higher-order aberration data. However, we observed significantly more variability in the lower-order prism terms (Z₁ ^±1) with the chin rest than with the bite bar. 

Eye rotation (fixation errors) also alters the optical path of both the entering and measurement beams and thus may also contribute to measurement variability. We examined this possibility by measuring ocular aberrations while systematically rotating the eye by instructing subjects to fixate different rings in the target (see the Methods section). Sample data from one eye (CT) fixating at nine different locations along the horizontal meridian are shown in Figure 7 . Horizontal eye rotation produced systematic changes in the measured horizontal prism (Z₁ ⁺¹), horizontal coma (Z₃ ⁺¹), and horizontal–vertical astigmatism (Z₂ ⁺²). Regression analysis of such data (e.g., horizontal coma, as shown in Fig. 8 ) allowed us to estimate the variability in eye rotation necessary to create the variability in these terms that occurs under normal fixation. In the case of horizontal coma, for example, to generate the 95% probability range in Z₃ ⁺¹ of 0.0612 μm, the 95% probability fixation range would have to be 3.5°. Because fixation variability is approximately one order of magnitude smaller than this (95% probability range = ±0.2° ³⁰ ), we conclude that fixational noise is not a significant contributor to the variability in measured aberrations. 

Discussion

By repeating our aberrometry measurements many times over a range of time scales we confirmed that significant levels of higher-order aberrations are consistently present in human eyes. However, we also found that such a general statement cannot be made for every third- and fourth-order Zernike mode. Significant levels of any given mode may be present on one frame, trial, day, or month but may not be present at some other time. These fluctuations, which clearly increase with increasing time scale, raise important questions about their impact on retinal image quality. If the observed variability in aberrations stems from measurement noise, then we conclude that the eye’s optics and thus image quality are stable over time. However, we argue in this discussion that the measured variability reflects true variability in the optics of the eye, and therefore image quality also varies with time. 

Sources of Variability

We have several indications that measurement noise was a minor contributor to the observed variability in aberration measurements. First, in a companion study ²⁵ that used the COAS aberrometer to measure aberrations of model eyes, we found that repeated measures within a second produced almost identical aberration measurements (between frame variance of higher-order RMS = 1 × 10⁻⁶ μm²). Over this same time scale, variance of RMS in the human eye data (variance = 81 × 10⁻⁶ μm²) was 80 times larger (Fig. 9) . This comparison shows that the short-term variability in the human eye data does not reflect instrument noise, but most likely reflects changes in the eye’s optics, such as microfluctuation of accommodation, instability of the tear film, or small fixational eye movements, all occurring over a very short time scale. Second, although both eye position and rotation vary during any series of measurements, our control experiments indicate that these two factors have little impact on measure variability in aberrations. Third, our previous measurements in model eyes show that operator realignment between measurements introduces a significant but still small amount of variability into the data (between realignment variance of higher-order RMS = 92 × 10⁻⁶ μm² in the model eye), which is approximately 3.5 times smaller than the between-trial variability (variance = 324 × 10⁻⁶ μm²) in human eyes. Therefore, we conclude that 72% of the trial-to-trial variance of higher-order RMS can be attributed to the eye (again, due to accommodation fluctuation, tear film instability, and eye movements), and not instrument alignment. Finally, we note that the variability in the higher-order RMS wavefront error over multiple days and multiple months is even larger (e.g., 441 and 973 × 10⁻⁶ μm², respectively) than within a day. It indicates that the optical fluctuations of the eye are larger over longer time scales, which includes not only all the fluctuations that exist in shorter time scales, but also fluctuations that may reflect the genuine long-term changes in the eye’s optics. 

Impact of Variability of Higher Order Aberrations on Image Quality

The above analysis strongly suggests that the variability we observed in aberrations is due to genuine changes in the optical characteristics of the eyes we tested. Therefore, these changes in optics should be manifest as fluctuations in retinal image quality. To examine the impact of this variability, we computed optical transfer functions (OTFs) for every aberration measurement. It is worth pointing out that even though at this time several laboratories (including ours) are trying to ascertain which metrics of image quality most accurately represent the visual quality, there is no clear answer yet (Cheng X, et al. IOVS 2003;44:ARVO E-Abstract 2123; Applegate RA, et al. IOVS 2003;44:ARVO E-Abstract 2124). Therefore, any fluctuation in OTFs or other image-quality metrics may not directly translate to variation in quality of vision. For example, radial-averaged modulation transfer functions (rMTFs) computed for 2 days in the week for which the wavefronts were least similar are shown in Figure 10 (line 1–4) for two eyes (AB and CT). Lines 1 and 2 show rMTFs calculated from aberration maps that included the second-order as well as the higher-order aberrations on the 2 days. Lines 3 and 4 show rMTFs on the same 2 days calculated from just the higher-order aberrations by setting the second-order aberrations (defocus and astigmatism) to zero, which assumes perfect correction and zero fluctuations of spherical and astigmatic defocus. In both cases, and for both eyes, we can see that the rMTFs were virtually identical, even though the aberrations differed on the 2 days. This stability in rMTF is not surprising for subject AB, who had a very stable RMS (higher-order RMS differed by only 6% between these 2 days). Small differences can be seen in the higher-order rMTFs of subject CT (lines 3 and 4), in whom the higher-order RMS varied by approximately 20% between the two least similar days of the week. These two examples emphasize that the observed fluctuations in monochromatic aberrations do not translate into significant fluctuations in rMTF, even when the second-order aberrations are corrected perfectly. 

Although correcting monochromatic aberrations has been limited in the past to second-order sphere and cylinder, newer wavefront correction technologies have promoted the idea of simultaneous correction of lower- and higher-order aberrations. ¹⁰ We therefore considered the potential impact of optical variability on image quality of an eye with full correction of higher and lower-order aberrations. For example, if we fully correct AB’s and CT’s monochromatic aberrations measured on a given day, we would expect the monochromatic rMTF to be diffraction limited on that day (Fig. 10 , line 7). However, as we have shown, these eyes had different aberrations on other days. Thus, the correction applied to eliminate aberrations on 1 day of the week is an inappropriate correction for the other days. The postcorrection level of aberration jumps from zero on the day of correction (assuming correction is perfect) to a small level of aberration on another day. This emerging low level of aberration is the difference of the aberrations measured on these 2 days. For the 2 days used in Figure 10 , higher-order RMS differs by 0.058 μm (0.04 D, AB) and 0.073 μm (0.06 D, CT). The postcorrection rMTFs on the day of the week with optics most different from the day of correction are shown in Figure 10 as line 6. In both eyes, rMTF has been reduced significantly below that of the perfect, diffraction-limited eye because of day-to-day variability. As expected, when only considering the anticipated higher-order aberrations, AB still has very high rMTF because this eye had very small day-to-day fluctuations in higher-order RMS (Fig. 4) . However, lower-order aberrations also varied throughout the week, and when we calculate postcorrection image quality for residual aberrations including higher- and lower-order aberrations, the resultant rMTFs are much poorer (Fig. 10 , line 5). 

The analysis in Figure 10 emphasizes that variability in aberrations limit the postcorrection image quality. Using this same strategy, we examined the predicted postcorrection image quality for every day we collected data during the 5-month measurement period. This chronology of image quality is shown in Figure 11 , where we have summarized the benefit of correcting higher-order aberrations in terms of improvement in the visual Strehl ratio (VSR, the contrast-sensitivity–weighted OTF/contrast-sensitivity–weighted OTF for diffraction limited optics (Cheng X, et al. IOVS 2003;44:ARVO E-Abstract 2123; Applegate RA, et al. IOVS 2003;44:ARVO E-Abstract 2124) for subject AB. Open symbols show the mean VSR computed for the uncorrected eye. Closed symbols show the mean VSR assuming the eye is corrected with a prescription based on the mean of five measurements on day 1. The shaded regions show the 95% probability range (computed as ±2 SD) for the VSR before and after correction. Had more measurements been made on the same eye, we would have expected 95% of those measurements to fall within the shaded areas. The graph on the left shows the improvement in VSR created by correcting higher-order aberrations, assuming perfect correction of the lower-order aberrations at all times, and thus it is only fluctuations in higher-order aberrations that degrade image quality. The graph on the right shows VSR improvement when fluctuations in astigmatism also degrade image quality. The main conclusion to be drawn from the data in Figure 11 is that the improvement in image quality (in terms of VSR) created by correcting higher-order aberrations is substantial over the long-term provided there is zero variation in the lower-order Zernike modes. The improvement of VSR is more modest, but still very significant, if astigmatism also fluctuates after correction. 

Toward a Rational Strategy for Correcting Higher Order Aberrations

These analyses of improvement in image quality by correcting higher-order aberrations were based on correcting all the Zernike aberration modes. A more practical approach in a clinical setting might be to correct only those modes that are worth correcting. Therefore, we must determine a rational treatment plan. Our mode-by-mode analysis showed that most Zernike modes are statistically significant but few are stable (according to ANOVA). Therefore, a conservative strategy might be to treat only those modes that are significant and stable over time. Correction of a statistically insignificant aberration is equivalent to correcting an aberration that is not present and thus results in an average increase in the eye’s aberration. One might also argue that correction of unstable optical aberrations would be ineffective, in that the necessary correction actually changes over time. Significant Zernike modes can be identified as those for which the 95% confidence interval for the mean does not include zero (see examples in Fig. 5 ). One test of stability is based on ANOVA. For example, if the between-day (or between-month) variance is significantly larger than within-day variance we can classify those modes as unstable. This is obviously a very stringent criterion and only six individual Zernike modes (of 27 modes from three eyes) met this criterion at the year time scale. We tested this strategy with a virtual experiment in which we applied the stable components of the ideal treatment measured on day 1 to subsequent days and then computed image quality for the residual aberration map. As expected, little improvement in VSR is realized by correcting just this select subset of Zernike modes (Fig. 12 , squares) because most of the aberrations are left uncorrected. 

Correction of unstable aberrations may result in a reduction rather than an improvement in retinal image quality. For example, if, at a given time, the aberration level is less than half the magnitude of the mean, correcting the mean level of aberration results in increased aberrations. Correction of stable aberrations, with 95% probability distributions that never approach half the mean therefore generates reliable improvements in image quality. Therefore, identifying the proportion of time during which a fixed correction of aberrations would lead to a decrement rather than an improvement in image quality provides a basis to identify those Zernike modes that are sufficiently stable to warrant correction. We used this approach to calculate the VSR after correcting only those Zernike modes that would result in reduced aberrations 90% or more and 75% or more of the time. The results showed that the strict 90% criterion resulted in less improvement in VSR (triangles in Fig. 12 ) than was achieved by correcting the modes that resulted in reduced aberrations 75% or more of the time (results not shown because they superimposed on the correction of all modes data, filled dots in Fig. 12 ). 

We also compared the improvement in VSR obtained by adopting the same significance and stability criteria with those of correcting all modes with the mean of repeated measurements (Fig. 12 , filled dots). Overall, correcting all modes produced the largest VSR improvement. This suggests that correcting only those modes that meet strictly defined definitions of significance and stability, which in theory may have generated optimum image quality, in practice produces little improvement in image quality. Thus, the preferred strategy to achieve the greatest improvement in image quality over time, at least in the eyes in our study, was to correct all modes, regardless of their magnitude or stability. 

 Supported by National Eye Institute Grant R01-EY05109 (LNT) and the Borish Center for Ophthalmic Research, University of Indiana.

 Submitted for publication March 13, 2003; revised May 21, 2003; accepted June 9, 2003.

 Disclosure: X. Cheng, None; N.L. Himebaugh, None; P.S. Kollbaum, None; L.N. Thibos, None; A. Bradley, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

Contour wavefront aberration maps (0.1 μm interval) of total higher-order (third plus fourth) wavefront error at four time scales in subject CT (6-mm pupil, −3 to 3 mm). First column: five measurements taken within 1 second without realignment of the aberrometer; second column: five individual measurements taken within 1 hour on a particular day, with realignment of the aberrometer between trials; third and fourth columns: five repeated aberration measurements taken on five consecutive days and in five consecutive months, respectively.

Figure 2.

 

Mean (0.1-μm interval) and SD (0.05-μm interval) wavefront maps of higher-order aberrations on four time scales for the same eye (CT) shown in Figure 1 . The mean map at the shortest time scale was calculated by averaging five individual measurements within a second (e.g., column 1 in Fig. 1 ). The SD map on this time scale shows the point-by-point standard deviations of the five wavefront maps. The maps of the mean measurements obtained on the other three time scales were calculated by first averaging five repeated measures taken on a short time scale (e.g., second) and then computing the mean and SD of five such maps collected over a longer time scale (e.g., hour). Pupil coordinates (−3 to 3 mm) are indicated on the year maps.

Figure 3.

 

A test of the null hypothesis that RMS wavefront error is caused by noisy measurements of aberration-free eyes. (▪) frequency histogram of 25 repeated measures of higher-order RMS error obtained in 1 week; (□) frequency histogram of RMS error computed from a hypothetical eye for which the distribution of every higher-order Zernike mode had the same variance as the actual eye but had zero mean.

Figure 4.

 

Temporal variation of higher-order RMS error measured in four eyes on four time scales. (A) On the second time scale, the symbols show individual measurements of higher-order RMS within one trial. (B–D) Symbols and error bars show means and standard deviations of RMS across five repeated measurements within one measurement session. Five measurement sessions were completed within 1 hour (B), 1 week (C), and 1 year (D). Filled symbols indicate greater variability between sessions than within a session (P < 0.05), as determined by ANOVA.

Figure 5.

 

Pyramid box-and-whisker plots showing the mean, 95% confidence interval (boxes, ±2 SEM) for the mean, and 95% probability range (whiskers, ±2 SD) of each Zernike coefficient on the four time scales in one subject (LT). Note that the confidence intervals for the second time scale are for five measurements, whereas the confidence intervals for the other time scales were calculated for 25 repeated measures. Filled circles: data showing greater variability between sessions than within a session, according to ANOVA (P < 0.05), which indicates temporal instability. ANOVA was not performed on the 1-second time scale.

Figure 6.

 

Comparison of day-to-day variability in higher-order RMS with two systems of head restraint: a bite bar and a chin rest. Symbols and error bars show the mean ± 1 SD across five repeated measurements within 1 day in four subjects.

Figure 7.

 

Effect of fixation error on aberration coefficients (subject CT). (•) Mean Zernike coefficient measurements at each fixation position along the horizontal meridian. (∗) Mean of six measurements for central fixation. Error bars indicate ±1 SD of three measurements (in most cases error bars are smaller than the symbols).

Figure 8.

 

Example of measured horizontal coma as a function of horizontal fixation position (subject CT). Filled circles and error bars show mean ± 1 SD of the measured horizontal coma at each fixation location. Straight line: linear regression of the aberration measurements. Open circles: lower and upper limits of the 95% probability range of the measured horizontal coma at normal central fixation. Shaded area: range of fixation eccentricity at which the aberration measurements fall within the 95% probability range of the central fixation measurements, which covers approximately 3.5°.

Figure 9.

 

Comparison of variances of higher-order RMS measured in human eyes (▪) over time scales of 1 second, 1 hour, 1 weeks, and 1 year, with the variances measured in a model eye (□) over the second and hour time scales.

Figure 10.

 

Impact of aberration variability on image quality. Monochromatic radial MTFs are shown for subject AB and CT for the 2 days within 1 week on which they had the least similar higher-order aberrations. Lines 1 and 2 show rMTFs calculated from second-order and higher-order aberrations on the 2 days. Lines 3 and 4 show rMTFs calculated for higher-order aberrations (defocus and astigmatism modes set to zero). Lines 5 to 7 show rMTFs after correcting aberrations. Line 7 has both higher and lower orders perfectly corrected (diffraction limited). Lines 5 and 6 show the rMTFs after correcting 1 day’s aberrations by using the other day’s aberration measurements. Line 5 includes uncorrected lower- and higher-order aberrations. Line 6 assumes perfect correction of the lower-order aberrations.

Figure 11.

 

Persistent improvement in image quality derived from aberration correction (subject AB). Open symbols: mean VSRs computed for the uncorrected eye. Filled symbols: mean VSRs computed by assuming the eye is corrected with a prescription based on the mean of five measurements on day 1. The shaded regions show the 95% probability range (computed as ±2 SD) for VSR before and after correction. Left: VSRs after correcting higher-order aberrations (assuming perfect, lasting correction of the lower-order aberrations); right: VSRs when fluctuations in astigmatism also contribute to image degradation.

Figure 12.

 

Evaluation of strategies for correcting higher-order aberrations (subject AB). Shown are image quality before correction, after correcting all Zernike modes using the mean aberration measurements on day 1; improvement in VSRs after correcting only those Zernike modes that resulted in reduced aberrations more than 90% of the time; and improvement after correcting only those Zernike modes that were significant and stable according to ANOVA.