The Effect of Optical Defocus on the Test–Retest Variability of Visual Acuity Measurements

Abstract

purpose. To determine the effect of optical defocus on the test–retest variability (TRV) of visual acuity measurements in normal subjects.

methods. Normal subjects underwent repeated visual acuity measurement with optical defocus of 0, 0.50, and 1.00 D. All measurements were taken using the Early Treatment Diabetic Retinopathy Study (ETDRS) version of the Bailey-Lovie logMAR chart. TRV was quantified in terms of its 95% range, both empirically and using the approach of Bland and Altman.

results. According to the Bland and Altman approach, the estimated 95% TRV ranges were ±0.11 logarithm of the minimum angle of resolution (logMAR) for 0-D defocus, ±0.18 logMAR for 0.50-D defocus, and ±0.25 logMAR for 1.00-D defocus.

conclusions. Optical defocus has a considerable effect on the TRV of visual acuity measurements. These findings have important implications for both clinical practice and clinical research. Uncorrected refractive errors as small as 0.50 D may compromise the detection of visual change in individuals, and contribute to unnecessarily large sample sizes in clinical trials in which visual acuity is used as a primary outcome measure.

Visual acuity is the primary measure of visual function in both clinical practice and clinical research. It is well established that, when visual acuity is measured repeatedly on a given individual, the acuity score tends to vary, even in the absence of true clinical change. ^{1 2 3 4 5 6 7 8 9 10} This variability is a form of measurement noise and is referred to hereafter as test–retest variability (TRV). Our ability to detect true clinical change using visual acuity data is inversely related to the TRV of the test used. One way of quantifying TRV is in terms of its 95% range—that is, the range of values in which, 95 times in a 100, the differences between repeated measures of acuity conducted in the absence of any change will lie. Published estimates of 95% TRV ranges vary from ±0.07 logarithm of the minimum angle of resolution (logMAR) ^{3 4 5 6 7 8 9 10} to ±0.33 logMAR. ⁷ Some of this variation may be explained by differences in test design and/or scoring method. The Snellen chart’s numerous limitations have been well described. ^{11 12} Most pertinent with respect to TRV is the large-scale increment, which is predominantly a consequence of using a line-by-line scoring method. An excessively large-scale increment is known to be associated with increased TRV. ^{2 5 10} Charts based on the design principles of Bailey and Lovie ¹² eliminate some of the extraneous sources of variability inherent in Snellen measurements. They also allow the use of interpolated scoring methods that result in a smaller scale increment and, in turn, lower TRV. 

If we consider only those published estimates of TRV made using Bailey-Lovie type charts and interpolated scoring, the range of reported values is reduced, but still substantial (2.7-fold variation, Table 1 ). What are the causes of this large variation? No association between age of subjects and TRV was found in the studies of Beck et al. ¹⁷ and Lovie-Kitchen and Brown. ¹⁸ Reeves et al. ⁶ suggested three other potential sources of variation in TRV: The presence or absence of disease, whether or not the measurements were conducted in a single session, and the presence or absence of uncorrected refractive error. Subjects with cataract appear at both extremes of TRV in Table 1 , suggesting that presence of cataract is not a major determinant of TRV. Three of the four studies that estimated TRV over two visits reported 95% ranges of less than ±0.10 logMAR, which is not consistent with a hypothesis of increasing TRV with longer periods between examinations. 

Although there has been speculation that a relationship exists between uncorrected refractive error-optical defocus and the TRV of visual acuity measurements, ⁶ the empiric data are ambiguous. Siderov and Tiu ⁸ reported a 95% range of ±0.16 logMAR for both aided and unaided measurements, whereas Elliott and Sheridan ³ found TRV in unaided normal subjects to be three times that observed when a full correction was worn (95% ranges of ±0.21 and ±0.07 logMAR, respectively). Neither study gave details of the refractive errors in their subjects. Elliott and Sheridan ³ give the mean unaided visual acuity of their population as 6/9 suggesting that the overall degree of ametropia was not high. Siderov and Tiu, ⁸ in contrast, recruited from a university optometry clinic, and it is possible that the extent of ametropia was greater in their sample. 

To our knowledge, the only published study in which the effects of optical defocus have been systematically investigated in a way that sheds light on its relationship with TRV is that of Carkeet et al. ¹⁹ They assessed the effect of two discrete levels of optical defocus on the shape of the frequency-of-seeing curve as determined by Probit analysis. ²⁰ The results showed that optical defocus was associated with a flattening of the frequency-of-seeing curve. Based on this finding, Carkeet et al. ¹⁹ commented that the 95% TRV range would be expected to be larger under conditions of defocus than in subjects with good correction. 

The purpose of this study was to investigate whether a relationship between optical defocus and TRV of visual acuity measurements exists in normal subjects. 

Materials and Methods

Subjects

Normal subjects were recruited from the staff of Moorfields Eye Hospital (London, UK). The tenets of the Declaration of Helsinki were adhered to in full. Inclusion criteria were age 50 years or less, hyperopia not exceeding +0.50 D (mean sphere) and myopia not exceeding −10.00 D (mean sphere), astigmatism not exceeding 1.50 D, absence of any ocular abnormality including media opacity, no history of ocular abnormality including amblyopia, no history of regular use of the ETDRS logMAR chart, and acuity better than +0.20 logMAR (Snellen equivalent 6/9.5). 

Testing Procedure

Before testing, each subject underwent formal subjective refraction with binocular balancing using the intermediate contrast technique of Humphriss and Woodruff ²¹ and an end point of maximum plus/minimum minus for maximum visual acuity. The chart used for refraction was used for that purpose only. One eye only of each subject was assessed. ²² When both eyes met the inclusion criteria, the right eye was used as the study eye. Each subject then underwent two acuity measurements with each of three refractive corrections: full refractive correction (0 D optical defocus), full refractive correction plus +0.50 D defocus, and full refractive correction plus +1.00 D defocus. 

Each subject therefore underwent six acuity measurements using the ETDRS logMAR acuity chart in a session lasting approximately 15 to 20 minutes. This chart is based on the design suggested by Bailey and Lovie, ¹² but incorporates the recommendations of the National Academy of Sciences-National Research Council (NAS-NRC). ²³ The chart has been described in detail by Ferris et al. ¹¹ Three versions of the ETDRS logMAR chart were used for the measurements at 0 D defocus, +0.50 D defocus, and +1.00 D defocus: the refraction chart, chart 1, and chart 2, respectively. The refraction chart was used for the measurements at zero defocus because the control of letter difficulty is less rigorous than in charts 1 and 2. ¹¹ Accordingly, if the refraction chart were associated with a higher degree of TRV than charts 1 and 2, it would lessen any measured effect of defocus on TRV rather than exacerbate it. To limit memory effects, subjects were required to read the chart forward for one measurement and backward for the other. The order of the six measurements was randomized (with the sole restriction of the same chart never being used for consecutive measurements) to limit learning or fatigue effects as potential confounders. All charts were viewed from a distance of 6.3 m. This results in a range of effective letter sizes from +0.80 to −0.50 logMAR (Snellen equivalent 6/37.9–6/1.9) as opposed to the range at the standard 4-m distance of +1.00 to −0.30 logMAR (Snellen equivalent 6/60–6/3). The 6.3-m testing distance was chosen to prevent subjects with very good acuity from reading all or part of the bottom line on the chart. It is desirable to prevent this, because the resultant truncation or ceiling effects may artificially reduce the TRV of those with good acuity. ⁶ A longer testing distance was avoided because, for a testing distance of 6.3 meters, there would be three rows of letters above the +0.50 logMAR letter size, which is the expected threshold for 1.00 D of optical defocus. ^{24 25} The subject’s responses were recorded on specially designed data proformas. 

Scoring and End Points

Each chart was attempted using a forced-choice testing paradigm. The end point for each measurement was taken as a full row of errors. Such a termination rule is desirable to limit TRV with Bailey-Lovie type logMAR charts. ²⁶ Each chart was scored using an interpolated scoring method, whereby credit is given for each letter correctly named. ^{11 27} As mentioned, this method tends to limit TRV through the use of a small-scale increment. For each subject, at each level of optical defocus, the difference in the two acuity measurements was calculated by subtracting the score of the backward measurement from that of the forward measurement. 

Statistical Methods

For each individual, the difference in the two acuity measurements at a given degree of defocus was calculated. The SD of these differences across all 40 individuals was estimated. TRV was quantified in terms of its 95% range. This 95% range was estimated in two ways: directly from the observed distribution and using the approach of Bland and Altman ²⁶ (95% range, ±1.96 SD, making the assumption that the differences are normally distributed). Evidence of differences in standard deviations across the three degrees of defocus was sought using Levene’s test, ^{28 29} which is robust to departures from normality. The assumption of normality itself was assessed for each degree of defocus (0, 0.50, and 1.00 D) using the Shapiro-Wilk W-test and by inspection of quantile-normal plots. All analyses were performed on computer (Stata, ver. 8.2; Stata, College Station, TX). 

Results

Forty subjects were recruited. Age at last birthday ranged from 21 to 50 years (mean, 33 years). Refractive error ranged from +0.50 to −8.75 D (median spherical equivalent, −1.25 D). Astigmatism ranged from 0 to 1.25 D (median, 0 D). Acuity ranged from −0.32 to +0.12 logMAR with a median of −0.14 logMAR (Snellen equivalents: range 6/3–6/8, median 6/4.3). Three individuals had acuity worse than 0.00 logMAR (Snellen 6/6), one of which was worse than +0.10 logMAR (Snellen 6/7.5). 

The mean difference at each degree of defocus was close to 0 (Table 2) , indicating that the chart was no more difficult when attempted backward. There was very strong evidence that the standard deviation of the differences varied with the degree of defocus (P = 0.0002), with the standard deviation (and hence the TRV) increasing as the level of optical defocus increased. At 0 D defocus the observed 95% TRV range and the 95% TRV range based on the assumption of normality were identical for practical purposes, with no evidence of non-normality (P = 0.38). At +1.00 D defocus the observed 95% range was slightly wider than that predicted by normal theory. However, the distribution of the observed data was compatible with normality (P = 0.38), and the confidence intervals for the upper and lower bounds derived from the observed distribution both included the bounds predicted by normal theory. At +0.50 D defocus, the observed 95% TRV range showed a degree of asymmetry and there was weak evidence against the hypothesis of normality (P = 0.04), largely due to one observation with an outlying value of +0.30 logMAR. Excluding this observation from the analysis resulted in an observed 95% range of −0.20 to +0.10 logMAR and a predicted 95% range based on normal theory of ±0.15 logMAR. Exclusion of this observation from the comparison of standard deviations across the three degrees of defocus did not compromise the strength of the evidence against the null hypothesis of equal standard deviations (P = 0.00005). 

Discussion

The results of this study suggest that optical defocus strongly influences the TRV of visual acuity data. This finding is consistent with the work of Carkeet et al. ¹⁹ who demonstrated a significant flattening of the frequency-of-seeing curve (as described using Probit size ²⁰ ) with optical defocus of 1.00 or 2.00 D. The data suggest that the increase in TRV may be considerable, even with degrees of optical defocus as small as 0.50 D, causing a blur that is consistent with an acuity of approximately 6/9 (+0.18 logMAR). ^{24 25} This study has estimated TRV, in terms of its 95% range, to be ±0.10 logMAR in normal subjects wearing a full refractive correction. This level of TRV appears consistent with the results of Elliott and Sheridan ³ (±0.07 logMAR), Bailey et al. ² (±0.09 logMAR), Arditi and Cagenello ¹ (±0.09 logMAR), and Rosser et al. ¹⁴ (±0.11 logMAR). All studies involved normal subjects wearing their full refractive corrections. 

Review of published TRV estimates does not suggest a strong relationship between ocular abnormality and greater TRV (see Table 1 ). At least three studies have reported on the effects of ocular abnormality on TRV. Vanden Bosch and Wall ¹⁰ found no difference in repeatability between patients with various forms of maculopathy (mean VA +0.32 logMAR) and age-matched normal subjects (mean VA, −0.11 logMAR). Blackhurst and Maguire ³⁰ reported that, in subjects with macular degeneration, a smaller percentage of test–retest measurements fell within a given range compared with that in normal subjects. This difference did not, however, attain statistical significance. Elliott and Sheridan ³ found TRV to be greater in subjects with cataract than in those without by ±0.02 logMAR (equivalent to one ETDRS letter). In all three studies, subjects were tested wearing their full refractive correction. It has been suggested that small differences such as these may be due to the truncated nature the of measurement scale, which may artificially reduce TRV for higher (better) acuity scores. ⁶ The present study used a 6.3-m testing distance to avoid such effects. There is some evidence that TRV is similar in abnormal subjects and acuity-matched normal subjects. ³¹ Thus, it appears that a relationship between TRV and ocular abnormality may exist, but that it is likely to be weak compared with that between uncorrected refractive error and TRV. 

In summary, our study suggests that even small degrees of optical defocus may produce a considerable increase in the TRV of visual acuity measurements. Such a phenomenon would have important implications for both clinical practice and clinical research, as the width of the 95% TRV range determines the ability of a test to detect change. Based on these data, in individuals whose vision is being monitored, the detection of a deterioration in vision will be delayed if refractive errors as small as 0.50 D are left uncorrected. For a clinical research study using visual acuity as a primary outcome measure, an unnecessarily large sample size is needed to detect a given degree of change if visual acuity measurements are not conducted with a full refractive correction. 

However, the results of this study should be interpreted in light of evidence that suggests that the reduction in visual acuity associated with optical defocus may be less pronounced after a period of adaptation. ^{32 33} It is therefore conceivable that any increase in TRV associated with optical defocus may be less pronounced in subjects who are habitually in a state of defocus than in those who are not. A separate study is necessary to elucidate the strength of any relationship between adaptation time and TRV. In the absence of further evidence and in the light of the apparent strength of the relationship between defocus and TRV, it seems prudent for clinicians and clinical researchers to measure visual acuity with a full refractive correction wherever possible. 

Additional investigation is needed to establish whether the deleterious effects of optical defocus on TRV are mitigated by increased adaptation time. Further research is also warranted to determine whether the effect of optical defocus on TRV is influenced by the presence of eye disease, and, if so, whether the effect varies with different forms of ocular abnormality. 

 Supported by the Trustees of Moorfields Eye Hospital National Health Service, Grant MURI 1011.

 Submitted for publication December 8, 2003; revised January 6, 2004; accepted January 11, 2004.

 Disclosure: D.A. Rosser, None; I.E. Murdoch, None; S.N. Cousens, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U. S. C. §1734 solely to indicate this fact.

Table 1.

 

 

Published Estimates of Test–Retest Variability Using Charts Based on Bailey and Lovie’s Design

Table 1.

 

Published Estimates of Test–Retest Variability Using Charts Based on Bailey and Lovie’s Design

Author	95% TRV Range*	Test Sessions (n)	Subjects	Refractive Correction
Elliott and Sheridan 3	±0.07	2	Normal	Full
van den Brom et al. 13	±0.08	1	Cataract	Full
Elliott and Sheridan 3	±0.09	2	Cataract	Full
Arditi and Cagenello 1	±0.09	2	Normal	Full
Bailey et al. 2	±0.10	1	Normal	Full
Rosser et al. 14	±0.11	1	Normal	Full
Lovie-Kitchen 4	±0.16	1	Normal	Unaided
Rosser et al. 7	±0.18	1	Cataract	Habitual
Reeves et al. 15	±0.19	2	Mixed	Full

*  Estimated using the approach of Bland and Altman. ¹⁶

Table 2.

 

 

Mean Differences between Paired Acuity Measurements and their 95% Ranges by Degree of Optical Defocus

Table 2.

 

Mean Differences between Paired Acuity Measurements and their 95% Ranges by Degree of Optical Defocus

Degree of Optical Defocus (D)	Mean Difference (logMAR)	SD	Observed 95% TRV Range	Predicted 95% TRV Range* (Assuming Normality)
0.00	0.002	0.051	−0.100 to +0.100	±0.10
+0.50	−0.013	0.093	−0.200 to +0.295	±0.18
+1.00	−0.004	0.124	−0.279 to +0.259	±0.24

*  ±1.96 SD.