April 2004
Volume 45, Issue 4
Clinical and Epidemiologic Research  |   April 2004
The Effect of Optical Defocus on the Test–Retest Variability of Visual Acuity Measurements
Author Affiliations
  • Daniel A. Rosser
    From the Institute of Ophthalmology, University College London, London, United Kingdom;
    Moorfields Eye Hospital, London, United Kingdom;
  • Ian E. Murdoch
    From the Institute of Ophthalmology, University College London, London, United Kingdom;
    Moorfields Eye Hospital, London, United Kingdom;
  • Simon N. Cousens
    London School of Hygiene and Tropical Medicine, London, United Kingdom.
Investigative Ophthalmology & Visual Science April 2004, Vol.45, 1076-1079. doi:https://doi.org/10.1167/iovs.03-1320
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Daniel A. Rosser, Ian E. Murdoch, Simon N. Cousens; The Effect of Optical Defocus on the Test–Retest Variability of Visual Acuity Measurements. Invest. Ophthalmol. Vis. Sci. 2004;45(4):1076-1079. doi: https://doi.org/10.1167/iovs.03-1320.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

purpose. To determine the effect of optical defocus on the test–retest variability (TRV) of visual acuity measurements in normal subjects.

methods. Normal subjects underwent repeated visual acuity measurement with optical defocus of 0, 0.50, and 1.00 D. All measurements were taken using the Early Treatment Diabetic Retinopathy Study (ETDRS) version of the Bailey-Lovie logMAR chart. TRV was quantified in terms of its 95% range, both empirically and using the approach of Bland and Altman.

results. According to the Bland and Altman approach, the estimated 95% TRV ranges were ±0.11 logarithm of the minimum angle of resolution (logMAR) for 0-D defocus, ±0.18 logMAR for 0.50-D defocus, and ±0.25 logMAR for 1.00-D defocus.

conclusions. Optical defocus has a considerable effect on the TRV of visual acuity measurements. These findings have important implications for both clinical practice and clinical research. Uncorrected refractive errors as small as 0.50 D may compromise the detection of visual change in individuals, and contribute to unnecessarily large sample sizes in clinical trials in which visual acuity is used as a primary outcome measure.

Visual acuity is the primary measure of visual function in both clinical practice and clinical research. It is well established that, when visual acuity is measured repeatedly on a given individual, the acuity score tends to vary, even in the absence of true clinical change. 1 2 3 4 5 6 7 8 9 10 This variability is a form of measurement noise and is referred to hereafter as test–retest variability (TRV). Our ability to detect true clinical change using visual acuity data is inversely related to the TRV of the test used. One way of quantifying TRV is in terms of its 95% range—that is, the range of values in which, 95 times in a 100, the differences between repeated measures of acuity conducted in the absence of any change will lie. Published estimates of 95% TRV ranges vary from ±0.07 logarithm of the minimum angle of resolution (logMAR) 3 4 5 6 7 8 9 10 to ±0.33 logMAR. 7 Some of this variation may be explained by differences in test design and/or scoring method. The Snellen chart’s numerous limitations have been well described. 11 12 Most pertinent with respect to TRV is the large-scale increment, which is predominantly a consequence of using a line-by-line scoring method. An excessively large-scale increment is known to be associated with increased TRV. 2 5 10 Charts based on the design principles of Bailey and Lovie 12 eliminate some of the extraneous sources of variability inherent in Snellen measurements. They also allow the use of interpolated scoring methods that result in a smaller scale increment and, in turn, lower TRV. 
If we consider only those published estimates of TRV made using Bailey-Lovie type charts and interpolated scoring, the range of reported values is reduced, but still substantial (2.7-fold variation, Table 1 ). What are the causes of this large variation? No association between age of subjects and TRV was found in the studies of Beck et al. 17 and Lovie-Kitchen and Brown. 18 Reeves et al. 6 suggested three other potential sources of variation in TRV: The presence or absence of disease, whether or not the measurements were conducted in a single session, and the presence or absence of uncorrected refractive error. Subjects with cataract appear at both extremes of TRV in Table 1 , suggesting that presence of cataract is not a major determinant of TRV. Three of the four studies that estimated TRV over two visits reported 95% ranges of less than ±0.10 logMAR, which is not consistent with a hypothesis of increasing TRV with longer periods between examinations. 
Although there has been speculation that a relationship exists between uncorrected refractive error-optical defocus and the TRV of visual acuity measurements, 6 the empiric data are ambiguous. Siderov and Tiu 8 reported a 95% range of ±0.16 logMAR for both aided and unaided measurements, whereas Elliott and Sheridan 3 found TRV in unaided normal subjects to be three times that observed when a full correction was worn (95% ranges of ±0.21 and ±0.07 logMAR, respectively). Neither study gave details of the refractive errors in their subjects. Elliott and Sheridan 3 give the mean unaided visual acuity of their population as 6/9 suggesting that the overall degree of ametropia was not high. Siderov and Tiu, 8 in contrast, recruited from a university optometry clinic, and it is possible that the extent of ametropia was greater in their sample. 
To our knowledge, the only published study in which the effects of optical defocus have been systematically investigated in a way that sheds light on its relationship with TRV is that of Carkeet et al. 19 They assessed the effect of two discrete levels of optical defocus on the shape of the frequency-of-seeing curve as determined by Probit analysis. 20 The results showed that optical defocus was associated with a flattening of the frequency-of-seeing curve. Based on this finding, Carkeet et al. 19 commented that the 95% TRV range would be expected to be larger under conditions of defocus than in subjects with good correction. 
The purpose of this study was to investigate whether a relationship between optical defocus and TRV of visual acuity measurements exists in normal subjects. 
Materials and Methods
Normal subjects were recruited from the staff of Moorfields Eye Hospital (London, UK). The tenets of the Declaration of Helsinki were adhered to in full. Inclusion criteria were age 50 years or less, hyperopia not exceeding +0.50 D (mean sphere) and myopia not exceeding −10.00 D (mean sphere), astigmatism not exceeding 1.50 D, absence of any ocular abnormality including media opacity, no history of ocular abnormality including amblyopia, no history of regular use of the ETDRS logMAR chart, and acuity better than +0.20 logMAR (Snellen equivalent 6/9.5). 
Testing Procedure
Before testing, each subject underwent formal subjective refraction with binocular balancing using the intermediate contrast technique of Humphriss and Woodruff 21 and an end point of maximum plus/minimum minus for maximum visual acuity. The chart used for refraction was used for that purpose only. One eye only of each subject was assessed. 22 When both eyes met the inclusion criteria, the right eye was used as the study eye. Each subject then underwent two acuity measurements with each of three refractive corrections: full refractive correction (0 D optical defocus), full refractive correction plus +0.50 D defocus, and full refractive correction plus +1.00 D defocus. 
Each subject therefore underwent six acuity measurements using the ETDRS logMAR acuity chart in a session lasting approximately 15 to 20 minutes. This chart is based on the design suggested by Bailey and Lovie, 12 but incorporates the recommendations of the National Academy of Sciences-National Research Council (NAS-NRC). 23 The chart has been described in detail by Ferris et al. 11 Three versions of the ETDRS logMAR chart were used for the measurements at 0 D defocus, +0.50 D defocus, and +1.00 D defocus: the refraction chart, chart 1, and chart 2, respectively. The refraction chart was used for the measurements at zero defocus because the control of letter difficulty is less rigorous than in charts 1 and 2. 11 Accordingly, if the refraction chart were associated with a higher degree of TRV than charts 1 and 2, it would lessen any measured effect of defocus on TRV rather than exacerbate it. To limit memory effects, subjects were required to read the chart forward for one measurement and backward for the other. The order of the six measurements was randomized (with the sole restriction of the same chart never being used for consecutive measurements) to limit learning or fatigue effects as potential confounders. All charts were viewed from a distance of 6.3 m. This results in a range of effective letter sizes from +0.80 to −0.50 logMAR (Snellen equivalent 6/37.9–6/1.9) as opposed to the range at the standard 4-m distance of +1.00 to −0.30 logMAR (Snellen equivalent 6/60–6/3). The 6.3-m testing distance was chosen to prevent subjects with very good acuity from reading all or part of the bottom line on the chart. It is desirable to prevent this, because the resultant truncation or ceiling effects may artificially reduce the TRV of those with good acuity. 6 A longer testing distance was avoided because, for a testing distance of 6.3 meters, there would be three rows of letters above the +0.50 logMAR letter size, which is the expected threshold for 1.00 D of optical defocus. 24 25 The subject’s responses were recorded on specially designed data proformas. 
Scoring and End Points
Each chart was attempted using a forced-choice testing paradigm. The end point for each measurement was taken as a full row of errors. Such a termination rule is desirable to limit TRV with Bailey-Lovie type logMAR charts. 26 Each chart was scored using an interpolated scoring method, whereby credit is given for each letter correctly named. 11 27 As mentioned, this method tends to limit TRV through the use of a small-scale increment. For each subject, at each level of optical defocus, the difference in the two acuity measurements was calculated by subtracting the score of the backward measurement from that of the forward measurement. 
Statistical Methods
For each individual, the difference in the two acuity measurements at a given degree of defocus was calculated. The SD of these differences across all 40 individuals was estimated. TRV was quantified in terms of its 95% range. This 95% range was estimated in two ways: directly from the observed distribution and using the approach of Bland and Altman 26 (95% range, ±1.96 SD, making the assumption that the differences are normally distributed). Evidence of differences in standard deviations across the three degrees of defocus was sought using Levene’s test, 28 29 which is robust to departures from normality. The assumption of normality itself was assessed for each degree of defocus (0, 0.50, and 1.00 D) using the Shapiro-Wilk W-test and by inspection of quantile-normal plots. All analyses were performed on computer (Stata, ver. 8.2; Stata, College Station, TX). 
Forty subjects were recruited. Age at last birthday ranged from 21 to 50 years (mean, 33 years). Refractive error ranged from +0.50 to −8.75 D (median spherical equivalent, −1.25 D). Astigmatism ranged from 0 to 1.25 D (median, 0 D). Acuity ranged from −0.32 to +0.12 logMAR with a median of −0.14 logMAR (Snellen equivalents: range 6/3–6/8, median 6/4.3). Three individuals had acuity worse than 0.00 logMAR (Snellen 6/6), one of which was worse than +0.10 logMAR (Snellen 6/7.5). 
The mean difference at each degree of defocus was close to 0 (Table 2) , indicating that the chart was no more difficult when attempted backward. There was very strong evidence that the standard deviation of the differences varied with the degree of defocus (P = 0.0002), with the standard deviation (and hence the TRV) increasing as the level of optical defocus increased. At 0 D defocus the observed 95% TRV range and the 95% TRV range based on the assumption of normality were identical for practical purposes, with no evidence of non-normality (P = 0.38). At +1.00 D defocus the observed 95% range was slightly wider than that predicted by normal theory. However, the distribution of the observed data was compatible with normality (P = 0.38), and the confidence intervals for the upper and lower bounds derived from the observed distribution both included the bounds predicted by normal theory. At +0.50 D defocus, the observed 95% TRV range showed a degree of asymmetry and there was weak evidence against the hypothesis of normality (P = 0.04), largely due to one observation with an outlying value of +0.30 logMAR. Excluding this observation from the analysis resulted in an observed 95% range of −0.20 to +0.10 logMAR and a predicted 95% range based on normal theory of ±0.15 logMAR. Exclusion of this observation from the comparison of standard deviations across the three degrees of defocus did not compromise the strength of the evidence against the null hypothesis of equal standard deviations (P = 0.00005). 
The results of this study suggest that optical defocus strongly influences the TRV of visual acuity data. This finding is consistent with the work of Carkeet et al. 19 who demonstrated a significant flattening of the frequency-of-seeing curve (as described using Probit size 20 ) with optical defocus of 1.00 or 2.00 D. The data suggest that the increase in TRV may be considerable, even with degrees of optical defocus as small as 0.50 D, causing a blur that is consistent with an acuity of approximately 6/9 (+0.18 logMAR). 24 25 This study has estimated TRV, in terms of its 95% range, to be ±0.10 logMAR in normal subjects wearing a full refractive correction. This level of TRV appears consistent with the results of Elliott and Sheridan 3 (±0.07 logMAR), Bailey et al. 2 (±0.09 logMAR), Arditi and Cagenello 1 (±0.09 logMAR), and Rosser et al. 14 (±0.11 logMAR). All studies involved normal subjects wearing their full refractive corrections. 
Review of published TRV estimates does not suggest a strong relationship between ocular abnormality and greater TRV (see Table 1 ). At least three studies have reported on the effects of ocular abnormality on TRV. Vanden Bosch and Wall 10 found no difference in repeatability between patients with various forms of maculopathy (mean VA +0.32 logMAR) and age-matched normal subjects (mean VA, −0.11 logMAR). Blackhurst and Maguire 30 reported that, in subjects with macular degeneration, a smaller percentage of test–retest measurements fell within a given range compared with that in normal subjects. This difference did not, however, attain statistical significance. Elliott and Sheridan 3 found TRV to be greater in subjects with cataract than in those without by ±0.02 logMAR (equivalent to one ETDRS letter). In all three studies, subjects were tested wearing their full refractive correction. It has been suggested that small differences such as these may be due to the truncated nature the of measurement scale, which may artificially reduce TRV for higher (better) acuity scores. 6 The present study used a 6.3-m testing distance to avoid such effects. There is some evidence that TRV is similar in abnormal subjects and acuity-matched normal subjects. 31 Thus, it appears that a relationship between TRV and ocular abnormality may exist, but that it is likely to be weak compared with that between uncorrected refractive error and TRV. 
In summary, our study suggests that even small degrees of optical defocus may produce a considerable increase in the TRV of visual acuity measurements. Such a phenomenon would have important implications for both clinical practice and clinical research, as the width of the 95% TRV range determines the ability of a test to detect change. Based on these data, in individuals whose vision is being monitored, the detection of a deterioration in vision will be delayed if refractive errors as small as 0.50 D are left uncorrected. For a clinical research study using visual acuity as a primary outcome measure, an unnecessarily large sample size is needed to detect a given degree of change if visual acuity measurements are not conducted with a full refractive correction. 
However, the results of this study should be interpreted in light of evidence that suggests that the reduction in visual acuity associated with optical defocus may be less pronounced after a period of adaptation. 32 33 It is therefore conceivable that any increase in TRV associated with optical defocus may be less pronounced in subjects who are habitually in a state of defocus than in those who are not. A separate study is necessary to elucidate the strength of any relationship between adaptation time and TRV. In the absence of further evidence and in the light of the apparent strength of the relationship between defocus and TRV, it seems prudent for clinicians and clinical researchers to measure visual acuity with a full refractive correction wherever possible. 
Additional investigation is needed to establish whether the deleterious effects of optical defocus on TRV are mitigated by increased adaptation time. Further research is also warranted to determine whether the effect of optical defocus on TRV is influenced by the presence of eye disease, and, if so, whether the effect varies with different forms of ocular abnormality. 
Table 1.
Published Estimates of Test–Retest Variability Using Charts Based on Bailey and Lovie’s Design
Table 1.
Published Estimates of Test–Retest Variability Using Charts Based on Bailey and Lovie’s Design
Author 95% TRV Range* Test Sessions (n) Subjects Refractive Correction
Elliott and Sheridan 3 ±0.07 2 Normal Full
van den Brom et al. 13 ±0.08 1 Cataract Full
Elliott and Sheridan 3 ±0.09 2 Cataract Full
Arditi and Cagenello 1 ±0.09 2 Normal Full
Bailey et al. 2 ±0.10 1 Normal Full
Rosser et al. 14 ±0.11 1 Normal Full
Lovie-Kitchen 4 ±0.16 1 Normal Unaided
Rosser et al. 7 ±0.18 1 Cataract Habitual
Reeves et al. 15 ±0.19 2 Mixed Full
Table 2.
Mean Differences between Paired Acuity Measurements and their 95% Ranges by Degree of Optical Defocus
Table 2.
Mean Differences between Paired Acuity Measurements and their 95% Ranges by Degree of Optical Defocus
Degree of Optical Defocus (D) Mean Difference (logMAR) SD Observed 95% TRV Range Predicted 95% TRV Range* (Assuming Normality)
0.00 0.002 0.051 −0.100 to +0.100 ±0.10
+0.50 −0.013 0.093 −0.200 to +0.295 ±0.18
+1.00 −0.004 0.124 −0.279 to +0.259 ±0.24
Arditi A, Cagenello R. On the statistical reliability of letter-chart visual acuity measurements. Invest Ophthalmol Vis Sci. 1993;34:120–129. [PubMed]
Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinical grading and the effects of scaling. Invest Ophthalmol Vis Sci. 1991;32:422–432. [PubMed]
Elliott DB, Sheridan M. The use of accurate visual acuity measurements in clinical anti-cataract formulation trials. Ophthalmic Physiol Opt. 1988;8:397–401. [CrossRef] [PubMed]
Lovie Kitchin JE. Validity and reliability of visual acuity measurements. Ophthalmic Physiol Opt. 1988;8:363–370. [CrossRef] [PubMed]
Raasch TW, Bailey IL, Bullimore MA. Repeatability of visual acuity measurement. Optom Vis Sci. 1998;75:342–348. [CrossRef] [PubMed]
Reeves BC, Wood JM, Hill AR. Reliability of high- and low-contrast letter charts. Ophthalmic Physiol Opt. 1993;13:17–26. [CrossRef] [PubMed]
Rosser DA, Laidlaw DA, Murdoch IE. The development of a “reduced logMAR” visual acuity chart for use in routine clinical practice. Br J Ophthalmol. 2001;85:432–436. [CrossRef] [PubMed]
Siderov J, Tiu AL. Variability of measurements of visual acuity in a large eye clinic. Acta Ophthalmol Scand. 1999;77:673–676. [CrossRef] [PubMed]
Sloan LL. Needs for precise measures of acuity: equipment to meet these needs. Arch Ophthalmol. 1980;98:286–290. [CrossRef] [PubMed]
Vanden Bosch ME, Wall M. Visual acuity scored by the letter-by-letter or probit methods has lower retest variability than the line assignment method. Eye. 1997;11:411–417. [CrossRef] [PubMed]
Ferris FL, Kassoff A, Bresnick GH, Bailey I. New visual acuity charts for clinical research. Am J Ophthalmol. 1982;94:91–96. [CrossRef] [PubMed]
Bailey IL, Lovie JE. New design principles for visual acuity letter charts. Am J Optom Physiol Opt. 1976;53:740–745. [CrossRef] [PubMed]
van den Brom HJ, Kooijman AC, Blanksma LJ, van Rij G. Measurement of visual acuity with two different charts: a comparison of results and repeatability in patients with cataract. Doc Ophthalmol. 1995;90:61–66. [CrossRef] [PubMed]
Rosser DA, Cousens S, Murdoch IE, Fitzke FW, Laidlaw DA. How sensitive to clinical change are ETDRS logMAR visual acuity measurements?. Invest Ophthalmol Vis Sci. 2003;44:3278–3281. [CrossRef] [PubMed]
Reeves B, Wood J, Hill A. Vistech VCTS 6500 charts, within- and between-session reliability. Optom Vis Sci. 1991;68:728–737. [CrossRef] [PubMed]
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–310. [PubMed]
Beck RW, Moke PS, Turpin AH, et al. A computerized method of visual acuity testing: adaptation of the early treatment of diabetic retinopathy study testing protocol. Am J Ophthalmol. 2003;135:194–205. [CrossRef] [PubMed]
Lovie Kitchin JE, Brown B. Repeatability and intercorrelations of standard vision tests as a function of age. Optom Vis Sci. 2000;77:412–420. [CrossRef] [PubMed]
Carkeet A, Lee L, Kerr JR, Keung MM. The slope of the psychometric function for Bailey-Lovie letter charts: defocus effects and implications for modeling letter-by-letter scores. Optom Vis Sci. 2001;78:113–121. [PubMed]
Finney DJ. Probit Analysis. 1971; 3rd ed. Cambridge University Press Cambridge, UK.
Humphriss D, Woodruff EW. Refraction by immediate contrast. Br J Physiol Opt. 1962;19:15–20. [PubMed]
Ray WA, O’Day DM. Statistical analysis of multi-eye data in ophthalmic research. Invest Ophthalmol Vis Sci. 1985;26:1186–1188. [PubMed]
Recommended standard procedures for the clinical measurement and specification of visual acuity. Report of working group 39. Committee on vision. Assembly of Behavioral and Social Sciences, National Research Council, National Academy of Sciences, Washington, D. C. Adv Ophthalmol. 1980;41:103–148. [PubMed]
Kempf GA, Collins SD, Jarman BL. Refractive errors in the eyes of children as determined by retinoscopic examination with a cycloplegic. 1928; United States Public Health Service Washington, DC. Publ. No. 182
Crawford JS, Shagrass C, Pashby TJ. Relationship between visual acuity and refractive error in myopia. Am J Ophthalmol. 1945;28:1220–1225. [CrossRef]
Carkeet A. Modeling logMAR visual acuity scores: effects of termination rules and alternative forced-choice options. Optom Vis Sci. 2001;78:529–538. [CrossRef] [PubMed]
Kitchin JE, Bailey IL. Task complexity and visual acuity in senile macular degeneration. Aust J Optom. 1981;64:235–242.
Levene H. Robust tests for equality of variances. Mann HB eds. Contributions to Probability and Statistics: Essays in honor of Harold Heotelling. 1960;278–292. Stanford University Press Stanford, CA.
Armatage P, Berry G. Statistical Methods in Medical Research. 1994; Blackwell Oxford, UK.
Blackhurst DW, Maguire MG. Reproducibility of refraction and visual acuity measurement under a standard protocol. The Macular Photocoagulation Study Group. Retina. 1989;9:163–169. [CrossRef] [PubMed]
Reeves BC, Harper RA, Hill AR, Wood JM. Is performance variability useful clinical information?. Doc Ophthalmol. 1991;77:128–129.
Mon-Williams M, Tresilian JR, Strang NC, Kochhar P, Wann JP. Improving vision: neural compensation for optical defocus. Proc R Soc Lond B Biol Sci. 1998;265:71–77. [CrossRef]
Webster MA, Georgeson MA, Webster SM, et al. Neural adjustments to image blur. Nat Neurosci. 2002;5:839–840. [CrossRef] [PubMed]

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.