Abstract
purpose. To compare the results from manifest refraction using trial lenses and a
standard visual acuity protocol to results from autorefraction for
obtaining refractive error and best corrected visual acuity in patients
enrolled in a randomized clinical trial.
methods. During a 4-month period, 29 patients with subfoveal choroidal
neovascularization (CNV), who were enrolled in the Submacular Surgery
Trials (SSTs) Pilot Study at the Wilmer Ophthalmological Institute,
gave verbal consent to participate in this study. Best corrected visual
acuity was obtained using Early Treatment Diabetic Retinopathy Study
(ETDRS) visual acuity charts and standardized room lighting after
performance of manifest refraction, according to the SST protocol, and
autorefraction. Refractive error (spherical equivalent) and visual
acuity scores were obtained in both eyes of all patients.
results. On average, manifest refraction gave a spherical equivalent that was
1.04 D more plus than autorefraction (95% limits of agreement =
0.74, 1.34). On average, the visual acuity score was 1.5 letters better
after manifest refraction than after autorefraction (95% limits of
agreement = 0, 3.0).The comparison of the two methods of
refraction was subdivided according to visual acuity level and eye
disease (age-related macular degeneration or ocular histoplasmosis
syndrome).
conclusions. Despite large differences in spherical equivalent between manifest
refraction and autorefraction, the visual acuity scores were close
(mean difference, 1.5 letters). Other studies comparing subjective
refraction and autorefraction have shown similar results.
Autorefraction in patients with subfoveal CNV may be a satisfactory
alternative to manifest refraction in clinical trials and field studies
in which best corrected visual acuity is of
interest.
Many ophthalmic clinical trials use best corrected visual
acuity as the primary outcome in the investigation of treatment
efficacy.
1 2 3 4 5 6 Methods used to obtain best corrected visual
acuity are either primarily subjective, such as manifest refraction
with trial lenses or a phoropter, or objective, such as retinoscopy or
autorefraction.
7 In this article we report a comparison of
manifest refraction with trial lenses, in accordance with a standard
protocol, with autorefraction for obtaining best corrected visual
acuity for patients enrolled in a randomized clinical trial.
The two refraction methods require different levels of examiner
education, training, and time to perform each procedure. Manifest
refraction requires a basic understanding of ophthalmic optics.
Typically, months of practical experience are needed for the clinician
to perform manifest refraction satisfactorily and reproducibly. In
clinical trials, manifest refraction typically is performed by
following a prescribed sequence of steps outlined in a study protocol
or manual of procedures. This technique must be practiced on many
patients with varying levels of visual acuity and types of refractive
error before it is mastered.
1 2 3 4 5 6 In contrast,
autorefraction does not require knowledge of ophthalmic optics or
practical experience in refraction. It requires only a basic
understanding of how to operate the autorefractor, which can be learned
from reading the instruction manual that comes with the autorefractor
and from minimal practice with patients.
8 9
Obtaining an objective refraction in a patient with reduced vision due
to refractive error usually takes only approximately 5 minutes per eye,
whereas subjective refraction in the same patient using the phoropter
or trial frames usually requires approximately 15
minutes.
10 In the experience of two authors (PRO, NMB),
refraction in a patient, with reduced vision and inability to fixate
centrally because of macular disease, performed with the autorefractor
or manifest refraction often requires more than 15 minutes. The
difference in time to perform both refraction techniques, whether in
patients with good or poor vision, becomes significant when large
numbers of patients are screened for inclusion in a study.
As an example, the Submacular Surgery Trials (SSTs), a set of
multicenter, randomized clinical trials, were designed to evaluate the
role of submacular surgery for treatment of subfoveal choroidal
neovascularization (CNV) secondary to age-related macular degeneration
(AMD), ocular histoplasmosis syndrome (OHS), and idiopathic causes.
Patients with AMD eligible for this trial must have visual acuity of
20/100 or worse in the study eye, and patients with CNV due to OHS or
idiopathic causes must have visual acuity of worse than 20/50 or worse
in the study eye.
1
Change in best corrected visual acuity is the primary outcome variable
of the SST. Therefore, all SST vision examiners are trained and
certified in manifest refraction and visual acuity testing as described
in the
SST Manual of Procedures. In addition, periodic
observation of the vision examiners at each participating clinical
center is necessary to ensure that all are performing in accordance
with the protocol.
1
Training and monitoring of vision examiners in large multicenter
studies is both time consuming and costly. In addition, manifest
refraction and visual acuity measurement in patients who have visual
acuity of 20/100 or worse with eccentric fixation and poor central
vision are very time consuming for the vision examiner. Autorefraction
was suggested as a possible solution to both problems.
This study was designed to compare manifest refraction and
autorefraction within a subset of patients participating in the SST
Pilot Study to determine whether the refractive errors and visual
acuity measurements from the two methods were similar, to estimate the
actual savings in time with autorefraction, and to identify situations
in which one method was preferable. One of the goals of the Pilot Study
was to evaluate the feasibility of various methods, such as
autorefraction, under consideration for the full-scale SST.
During a 4-month period, from July through October 1995, all SST
Pilot Study patients who were seen at the SST clinical center at the
Wilmer Ophthalmological Institute for follow-up examinations were
invited to participate in a comparison of the two refraction methods.
Written informed consent, approved by the Johns Hopkins Joint Committee
on Clinical Investigation (JCCI) and in accordance with the Declaration
of Helsinki agreement for research involving human subjects, was
obtained from all patients when they enrolled in the SST. The JCCI
approved the use of verbal consent to obtain autorefraction
measurements in a subset of SST participants at Johns Hopkins.
All refractions and vision measurements were performed by one
SST-certified vision examiner (PRO). A typical examination involved
manifest refraction followed by visual acuity measurement and
measurement of other aspects of vision with both the refraction and
acuity testing performed according to the SST protocol.
1 Refraction and visual acuity data then were recorded on a SST visual
acuity data form. Patients who agreed to autorefraction in addition to
routine manifest refraction had visual acuity remeasured after
autorefraction, again according to SST protocol. The data were recorded
on separate copies of the SST data form without reference to the
original measurement. Visual acuity testing for all patients was
performed in an SST vision examination lane where lighting conditions
were controlled.
1
The Lighthouse Distance Visual Acuity Charts and stand-mounted
illuminator box were used to determine the total number of letters read
correctly in each eye after manifest refraction and autorefraction.
These visual acuity charts contain 14 lines of letters, five letters
per line, with letter size progression that follows a log minimum angle
of resolution (MAR) scale.
11
To avoid affects of fatigue on measurement of visual acuity for the
SST, the protocol procedures always were performed first before
undertaking autorefraction. The major components of the SST manifest
refraction process consist of adjustment of spherical power, refinement
of cylinder axis and power, and refinement of spherical power. The SST
refraction protocol specifies that the dioptric power of the sphere and
cylinder for each step of the refraction procedure be increased with
decreasing levels of visual acuity
(Table 1) . The overall strategy of the SST refraction protocol was to“
push” plus power and add minus power only when objective
improvement was demonstrated. In this way, the highest plus and lowest
minus prescription would be obtained so that accommodation was
minimized.
1
In accordance with the SST protocol, best corrected visual acuity was
measured, beginning at 2.0 m. The Lighthouse Distance Visual
Acuity Chart 1 was used to measure vision for the right eye; and Chart
2, for the left eye. The lens correction obtained from the SST protocol
refraction for each eye was placed in the trial frames. The patient was
seated at a distance of 2.0 m from the chart, regardless of
whether the refraction was performed at 0.5 m, and instructed to
read the letters from left to right, beginning with the top line of the
chart and continuing to the smallest line. With the fellow eye well
occluded with the black occluder lens and tissue placed behind the
trial frames, the patient was encouraged to use eccentric fixation and
to guess.
When the patient read at least 15 letters correctly at 2 m, 30 was
added to the total number of letters read at 2 m to determine the
visual acuity score for that eye. Patients who read fewer than 15
letters on the chart at 2 m were positioned at 0.5 m from the
chart. A −0.75 sphere was added to the refraction obtained at 2 m. Again, the patient was asked to read the letters on the top line and
to proceed to the smallest line of letters. The visual acuity score was
then calculated as the total number of letters read at 0.5 m.
If the patient could not read any letters at 0.5 m, the visual
acuity score was recorded as 0, and the patient was checked for
light-perception vision using the indirect ophthalmoscope according to
the SST protocol.
1
After manifest refraction and best corrected visual acuity measurement,
the patient was escorted by the same examiner who performed manifest
refraction to another examination room to have autorefraction performed
on both eyes. Because of its convenience and availability, the AR-1600G
Auto Refractometer (Marco, Sunnyvale, CA) was used. The autorefractor
was checked at the beginning of the study and before each use to
confirm that it was operating properly.
Brief instructions were given to the patient while he or she was seated
at the autorefractor. The patient was asked to focus on the internal
target. In cases in which the patient could not fixate centrally
because of macular scarring or other lesions in the visual axis, the
patient was instructed to look straight ahead toward the sound of the
examiner’s voice.
Three independent measurements were taken for the right eye first. An
average of the three readings for each eye was automatically calculated
by the autorefractor at the end of the third reading.
After autorefraction, the patient returned to the SST visual acuity
examination room. The lens correction obtained from the average reading
for each eye was placed in the trial frames, and the left eye was well
occluded with both the occluder lens and tissue behind the trial
frames. In the manner described for manifest refraction, the patient
was seated at 2 m from the Lighthouse Distance Visual Acuity
Chart. To minimize possible memorization of letters, Chart 2 was used
to record visual acuity for the right eye and Chart 1 for the left eye.
From July through October 1995, 29 SST patients participated in
this study, for a total of 31 examinations on 62 eyes. (Two patients
were examined twice during this period.) Of the 62 eyes tested, 5 eyes
with reduced vision (from three patients) could not fixate for
autorefraction; thus, 57 pairs of refraction and visual acuity
measurements from 27 patients were analyzed to compare manifest
refraction and autorefraction. The 27 patients (including 10 men)
ranged in age from 28 to 83 years, with visual acuities ranging from
20/20 to 20/1600. Seventeen patients had AMD, and 10 had OHS.
The median spherical equivalent score obtained by manifest refraction
was 0.5 D, with scores ranging from −2.75 to 8.375 D. The median
spherical equivalent score obtained by autorefraction was −0.5 D, with
scores ranging from −5.625 to 3.625 D. A scatterplot of autorefracted
versus manifest refracted spherical equivalent scores is shown in
Figure 1B . Observations tended to fall near or below the line of equality,
indicating similar corrections from both methods or a more positive
correction from the manifest refraction, respectively.
Figure 1B is a plot of the difference between the spherical equivalent scores
obtained by the two methods versus the mean of the two scores. The
points are similarly distributed above and below the mean, and there is
no apparent pattern to indicate that the size of the differences is
related to the mean. Most of the differences are above 0, indicating a
more positive correction from manifest refraction.
Table 2 displays the mean difference, or bias, and the 95% limits of agreement
for this bias between refraction methods. On average, manifest
refraction gave a spherical equivalent that was 1.04 D more positive
than autorefraction (95% limits of agreement = −1.19, +3.24).
This difference was significantly different from 0 (
P < 0.001). Less than half (45.6%) of the spherical equivalent
measurement pairs differed by 1 D or less. Only 31.5% of the
measurement pairs differed by 0.5 D or less.
The median visual acuity score obtained by manifest refraction was 50
letters (Snellen equivalent 20/200), with scores ranging from 4 to 100
letters (20/1600–20/20). The median visual acuity score obtained by
autorefraction was 49 letters (20/200), with scores ranging from 10 to
100 letters (20/1280–20/20).
Figure 2A is a scatterplot of the
visual acuity scores obtained by autorefraction plotted against those
obtained by manifest refraction. The points generally cluster around
the line of equality.
Figure 2B displays the difference versus the mean
for visual acuity measurements. The points are reasonably scattered
around both the mean and 0, unlike spherical equivalent. On average,
the visual acuity score was 1.5 letters better after manifest
refraction than after autorefraction (95% limits of agreement, −9.7,
12.78;
Table 2 ). This bias estimate of 1.5 letters was not
significantly different from 0. Most pairs of measurements (82%) were
within five letters of each other, equivalent to one line on the ETDRS
(Early Treatment Diabetic Retinopathy Study) Visual Acuity Chart.
The mean difference in visual acuity scores was largest (4.1 letters,
95% limits of agreement = −9.5, 17.7) on eyes with good visual
acuity and smallest (−0.4 letters, 95% limits of agreement =−
10.6, 9.8) on eyes with poor visual acuity
(Table 3) . However, the mean difference in spherical equivalent between methods
was greater in eyes with poor visual acuity and lesser in eyes with
good visual acuity
(Table 3) .
Table 4 displays the bias between the two methods of refraction by disease (AMD
versus OHS). The mean difference in spherical equivalent was larger in
patients with AMD, whereas the mean difference in visual acuity score
was smaller in those with OHS.
Despite the relatively large differences in spherical equivalent
between the two refraction methods, visual acuity measurements were
very close across a large range of visual acuities (i.e., a mean
difference of only 1.5 letters). Other studies have shown similar
results. A study of the repeatability of ocular component measurements
of the right eyes of 40 patients aged 20 to 43 years compared
cycloplegic autorefraction, using the R-1 autorefractor (Canon, Lake
Success, NY), with cycloplegic subjective refraction and reported a
95% range of comparison of spherical equivalent between the two
methods to be ±1.10.
15 The population-based Beaver Dam
Eye Study used the 530 autorefractor (Humphrey, San Leandro, CA) to
obtain best corrected visual acuity in 4926 participants aged 43 to 83
years. The study reported that visual acuity scores obtained with
correction given by the Humphrey autorefractor were highly correlated
with scores obtained using the ETDRS protocol and logMAR charts,
although they were consistently higher by an average of 4.2
letters.
10
The issue of repeatability of results has not been addressed in this
study because replicate measurements for each method were not taken.
Bland and Altman have noted that agreement between two methods of
measurement may be limited by the repeatability of the methods, where
good repeatability is essential to good agreement.
12 13 Studies reported by Rosenfeld and Chiu
17 and Goss
and Grosvenor
18 have shown that replicate measurements
taken for subjective and objective measurements of refraction did not
reveal significant variability. The Beaver Dam Eye Study design
included five examiners, among whom no significant variation was found
in either refractive error or visual acuity
measurements.
10
Blackhurst et al.
14 performed a reproducibility study of
replicate visual acuity measurements after performing manifest
refraction in 164 eyes in 82 patients and reported an intraclass
correlation coefficient of 0.99 spherical equivalent and 0.99 for
visual acuity score, despite the finding that visual acuity and disease
process may affect the reliability of refraction and visual acuity
measurements. Patients included in this reproducibility study are
similar to SST patients in visual acuity, disease, and age. Our study
confirms that the agreement between methods of measurement varies
according to visual acuity level and disease. There was greater bias
between the methods in the measurement of spherical equivalent for
patients with poor vision and for patients with AMD. For visual acuity
measurement, the method bias was greatest for patients with AMD. For
visual acuity measurement, the method bias was greatest for patients
with good vision and for patients with OHS. This is consistent with the
findings stated by Blackhurst et al. that the diffuse disease process
of AMD may affect visual acuity level and/or measurement reliability.
Another analytical limitation of this study is that neither method of
refraction is considered the gold standard. It should be noted that the
SST refraction protocol is a modified and attenuated method of full
subjective refraction that was developed specifically for refraction in
patients with low vision. It has been time tested in previous
multicenter trials, particularly the Macular Photocoagulation Studies,
and it has been adopted by other prominent ophthalmic clinical
trials—for example, the Treatment of AMD with Photodynamic Therapy
trial and the Verteporfin in Photodynamic Therapy trial. Our results
reflect a comparison of one particular autorefractor with a specified
method of manifest refraction; thus, extrapolation of the results to a
full subjective refraction or to other autorefractor warrants further
study.
A design-related potential weakness of this study is the possible bias
on the data obtained by the vision examiner who performed all
refraction and vision measurements. The vision examiner (PRO) is a
certified SST vision examiner with approximately 23 years of refraction
experience. However, there was no conscious bias on the part of the
vision examiner toward patient selection or either type of refraction
method. All SST patients seen during the study period were invited to
participate in this study, regardless of visual acuity. In addition,
maximal effort to achieve the best results for both refraction methods
and vision measurements was given in an attempt to produce unbiased
data. Manifest refraction and vision measurement, in accordance with
the SST protocol, were always performed first to avoid possible
compromise of SST vision data due to the patient’s fatigue.
A report of the Beaver Dam Eye Study attributed its successful
recruitment for such a large study to reasonable participatory time
demands, minimized by the use of the Humphrey refractor which required
an average of 5 minutes per subject compared with the 15 minutes
required for manifest refraction defined by ETDRS standard
protocol.
10 The present investigation supports this
rationale. Except in cases in which the patient had extreme difficulty
in fixating centrally, the entire process of autorefraction of both
eyes averaged 10 minutes in our study. Therefore, the autorefractor can
reduce staff time required to obtain best corrections.
Autorefraction may be a satisfactory alternative to manifest refraction
in clinical trials and field studies for which measurement of best
corrected visual acuity is of interest, at least for patients who
undergo successful autorefraction. In our study of patients with
subfoveal CNV, approximately 10% of the patients with impaired central
vision were not able to fixate for autorefraction. Therefore, such
patients would require manifest refraction. A similar, but presumably
less frequent, problem has been reported in large population-based
studies.
16 17
Based on these findings, a more rigorous comparative study of these two
methods may be warranted to define more precisely the benefits and
limitations of autorefraction in multicenter studies of retinal and
macular diseases.
Supported by a Submacular Surgery Trials Pilot Study Grant, funded by the National Eye Institute and the Macular and Retinal Research Fund of the Wilmer Eye Institute.
Submitted for publication October 26, 1999; revised May 30 and September 26, 2000; accepted October 13, 2000.
Commercial relationships policy: N.
Corresponding author: Peggy R. Orr, Johns Hopkins University School of Medicine, 550 North Broadway, Suite 115, Baltimore, MD 21205-2010.
[email protected]
Table 1. SST Refraction Protocol Summary
Table 1. SST Refraction Protocol Summary
| Vision with Best Correction (Refraction Distance) | Sphere | | Cylinder | | | Sphere Refinement | |
| Power (a) | Increment | Axis (b) | Power (c) | Increment | Power (d) | Increment |
| 20/20–20/80 (2 m) | +0.50 | +0.50 | 0.50* | 0.25* | +0.25 | +0.37 | +0.25 |
| −0.37 | −0.25 | | | −0.25 | −0.37 | −0.25 |
| +0.50 | +0.50 | | | | +0.37 | +0.25 |
| <20/80–20/160 (2 m) | +1.00 | +1.00 | 1.00* | 1.00* | +1.00 | +0.50 | +0.50 |
| −1.00 | −1.00 | | | −1.00 | −0.50 | −0.50 |
| +1.00 | +1.00 | | | | +0.50 | +0.50 |
| 20/200–20/320 (2 m) | +2.00 | +2.00 | 1.00* | 1.00* | +1.00 | +1.00 | +1.00 |
| −2.00 | −2.00 | | | −1.00 | −1.00 | −1.00 |
| +2.00 | +2.00 | | | | +1.00 | +1.00 |
| <20/320 (0.5 m) | +2.00 | +2.00 | No cylinder power or axis adjustment | | | No refinement | |
| −2.00 | −2.00 | | | | | |
| +2.00 | +2.00 | | | | | |
Table 2. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements
Table 2. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements
| Measurement | Mean ± SD* | 95% Limits of Agreement, † |
| Spherical equivalent (manifest− autorefraction) | +1.04 ± +1.10 D | −1.19, +3.24 D |
| Visual acuity (manifest− autorefraction) | 1.5 ± 5.6 Letters | −9.7, 12.7 Letters |
Table 3. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements, by Visual Acuity Group
Table 3. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements, by Visual Acuity Group
| Measurement | Visual Acuity | Mean ± SD* | 95% Limits of Agreement‡ |
| Spherical equivalent (manifest − autorefraction) | Good | +0.63 ± 0.68 D* | −0.73, +1.99 D |
| Moderate | +0.91 ± 1.21 D* | −1.51,+3.33 D |
| Poor | +1.68 ± 1.08 D* | −0.48,+3.84 D |
| Visual acuity (manifest− autorefraction) | Good | 4.1 ± 6.8 Letters* | −9.5, 17.7 Letters |
| Moderate | 0.9 ± 4.2 Letters | −7.5, 9.3 Letters |
| Poor | −0.4 ± 5.1 Letters | −10.6, 9.8 Letters |
Table 4. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements, by Disease
Table 4. Difference between Refraction Methods for Spherical Equivalent and
Visual Acuity Measurements, by Disease
| Measurement | Disease | Mean ± SD | 95% Limits of Agreement |
| Spherical equivalent (manifest− autorefraction) | AMD | +1.19 ± 1.24 D* | −1.29, 3.67 D |
| OHS | +0.79 ± 0.77 D* | −0.75, 2.33 D |
| Visual acuity (manifest− autorefraction) | AMD | 0.66 ± 4.89 Letters | −9.12, 10.44 Letters |
| OHS | 2.86 ± 6.40 Letters | −9.94, 15.66 Letters |
Submacular Surgery Trials Study Group. Procedures of refraction and vision testing. Submacular Surgery Trials Manual of Procedures. June 1, 1998; National Technical Information Service Springfield, VA. Accession No. PB87-221834
Age-Related Eye Disease Study Group. Age-Related Eye Disease Study Manual of Procedures. February 1, 2000; Emmes Potomac, MD.
TAP Study Group. Visual acuity protocol. Treatment of AMD with PDT (TAP) Manual of Procedures. May 2, 1997; QLT Phototherapeutics; Ciba Vision Ophthalmics Vancouver, British Columbia, Canada; Bulach, Switzerland. Appendix 6
VIP Study Group. Visual acuity protocol Verteporfin in Photodynamic Therapy (VIP)
. December 19, 1997; QLT Phototherapeutics; Ciba Vision Ophthalmics Vancouver, British Columbia, Canada; Bulach, Switzerland. Appendix 6
Alcon Study Group. Alcon Study Manual of Procedures. August 1998;6 Alcon Research Fort Worth, TX.
Agouron Study Group. Agouron Study Manual of Procedures. June 3, 1998; Agouron Pharmaceuticals La Jolla, CA.
Michaels DD. Visual Optics and Refraction: A Clinical Approach. 1985;316:346–349. CV Mosby St. Louis. 3rd ed.
Strang NC, Gray LS, Winn B, Pugh JR. An evaluation of automated infra-red optometers. Optom Today. 1997;10:41–44.
Gray LS, Strang NC, Winn B, Pugh JR. The validity and reliability of autorefractor: an evaluation. Optician. 1996;212:22–24.
Klein R, Klein BEK, Linton KLP, DeMets DL. The Beaver Dam Eye Study: visual acuity. Ophthalmology
. 1991;98:1310–1315.
[CrossRef] [PubMed]Ferris FL, Kassof A, Bresnick GH, Bailey I. New visual acuity charts for clinical research. Am J Ophthalmol
. 1982;94:91–96.
[CrossRef] [PubMed]Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet
. 1986;1:307–310.
[PubMed]Altman DG. Some common problems in medical research. Practical Statitic for Medical Research. 1991; Chapman and Hall London.
Blackhurst DW, Maguire MG, The Macular Photocoagulation Study Group. Reproducibility of refraction and visual acuity measurement under a standard protocol. Retina
. 1989;9:163–169.
[CrossRef] [PubMed]Zadnick K, Mutti DO, Adams AJ. The repeatability of measurement of the ocular components. Invest Ophthalmol Vis Sci
. 1992;33:2325–2333.
[PubMed]Katz J, Tielsch JM, Sommer A. Prevalence and risk factors for refractive errors in an adult inner city population. Invest Ophthalmol Vis Sci
. 1997;38:334–340.
[PubMed]Rosenfield M, Chiu NN. Repeatability of subjective and objective refraction. Optom Vision Sci. 1995;72:577–579.
Goss DA, Grosvenor T. Reliability of refraction: a literature review. J Am Optom Assoc
. 1996;67:619–630.
[PubMed]