Performance of Efficient Test Procedures for Frequency-Doubling Technology Perimetry in Normal and Glaucomatous Eyes

Abstract

purpose. To validate the clinical performance of two new efficient threshold-estimation procedures for frequency-doubling technology (FDT) perimetry in both visually normal individuals and patients with glaucomatous visual field loss.

methods. Forty-one normal subjects (mean age, 48.3 ± 11.6 years) and 50 patients with glaucomatous visual field loss (mean age, 72.7 ± 10.0 years) were tested. Some of these participants were retested within a 3-month period. FDT perimetry was performed on a color monitor driven by a visual-stimulus–generating video board, with stimulus parameters designed to closely mimic those of the commercial FDT test. Visual field sensitivity was measured using three procedures: a modified binary search (MOBS) identical with the one used in the commercial FDT device, a rapid efficient binary search (REBS), and a procedure the uses Bayesian methods (zippy estimation of sequential testing; ZEST). The selection of optimum parameters for REBS and ZEST were based on results from previous simulations.

results. Both ZEST and REBS were 40% to 50% faster than MOBS. All three methods produced similar visual field sensitivity measures, with 95% of the differences occurring between ±2 dB for normal subjects and ±3 dB for glaucoma patients. Test–retest performance was similar for all three procedures.

conclusions. The test time for full-threshold FDT perimetry can be approximately halved, by using either the ZEST or REBS procedure, without affecting the accuracy or reliability of the measurements. These findings in normal subjects and patients with glaucoma provide clinical confirmation of our previous investigations of these test strategies that use computer simulation.

The goal of perimetry is to establish accurate and efficient estimates of visual field sensitivity. Ideally, test time should be kept to a minimum, with visual field sensitivity estimates having a predictable amount of reproducibility within acceptable limits for clinical diagnostic evaluation. To address these needs, recent proprietary test strategies such as the Swedish interactive test algorithm (SITA) have been successfully applied to standard automated perimetry. ^{1 2 3 4} The SITA procedure is based on a procedure that uses Bayesian methods for threshold estimation, similar to that used for routine psychophysical testing. ^{5 6 7} Recent studies have demonstrated that SITA is able to reduce testing time by 40% to 50%, whereas maintaining the same accuracy and reliability obtained using a conventional 4-2-2 dB staircase. ^{2 3 4}  

Frequency doubling technology (FDT) perimetry is a recently developed method for determining glaucomatous visual field loss. ^{8 9} The frequency-doubling effect occurs when a low spatial frequency grating (less that 4 cyc/deg) undergoes counterphase flicker at a high temporal rate (greater than 15 Hz), which results in the grating’s appearing to have twice its original spatial frequency. ^{10 11} The commercially available FDT perimeter (Humphrey Systems, Dublin, CA, and Welch Allyn, Skaneateles, NY) determines contrast sensitivity measures for detection of sinusoidal gratings with a spatial frequency of 0.25 cyc/deg undergoing counterphase flicker at 25 Hz. Stimuli (10° by 10°, except for a 5° diameter central circular target) are presented at 17 (C-20 pattern) or 19 (N-30 pattern) locations throughout the central visual field, by using a modified binary search (MOBS) algorithm. It takes approximately 5 minutes per eye to complete full-threshold testing with the current MOBS strategy. FDT perimetry has been shown to compare favorably with standard automated perimetry, both in normal observers, patients with glaucomatous visual field loss, and patients with neuro-ophthalmic disorders. ^{12 13 14}  

The commercial FDT full threshold perimetry test gives fast outcomes, primarily because of the smaller number of locations evaluated in comparison to standard automated perimetry. However, the use of efficient test strategies should reduce testing time for FDT perimetry even further. Recent computer simulations performed in our laboratory indicate that optimized test strategies could produce 40% to 50% reductions in test times for FDT perimetry without affecting the accuracy or reproducibility of the sensitivity measures. ¹⁵ In particular, a procedure using Bayesian methods known as zippy estimation of sequential testing (ZEST) ⁵ and an optimized version of the MOBS procedure ¹⁶ referred to as rapid efficient binary search (REBS) were both capable of supplying the equivalent information provided by MOBS in 40% to 50% less time. 

The purpose of the present study was to determine the clinical validity of our computer simulation predictions and to compare the performance characteristics of REBS and ZEST against the current MOBS test strategy. The accuracy, reproducibility, and efficiency of all three test procedures were examined in a clinical population comprising visually normal subjects and patients with glaucomatous visual field loss. 

Methods

Subjects

One eye of 41 normal control subjects (mean age, 48.3 ± 11.6 years) and of 50 patients with glaucomatous visual field loss (mean age, 72.7 ± 10.0 years) was tested. To be included in the study, control subjects were required to have normal findings in an eye examination, refractive errors of less than ±6.00 D sphere and less than 3.00 D of astigmatism, a best corrected visual acuity of 20/40 or better, no evidence of ocular disease or surgery, and no history of diabetes or other systemic diseases, and they could not be taking medications known to affect visual field sensitivity or contrast sensitivity. Normal control subjects also had an IOP of less than 21 mm Hg and normal visual fields when tested using the Humphrey Field Analyzer 24-2 full-threshold test strategy. Patients with glaucoma patients were required to have a clinical diagnosis of primary open-angle glaucoma, a history of IOP greater than 22 mm Hg before treatment, and a glaucomatous visual field loss on at least two prior occasions as established with a Humphrey Visual Field Analyzer 24-2 or 30-2 threshold procedure in the eye to be tested (glaucoma hemifield test [GHT] or corrected pattern standard deviation [CPSD] indices worse than the 5% probability level). Before testing, all subjects provided written informed consent in accordance with a protocol approved by the Legacy Health Systems Institutional Review Board and in accordance with the tenets of the Declaration of Helsinki. 

Stimuli

FDT perimetry was emulated using a visual-stimulus–generating video board (VSG 2/4; Cambridge Research Systems, Kent, UK) displayed on a γ-corrected 21-in. color monitor (Trinitron GDM-500PS; Sony, Tokyo, Japan). The monitor was operated at a frame rate of 100 Hz, with a mean background luminance of 52 cd/m². The stimulus properties were designed to mimic, as closely as possible, those of the commercially available FDT perimeter (Humphrey Systems and Welch Allyn). Seventeen stimuli were presented: four per visual field quadrant plus one in the central macular region. Each stimulus was a 10°-by-10° square that contained a 0.25 cyc/deg sinusoidal grating undergoing 25-Hz counterphase flicker. This represents a minor departure from the commercial FDT stimulus display, which uses a 5° diameter round stimulus for the central target, but because our comparison considered threshold outcome, this departure did not influence our interpretation. The total stimulus duration was 720-ms, which included a 160-ms interval in which the stimulus contrast was ramped up to the test contrast, a 400-ms period at the test contrast, and a 160-ms ramp down to zero contrast. The up and down ramps were included to avoid temporal transients. An interstimulus interval of 1 second was allowed. One of the 17 visual field locations was randomly selected for presentation, with the stipulation that no location could be selected twice in succession. 

Test Strategies

The three threshold estimation procedures mentioned earlier were compared: MOBS, REBS, and ZEST. The MOBS procedure is identical with the one used by the commercial FDT perimetry full-threshold test, whereas the REBS and ZEST procedures are the optimal binary search and maximum likelihood procedures, respectively, that were derived from our prior studies, using computer simulation. ¹⁵ It should be noted that in this study, all sensitivity values are presented in terms of contrast sensitivity in decibels—that is, contrast sensitivity = [log(1/contrast threshold) · 10]. Contrast sensitivities are thus represented within a 0- to 20-dB range. The commercial FDT perimeter reports contrast sensitivities that have been scaled by eccentricity-dependent weighting factors, to more closely correspond to the values obtained for full-threshold testing of standard automated perimetry using the Humphrey Field Analyzer. 

Modified Binary Search.

The MOBS test procedure is described in detail by Tyrrell and Owens. ¹⁶ Briefly, it begins with a range of possible thresholds that sets the absolute upper (highest possible value) and absolute lower (lowest possible value) limits. These values are“ pushed” onto the upper and lower “stacks,” respectively. In the case of FDT perimetry, stimulus contrast is varied in equal logarithmic steps from 1% (20 dB sensitivity) to 100% (0 dB sensitivity) contrast. A stimulus equal to the midpoint between the upper and lower stack values is selected for presentation. If the stimulus is seen, then this stimulus value provides a modified lower limit for threshold, and a new midpoint is selected for the next stimulus presentation. If the stimulus is not seen, then the stimulus value sets the upper limit, the range is narrowed, and a new midpoint is selected for the next stimulus presentation. This process continues until a criterion number of response reversals (“seen” to “not seen” and vice versa) have occurred and the difference between the upper and lower stack values is equal to or less than a specified interval. Threshold is defined as the midpoint between the upper and lower limits when both of these criteria have been met. A more detailed description of the MOBS procedure has been previously presented. ¹⁵  

The MOBS strategy incorporates two additional conditions. One involves a check for response errors. Because MOBS is designed to rapidly converge on the threshold value, seen and not-seen responses should alternate. Any time that two seen responses are made in a row, the algorithm presents a stimulus at the upper stack value. If this stimulus is not seen, then MOBS continues in its normal manner. If the stimulus at the upper limit is seen, this value is “popped” from the upper stack and is replaced by the previous upper stack value held in memory. The next stimulus presentation is thus the new midpoint between the modified upper and lower stack values. Similarly, two consecutive not-seen responses elicit a check of the lower limit. In this manner, MOBS can quickly recover from response errors and make large jumps to remain close to the correct location of threshold. The upper and lower limits thus function as last-in, first-out (LIFO) stacks. The second additional MOBS condition involves stimuli that are at the upper or lower limits. If stimuli at these limits are not seen on two successive presentations, the procedure is terminated, and the boundary limit (0 or 20 dB) is returned as the end point. 

The MOBS termination criteria for the commercial FDT perimeter requires four response reversals and an interval between the upper and lower limits within 3 dB (using the contrast sensitivity definition in this investigation). 

Rapid Efficient Binary Search.

The REBS strategy is based on MOBS but its termination criteria require two response reversals (upper- to lower-limit interval also within 3 dB). Our computer simulations have indicated that these criteria provide improved efficiency without compromising accuracy or reproducibility. ¹⁵  

Zippy Estimation of Sequential Testing.

The ZEST procedure ⁵ has been described in detail elsewhere. ^{5 15 17} In brief, a probability density function (pdf), which describes the relative likelihood of threshold values within the population, is assumed for thresholds before test commencement. At all times, the contrast of the stimulus is set to the mean of the pdf, because this is the most likely value of the threshold. The patient responds to the stimulus with the pdf modified by a “likelihood” function. The likelihood function describes the uncertainty in threshold and should reflect the patient’s frequency-of-seeing curve (sometimes referred to as the psychometric function) when threshold is finally achieved. The shape of this likelihood function has little bearing on the final threshold, although it influences the confidence interval around that value. We chose to use a likelihood function that reflected normal sensitivity and to terminate testing when the pdf reached a predefined SD or after a certain number of presentations. Our previous computer simulations have indicated that this approach is reasonable and that the parameters described in the following paragraph provide the best efficiency without compromising accuracy or reproducibility. ¹⁵  

Seventeen pdfs (one for each stimulus location) were used. The pdfs at each location were derived from a combination of a histogram of normal thresholds and a weighted histogram of depressed thresholds from a glaucomatous population. ¹⁵ The potential advantages of using a combined pdf have been discussed by Vingrys and Pianta. ¹⁷ The general appearance of the pdfs is trimodal, as shown in Figure 1a . 

Figure 1b shows the likelihood function used in the ZEST procedure. The chance of seeing a stimulus presented at threshold is 50% (as shown by the dotted lines). A stimulus 1 dB either side of threshold has a 75% and 25% chance of being seen. The chance of seeing a stimulus 2 dB or more removed from threshold is 99% or 1%. The procedure was terminated when the SD of the pdf returned a maximum of 1 dB. 

All the calculations in the ZEST procedure were performed on discrete functions. That is, a pdf consisted of a list of 21 probabilities: one for 0 dB, one for 1 dB, and so on through to 20 dB. Similarly, the likelihood function was defined for whole-number thresholds in the range 0 to 20 dB. The mean of the pdf to be presented at each stage was rounded to the nearest 0.1 dB. 

Testing Protocol

All subjects participated in a testing session approximately 30 minutes in duration. One eye of each patient was selected randomly for testing. Equal numbers of right and left eyes were tested. During the visit, three FDT perimetry tests were performed, using the three threshold algorithms: ZEST, MOBS, and REBS. Rest periods were allowed between tests as required by the subjects, and all subjects were provided with similar instruction and practice before the commencement of testing. The order of the three algorithms was randomized and counterbalanced across the study groups to minimize the possible confounding of learning and fatigue effects. 

All subjects were invited to return for a second visit to measure retest performance. A subset (30/41 normal subjects and 22/50 patients with glaucoma) agreed to participate and were retested at a subsequent visit. For the normal group, retest data were collected on all subjects who returned within 175 days of their initial test (mean, 19.9 ± 32.9 days). For the glaucoma group, retest data were collected within 116 days of the initial test (mean, 41.1 ± 36.8 days). The three tests were repeated in the same order and on the same eye as the initial test. 

Results

Test Efficiency

In Figure 2 , the mean number (±SD) of presentations (Fig. 2a) , test duration (Fig 2b) , and average time per presentation (Fig. 2c) for each procedure are shown. Inspection of Figure 2a reveals that both REBS and ZEST required approximately 40% fewer presentations than MOBS. There was no statistically significant difference between the mean number of presentations required by REBS or ZEST for testing subjects with glaucoma (paired t-tests, P = 0.30). However, on average, ZEST used 8% fewer presentations than REBS in the normal subject group (paired t-test, P = 0.02). 

Examination of Figure 2b shows that the reduction in test time afforded by ZEST for normal subjects was greater than predicted by the savings in presentation numbers and reflects a peculiarity of the method of stimulus timing. Because stimulus presentation was terminated whenever a button was pressed, the procedure with the most “yes” responses recorded a faster time, because the presentation window was aborted on response. The shape of the pdfs used with the ZEST procedure (positive skew toward lower thresholds) means that normal subjects saw many of the stimuli and thus returned a faster test time. This observation is supported by the data in Figure 2c , which show that the average time per presentation for ZEST was less than that for either MOBS or REBS in the normal control subjects. 

Test Accuracy

The thresholds estimated by the three methods are compared in Figure 3 , which displays the difference in thresholds measured by the three procedures. If the threshold value returned by two methods is identical, it is represented as 0 dB in Figure 3 . The differences were determined for each subject on a point-wise basis, so that 17 points represent the field data for each individual in Figure 3 . Data are presented as a box plot, with the edges of the box showing the 25th and 75th quantiles and the whiskers representing the 10th and 90th quantiles. Data outside the 10th and 90th quantiles are represented by filled symbols. It is apparent that similar threshold outcomes were achieved by all three test methods. Ninety-five percent of the differences lay within ±2 dB in normal subjects and ±3 dB in patients with glaucoma. 

Test Reproducibility

Figure 4 shows the mean (±SD) absolute difference found between test and retest for each strategy. Data have been used from all 17 test locations for each individual, as in Figure 3 . All three test strategies performed similarly and were not statistically different from each other (one-way repeated-measures ANOVA: normal subjects P = 0.78; patients with glaucoma P = 0.84). 

Figure 5 shows representative data from a normal control subject for each of the three FDT threshold estimation procedures. For comparison, the Humphrey Field Analyzer test results in this subject are also presented. Note that similar results were obtained for each of the FDT test procedures and that all compared favorably with the Humphrey Field Analyzer results. Similar findings in a patient with moderate glaucomatous visual field loss are presented in Figure 6 . Results for each of the three FDT procedures are comparable and correlate well with the Humphrey Field Analyzer results. The greatest difference between the thresholds measured for the three techniques occurred in the subject with advanced field loss shown in Figure 7 . This may be expected, because perimetric variability is increased with greater amounts of visual field loss. ¹³ However, even in this patient there was good comparability among the results of three FDT test procedures and good correlations with Humphrey Field Analyzer results. 

Discussion

The purpose of exploring new threshold algorithms for FDT perimetry is to identify a test strategy that is just as accurate and reproducible as the existing strategy, yet can be completed in a shorter test time. True accuracy cannot be directly assessed within a clinical population, because the real thresholds are unknown. Therefore, an assessment of clinical accuracy involves determining whether the results differ from those obtained with the existing test strategy. Estimates of true accuracy can be made, however, using computer simulation, as we have done in our previous investigation. ¹⁵ Test algorithms also must be acceptably robust to accommodate the unreliable responses that are often encountered within a clinical setting. Using computer simulation, we identified two test strategies (REBS and ZEST) that appeared to fulfill these criteria. In this study, we have demonstrated that these strategies indeed reduced the test time by approximately 40% in a clinical population of patients, while maintaining the accuracy and reliability of the measurements. 

The reduction in test time afforded by either ZEST or REBS has several potential advantages. Shorter test times reduce patient fatigue. This may result in fewer unreliable responses and may increase the patient’s comfort and cooperation. Decreasing the number of presentations required to achieve threshold at each location also makes it possible to assess more locations within an acceptable test time. The commercially available FDT perimeter tests 17 or 19 locations; however, more efficient test strategies provide the means for measuring a greater number of locations such as the Humphrey Field Analyzer 24-2 test pattern. ¹⁸ Further increases in test efficiency may be achieved by applying neighborhood logic or predictive algorithms during testing. ¹⁷ The ZEST and REBS strategies assessed in this study did not include such logic, because in a 17-location test the benefits are likely to be minimal. This may not be the case when large numbers of test locations are used. 

The ZEST procedure adopted in this study made use of an initial pdf that contained a combination of glaucomatous and normal thresholds. ¹⁷ As discussed by Vingrys and Pianta ¹⁷ the initial pdf can be further biased by an examiner’s intuition about the subject’s disease status. The possibility for further improvements in efficiency of the ZEST procedure are even greater at follow-up, because the most appropriate pdf for each location of the visual field now becomes that returned at the last examination. This not only assures enhanced testing efficiency but may also afford a method for the early detection of progression in visual field loss. 

FDT perimetry has already been shown to have high specificity and sensitivity for the detection of glaucomatous visual field loss. ⁹ Furthermore, FDT perimetry has been demonstrated to have different test–retest variability characteristics than conventional perimetry, which may be advantageous in the detection of progression. ^{13 19} The application of these faster test strategies means that thresholds can be achieved at 17 locations in approximately 3.0 to 3.5 minutes. Such rapid testing, in conjunction with the high specificity and sensitivity of FDT perimetry makes it an excellent choice for the detection of visual field loss. 

 Supported in part by National Eye Institute Grant EY03424 (CAJ), an unrestricted research support grant from Welch Allyn, and a University of Melbourne Collaborative Grant (AJV).

 Submitted for publication June 21, 2001; revised October 15, 2001; accepted November 21, 2001.

 Commercial relationships policy: C, F (CAJ); N (all others).

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked“ advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

(a) An example of the initial pdfs used for the ZEST procedure. Pdfs were derived by combining a histogram of normal thresholds with a weighted distribution of glaucoma thresholds. (b) The likelihood function used in the ZEST procedure with the 50% seeing point aligned with the mean of the pdf in the upper panel; left: “no” likelihood function; right: “yes” likelihood function.

Figure 2.

 

Comparison of the efficiency of the three test strategies in (a) number of presentations, (b) test time, and (c) average time per presentation. Data are expressed as the mean ± SD.

Figure 3.

 

Box plot of the differences in thresholds among the three test strategies at the initial visit. The box represents the 25th, 50th (median), and 75th quantile, whereas the whiskers show the 10th and 90th quantiles. Solid symbols: data outside the 10th and 90th quantile range.

Figure 4.

 

The absolute difference between the test and retest thresholds for each of the test strategies. Data are expressed as the mean (black bars) and SD (white bars).

Figure 5.

 

An example of the FDT perimetry data collected from the left eye of a subject with normal vision. Humphrey Field Analyzer (24-2 test pattern, SITA thresholding algorithm) raw thresholds and corresponding gray-scale plots are presented for comparison.

Figure 6.

 

An example of the FDT perimetry data collected from the left eye of a subject with moderate glaucomatous visual field loss. Humphrey Field Analyzer (24-2 test pattern, full-threshold algorithm) raw thresholds and corresponding gray-scale plots are presented for comparison.

Figure 7.

 

An example of the FDT perimetry data collected from the left eye of a subject with advanced glaucomatous visual field loss. Humphrey Field Analyzer (30-2 test pattern, full-threshold algorithm) raw thresholds and corresponding gray-scale plots are presented for comparison.