Multisampling Suprathreshold Perimetry: A Comparison with Conventional Suprathreshold and Full-Threshold Strategies by Computer Simulation

Abstract

purpose. To compare a multisampling suprathreshold strategy with conventional suprathreshold and full-threshold strategies in detecting localized visual field defects and in quantifying the area of loss.

methods. Probability theory was applied to examine various suprathreshold pass criteria (i.e., the number of stimuli that have to be seen for a test location to be classified as normal). A suprathreshold strategy that requires three seen or three missed stimuli per test location (multisampling suprathreshold) was selected for further investigation. Simulation was used to determine how the multisampling suprathreshold, conventional suprathreshold, and full-threshold strategies detect localized field loss. To determine the systematic error and variability in estimates of loss area, artificial fields were generated with clustered defects (0–25 field locations with 8- and 16-dB loss) and, for each condition, the number of test locations classified as defective (suprathreshold strategies) and with pattern deviation probability less than 5% (full-threshold strategy), was derived from 1000 simulated test results.

results. The full-threshold and multisampling suprathreshold strategies had similar sensitivity to field loss. Both detected defects earlier than the conventional suprathreshold strategy. The pattern deviation probability analyses of full-threshold results underestimated the area of field loss. The conventional suprathreshold perimetry also underestimated the defect area. With multisampling suprathreshold perimetry, the estimates of defect area were less variable and exhibited lower systematic error.

conclusions. Multisampling suprathreshold paradigms may be a powerful alternative to other strategies of visual field testing. Clinical trials are needed to verify these findings.

The full-threshold visual field test ¹ is currently regarded as a quasi-standard in perimetry. It is time consuming, and, particularly at damaged field locations, its threshold estimates are highly variable. ² The Swedish interactive test algorithm (SITA) standard (a more efficient threshold strategy based on maximum-likelihood estimation) has much shorter test times, but its variability is similar to that of the full-threshold strategy. ^{3 4 5} Threshold tests are demanding procedures, and many patients produce consistent results only after some training. ^{6 7 8 9} Many clinical applications, however, call for fast, simple, and reliable visual field tests that can be performed by patients with no training. 

In suprathreshold perimetry, stimuli are presented above the estimated detection threshold of a normal visual field location. If the patient responds, it is assumed that the corresponding test location does not have significant loss. In normal observers and those with early glaucoma, most stimulus presentations occur well above threshold, and the observer may be less uncertain of how to respond. Suprathreshold tests may therefore be easier tasks to perform with these patients, who often have little or no experience with perimetry. Although it is widely accepted that suprathreshold tests may be less sensitive to shallow visual field loss than threshold tests, they have often been used in epidemiologic screening ^{10 11 12 13} and are routinely used in primary eye care. ^{14 15} As with threshold perimetry, the results of conventional suprathreshold tests exhibit large test–retest variability in patients with glaucoma. ¹⁶  

In this study, we address the issue of the sensitivity and test variability in suprathreshold perimetry. Using probability theory, we examine the influence of the pass criterion (i.e., the number of stimuli that have to be seen for a test location to be classified as normal) on a suprathreshold test’s performance. We propose an optimized suprathreshold strategy, based on a pass criterion designed to reduce variability and improve sensitivity, without sacrificing specificity. The performance of this multisampling suprathreshold strategy in detecting visual field loss and quantifying its area is evaluated by computer simulation and compared with that of the full-threshold and conventional suprathreshold strategies. 

Methods

Evaluation of Pass Criteria Using Probability Theory

Classification curves were derived from the binomial distribution. They describe, for various pass criteria, the relationship between the probability of a stimulus being seen and the corresponding probability that the test location will be classified as defective. 

Equation 1 gives the function for obtaining exactly k responses to n stimuli (P _n(k)), given the probability p of a response. (The ! symbol stands for the factorial expansion.) The classification function (i.e., the probability of obtaining at least k responses to n stimuli (F _n(k)) is the cumulative form of this function (equation 2) .  

\[P_{n(k)}\ {=}\ \frac{n!}{k!(n\ {-}\ k)!}\ p^{k}(1\ {-}\ p)^{n{-}k}\]

 

\[F_{n(k)}\ {=}\ {{\sum}_{k}^{n}}P_{n(k)}\]

 

We derived classification curves for suprathreshold examinations with 1 to 5 stimulus presentations per test location (see the Results section). From several alternatives, we selected a pass criterion that demands three seen stimuli (of up to five presentations at each location, i.e., 3/5) for subsequent simulations. The suprathreshold strategy based on this criterion is referred to as a multisampling suprathreshold. 

Evaluation by Computer Simulation

Computer simulations were performed to derive measures of test performance for the multisampling suprathreshold strategy, as well as for two established strategies for comparison (described in the Perimetric Strategies subsection). 

The stimulus matrix used in all simulations corresponded to that of the 24-2 test of the Humphrey Field Analyzer (HFA; Zeiss-Humphrey Systems, Inc., Dublin, CA), excluding the two most nasal locations and that of the blind spot. At each of the 51 test locations, the average age-corrected sensitivity was calculated from normal control data (see the Patient Data section) to establish a reference field. Normal visual fields were simulated by varying the reference field through adding a constant value to the sensitivity at each location. This value was a random number from a normal distribution with a mean of 0.0 and a standard deviation of 1.5 dB. Point-wise normative limits for 5% pattern deviation probability were derived by simulating normal visual fields with the full-threshold strategy (empiric 5th percentiles of 10,000 simulated estimates). Pattern deviation was defined as the deviation of the test location relative to the reference field, corrected for overall elevation or depression (estimated by the 85th percentile of total-deviation values, i.e., the seventh least-defective location in the field). 

The simulation software was written in commercial software (Delphi, ver. 4; Borland Software Corp., Scotts Valley, CA). In simulating a visual field test, a perimeter unit would repeatedly pass references of a stimulus (i.e., location and intensity) to a patient unit, which would then return a “seen” or “not seen” response, based on sensitivity and response variability at the given location and on the likelihood of a false-positive or -negative patient error. 

The model for response variability used in the simulations has been described previously. ¹⁷ It was established from frequency-of-seeing (FOS) data, from which sensitivity (50%-seeing threshold, in decibels) and response variability (slope of FOS curve, standard deviation in decibels) parameters were estimated by probit analysis. The log-transformed response variability varied linearly with sensitivity, independent of stimulus eccentricity and age. 

Perimetric Strategies

Both suprathreshold strategies were threshold related—that is, the stimulus intensities were adapted to the general height (GH; defined as the 85th percentile of all sensitivity estimates in a visual field) of the simulated patient’s visual field, using an algorithm designed to give more precise estimates than that used in the suprathreshold programs of the HFA. ^{18 19} In brief, six stimuli were presented in a 1-dB up–down staircase at each of four midperipheral seed locations, and the GH was estimated from an average of the four last stimulus levels. To guard against the influence of localized and diffuse losses affecting the seed locations, only staircases with two or more reversals were averaged. The test intensity would be set to the age-setting if fewer reversals had occurred. The suprathreshold increment was 5 dB in both suprathreshold strategies. 

Multisampling Suprathreshold Strategy.

Based on the analysis of classification curves, a criterion of three seen or three missed stimuli was chosen to classify a location as normal or defective, respectively. This requires between three and five stimuli to be presented at each location (pass criterion 3/5). 

Conventional Suprathreshold Strategy.

At each location, a single stimulus is presented. If there is no response, the presentation is repeated, and locations are classified as defective only if both presentations were missed (pass criterion 1/2). 

Full-Threshold Strategy.

The full-threshold strategy ¹ is a staircase algorithm with a step size of 4 dB, reduced to 2 dB after the first response reversal. Sensitivity is estimated by the intensity of the last seen stimulus after the second reversal. Similar to the implementation in the HFA, our simulations commenced testing at four seed locations in the midperiphery of the visual field, and the remaining staircases were started 3 dB above or below the expected value (estimated as the average sensitivity at neighboring points). For consistency between the three strategies, we did not, however, repeat staircases when estimates differed by more than 4 dB from the expected value, and no double determinations were performed to estimate the short-term fluctuation. 

Patient Data

The three samples of visual field data used in this study were as follows: (1) Data from normal control subjects were used to establish a normal reference field of age-normal sensitivity estimates on which normal visual fields were modeled. (2) Test–retest full-threshold results from patients with glaucoma (complete glaucoma sample) were used to establish the validity of the response variability model on which our simulations are based. ¹⁷ (3) From the latter sample, we selected results with repeatable early to moderate defects (glaucoma subsample) as end points for simulations to investigate the sensitivity of the three strategies to increasing levels of visual field loss. 

Normal Control Sample.

This sample consisted of 109 full-threshold tests from 109 normal control subjects from a prospective study at the Department of Ophthalmology, Dalhousie University, Halifax, Canada. ²⁰ Inclusion criteria were normal findings in an ocular examination, visual acuity of 6/9 or better, a negative family history of glaucoma, intraocular pressure 19 mm Hg or less and previous experience with full-threshold perimetry. 

Complete Glaucoma Sample: Normal to End-Stage Glaucomatous Field Loss.

Test–retest full-threshold data from 190 glaucoma patients were collected at the Manchester Royal Eye Hospital. ²¹ Patients were included if the visual acuity was 6/18 or better, and if at least one full-threshold test had been performed earlier. A history of unreliable test performance did not lead to exclusion. Patients underwent two HFA 24-2 full-threshold tests, administered by the same examiner using the same instrument, during separate sessions within 2 weeks. Both eyes were examined in 152 patients and one eye in 38 patients, yielding 342 pairs of test–retest results. The mean deviation (MD) ranged from +2.6 dB to −28.9 dB (median, −3.0 dB). We ignored a small learning effect (test–retest MD difference, mean −0.4 dB; 95% confidence interval [CI; −0.5 to −0.2 dB]). 

Glaucoma Subsample: Early to Moderate Visual Field Loss.

A subsample of 113 test–retest pairs of 90 patients was selected from the complete glaucoma sample. Tests were included if, on both tests, the MD was better than −10.0 dB and the glaucoma hemifield test (GHT) result was either outside normal limits on both occasions (n = 71) or outside normal limits on one occasion and borderline or “general reduction of sensitivity” on the other (n = 42). The MDs ranged from +1.5 to −9.5 dB (mean, −4.2 dB). Figures 1a 1b 1c show examples of either end of this spectrum. 

Simulation Experiments

Simulation 1: Validating the Model.

To assess the validity of our simulations, we compared test–retest variability in the real data with that from simulated data. To simulate test–retest data, 5000 pairs of real test–retest visual fields were randomly selected, with replacement, from the complete glaucoma sample (normal to end-stage glaucomatous loss) and combined by averaging. For each of these true fields, two full-threshold tests were simulated to produce 5000 artificial test–retest pairs. We subsequently compared the 90% test–retest limits (5th and 95th percentiles of retest thresholds, stratified by baseline threshold) from real patient data and from simulated data. 

Simulation 2: Comparing the Tests’ Sensitivity to Localized Field Loss.

Each field of the subsample (averaged pairs with early to moderate field loss) was set to represent the end point (loss stage 1) of a deteriorating series of fields. The start field (loss stage 0) was a normal field with the same GH as the end point (loss stage 1). Between series, the GH of the fields varied in the same way as for the simulated normal fields from which the normative pattern deviation limits had been generated. Intermediate levels of loss (0.1, 0.2… 0.9) were calculated by linear interpolation between the sensitivities of the starting and ending fields. For each level of loss, we simulated 3000 test results with each strategy. Criteria for abnormality were set to give specificities above 95% when normal fields were simulated (Table 1) . 

An ideal scenario was derived from simulations during which the response variability was set to zero. Because the shape of the simulated normal fields did not vary, a field would be classified as abnormal as soon as the pattern deviation at any location exceeded 1 dB (limited only by the discrete steps of the measurement scale). 

Simulation 3: Estimating Defect Area.

Clustered visual field defects (depth, 8 and 16 dB), extending over 1 to 25 locations of the stimulus matrix, were superimposed on simulated normal visual fields. Areas of loss did not extend across the horizontal midline. For each defect size, the median and the 5th and 95th percentiles of the estimated number of defective test locations (pattern deviation probability beyond the 5% level) were derived from 1000 simulations. 

Results

Classification Curves

The classification curves show how different suprathreshold paradigms perform at various levels of response probability (Figs. 2a 2b 2c 2d 2e) . 

When a stimulus is presented at a normal test location, at a given suprathreshold increment above an arbitrarily defined normative detection threshold, it results in a response most of the time. In a defective area of the field, the probability of a response is lower, depending on the level of damage, the suprathreshold increment, and the observer’s psychometric function. Owing to the physiological response variability of the observer and because of occasional response errors (false-positive and -negative responses), the response probability varies asymptotically between 0 and 1. 

The performance of a near-ideal paradigm is illustrated by the family of classification curves in Figure 2a , derived from the binomial distribution with n = 1000 presentations and pass criteria (k) of 250, 500, and 750. At normal test locations, where the probability of a response is high, the likelihood of being classified as defective is virtually zero. However, at damaged test locations, where the probability of a response is lower than some arbitrary choice of cutoff, the classification curves rapidly approach 1. These curves suggest that the near-ideal paradigm provides for a highly sensitive and highly specific test. Its outcomes would be robust to large proportions of false-negative and -positive response errors and independent of the response variability of the visual field. Such performance, however, is far removed from that attainable in clinical perimetry, in which the number of responses collected is limited by the subject’s attention span, the time available for examinations, and the number of test locations. 

The diagonal lines in Figures 2a 2b 2c 2d 2e represent the classification curve of a suprathreshold test based on presentation of only a single stimulus (pass criterion 1/1). This pass criterion fully reflects the response variability of the visual field location (slope of the psychometric function), and it is equally affected by false-positive and -negative errors on the part of the patient. To avoid false-positive test results, the conventional suprathreshold strategy repeats presentations if a stimulus has been missed (pass criterion 1/2). The classification curve of this criterion (Fig. 2b , lower curve) is convex downward. Toward the right limit of the abscissa, the 1/2-curve runs very low, reflecting the fact that this criterion is fairly tolerant against occasional false-negative response errors. However, the 1/2 curve runs below the diagonal even at low response probabilities (toward the left limit of the abscissa, i.e., where genuine visual field loss is likely), illustrating that the tolerance to occasional lapses (specificity) is obtained at the cost of sensitivity. This particular tradeoff would be reversed if the 2/2 criterion were chosen (Fig. 2b , upper curve)—that is, by presenting two stimuli at each location and by flagging as defective any test location at which one or both presentations were missed. Such a test would have high sensitivity, but poor specificity. 

The classification curves steepen when more stimuli are presented, reflecting the inverse relationship between sample sizes and the variability of estimates (Figs. 2c 2d 2e) . Although several different pass criteria could have been chosen to increase the sensitivity beyond that achieved with the conventional suprathreshold criterion (e.g., 2/3, 3/4, 4/5), the 3/5 criterion was the most attractive. The lower extreme of its classification curve was almost indistinguishable from that of the 1/2 criterion (promising good specificity), whereas its upper part was substantially higher (promising better sensitivity). Although the classification curve of the 2/3 criterion appeared similar, its lower extreme ran a little higher, suggesting a slightly lower specificity (Fig. 2f) . We therefore chose to explore the performance of the 3/5 criterion in our simulations. 

Results of Simulation Experiments

Results of Simulation 1: Validating the Model.

Figure 3 shows the test–retest intervals of full-threshold perimetry, established both from real data and from simulated test results. The variability of full-threshold estimates is well documented. Whereas the intervals are moderate for high sensitivities, they increase considerably when sensitivity is lower. ^{2 5} Although the simulations tended to overestimate variability of high sensitivity estimates and to underestimate variability in the midrange, the good overall agreement between real and simulated intervals validated the response behavior model of our simulations. 

Results of Simulation 2: Comparing the Tests’ Sensitivity to Field Loss of Increasing Magnitude.

Simulation 2 explored the sensitivity of the three strategies to increasing levels of visual field loss. All strategies departed extensively from the ideal scenario (no response variability, no between-subject variability in the shape of the visual field; Fig. 4 ). 

At a loss stage of 0.5, nearly all fields had localized loss, but this was detected in less than two thirds by the multisampling suprathreshold and full-threshold strategies, and in only one third of cases by the conventional suprathreshold strategy. Throughout the range of defects evaluated, the full-threshold and multisampling suprathreshold strategies performed similarly; both strategies detected visual field loss earlier than the conventional suprathreshold strategy. 

Results of Simulation 3: Estimating Defect Area.

Figure 5 shows the variability (length of the reference intervals) and the systematic error (position of the median with respect to the diagonal) in estimates of defect area with the three strategies, for enlarging defects of constant depth. 

All strategies underestimated the area affected by shallow loss (8 dB, Figs. 5a 5b 5c ). With deeper defects (16 dB), the area of loss was estimated more accurately (Figs. 5d 5e 5f) . The multisampling suprathreshold strategy gave the smallest systematic errors. The variability of the estimates of all three strategies increased with the area of field loss. However, the estimates of multisampling suprathreshold perimetry had lower variability than those of the other strategies. 

Discussion

Throughout the range of defects evaluated, the multisampling strategy detected localized visual field loss with sensitivity similar to that of the full-threshold strategy and earlier than the conventional suprathreshold strategy (Fig. 4) . The pattern deviation plots of full-threshold perimetry underestimated the area of visual field affected by localized loss, similar to conventional suprathreshold perimetry. The multisampling suprathreshold technique gave less variable and more accurate estimation of defect area. 

The sensitivity of a diagnostic test is closely related to its variability in normal and abnormal populations. To achieve a high level of specificity, a variable test requires a strict criterion for what constitutes a positive result, which in turn reduces its sensitivity. The disparity between the performance curve of the ideal scenario (no response variability and no between-subject variability in the shape of the visual field) and those of the three strategies highlight the delay between the occurrence of visual field loss and its reliable detection by clinical tests. In a visual field test, abnormality is only then statistically detectable if it exceeds the noise level of variability inherent in psychophysical examinations. ²² Current techniques for analysis of threshold data, for example, rely on statistical significance testing to classify individual test locations for total and pattern deviation probability maps. The simulations show that this approach consistently underestimates the area of localized visual field loss. 

Threshold tests demand a high level of attention, are perceived as demanding by novice subjects, and usually require some level of training before reliable results are obtained. ^{6 7 8 9} Although it might be speculated that suprathreshold tests are somewhat easier for the patient to perform, the conventional suprathreshold strategy demands only a single stimulus to be seen for a location to be classified as normal. It is therefore highly sensitive to false-positive patient errors, and the small samples of responses collected lead to unacceptable test variability when fields are defective. ¹⁶  

The simulations support our hypothesis that, with increased sampling, a suprathreshold strategy may perform as well as a threshold test in detecting visual field loss and perhaps is better at quantifying its spatial extent. By sacrificing information on the depth of a defect (which is difficult to obtain, owing to the large response variability in damaged areas), multisampling suprathreshold strategies may obtain better information on its size and location. This performance gain with respect to the conventional suprathreshold technique comes at the cost of a larger number of stimulus presentations. However, most tested locations (whether normal or defective) will only require three presentations, and the largest number of presentations (5) of the 3/5-multisampling criterion sets an upper limit that compares well with modern threshold techniques. 

Simulations often simplify reality. Their simplicity confers both benefit and bane. A system may be studied quickly and in the absence of confounding factors, but the results may never reflect the entire complexity of real life. For example, our model of response variability did not account for the possible increase in variability between test sessions. Because it was based on constant-stimulus data of patients who had previously met the conventional reliability criteria (<33% false-positive and -negative response errors, <20% fixation losses), we may underestimate the variability in patients who respond poorly, or overestimate the variability inherent in suprathreshold paradigms. Furthermore, there were minor differences between the implementations of the full-threshold strategy in our simulation and in the HFA. However, the good agreement between simulated and real patient data suggests that our simulations account for most of the variability observed in real visual field tests and that different implementations of the full-threshold strategy do not greatly influence the results. Although several assumptions made for simulating normal and damaged visual fields (no variations in the shape of the visual field in normal subjects, purely localized visual field loss) are not met in clinical populations, ^{23 24} they were identical for all simulated strategies and are therefore unlikely to affect any comparison between them. 

Our simulations explored the detection and quantification of localized loss. Diffuse loss, even though it may be common in glaucoma, is not pathognomonic to the disease. By design, threshold-related suprathreshold paradigms are insensitive to diffuse visual field loss, because the test intensity is adapted (within given limits) to the estimated general height of the visual field. However, the GH estimate is itself an important index for diffuse reduction of visual field sensitivity. 

Computer simulations cannot replace clinical trials, but they are well suited to precede and complement them. Our results indicate that multisampling suprathreshold techniques may be a powerful alternative to other, established strategies of visual field examination. Clinical trials are now needed to verify these results with real patients. 

 Supported by the Wellcome Trust, Manchester Royal Eye Hospital Endowment Funds, and the National Health Service Research and Development Levy.

 Submitted for publication October 8, 2002; revised November 16, 2002; accepted January 10, 2003.

 Disclosure: P.H. Artes, None; D.B. Henson, None; R. Harper, None; D. McLeod, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

Examples of visual field data in the subsample with early to moderate visual field loss used in simulation 2. The HFA Statpac MD values are given at the top of each diagram. (a, b) Examples of early field loss; (c, d) the more advanced end of the spectrum.

Table 1.

 

Criteria for Abnormality for Multisampling Suprathreshold, Conventional Suprathreshold, and Full-Threshold Strategies, Derived as the 95th Percentile of Simulated Normal Fields

Strategy	Criterion for Abnormal Test Result	Specificity (%)
Multisampling suprathreshold	>2 locations classified as defective	97.4
Conventional suprathreshold	>2 locations classified as defective	98.5
Full-threshold	>1 location with PDP <5%	95.6

Figure 2.

 

Classification curves for suprathreshold paradigms. (a) A family of near-ideal classification curves (derived with 1000 stimulus presentations) approaching step-like shape. (b) The two possible classification curves for two stimulus presentations: the lower curve represents the 1/2 criterion of the conventional suprathreshold strategy. (c–e) Curves for three, four, and five presentations, respectively. For comparison, the curves of the 1/2, 2/3 (dotted line), and 3/5 criteria are shown alongside each other in (f).

Figure 3.

 

Point-wise 90% test–retest intervals (TRIs) for full-threshold perimetry. For each threshold level of the initial test, the filled circles indicate the 5th and 95th percentiles of the distribution of retest values established from patient data. The lines indicate the 5th and 95th percentiles established from simulations. For an initial threshold estimate of 30 dB, the estimate of a subsequent test is likely to fall between 32 and 26 dB approximately 90% of the time (TRI from patient data; simulated TRI, 32–24 dB). For an initial estimate of 10 dB, the patient data TRI extends from 0 to 24 dB (simulated CI, 0–22 dB).

Figure 4.

 

Proportion of abnormal test results against stage of loss for the ideal scenario and the full-threshold, multisampling suprathreshold, and conventional suprathreshold strategies. At loss stage zero, the entire population of visual fields is normal, and the ordinate values reflect the rate of false-positive decisions (Table 1) . Because the limits for abnormality were derived not from real but from simulated normal fields, the absolute proportions may not represent real-world performance.

Figure 5.

 

Systematic error and variability in measures of defect area with the three strategies. For the full-threshold strategy, estimated defect area is represented as the number of test locations with significant (P < 5%) pattern deviations—for the suprathreshold strategies, by the number of test locations classified as defective. Systematic errors manifest as departures of the median values from the diagonal line; variability is indicated by the length of the 90% prediction intervals. In the absence of response variability, the intervals would collapse onto the diagonal line.

The authors thank Adrian Faulkner for providing the repeat full-threshold visual field data from patients with glaucoma, and Balwantray C. Chauhan for providing visual field data from normal control subjects enrolled in a longitudinal study at the Department of Ophthalmology, Dalhousie University, Halifax (supported by Grant MT-11357 from the Canadian Institutes of Health Research).