Properties of Perimetric Threshold Estimates from Full Threshold, SITA Standard, and SITA Fast Strategies

Abstract

purpose. To investigate the distributions of threshold estimates with the Swedish Interactive Threshold Algorithms (SITA) Standard, SITA Fast, and the Full Threshold algorithm (Humphrey Field Analyzer; Zeiss-Humphrey Instruments, Dublin, CA) and to compare the pointwise test–retest variability of these strategies.

methods. One eye of 49 patients (mean age, 61.6 years; range, 22–81) with glaucoma (Mean Deviation mean, −7.13 dB; range, +1.8 to −23.9 dB) was examined four times with each of the three strategies. The mean and median SITA Standard and SITA Fast threshold estimates were compared with a “best available” estimate of sensitivity (mean results of three Full Threshold tests). Pointwise 90% retest limits (5th and 95th percentiles of retest thresholds) were derived to assess the reproducibility of individual threshold estimates.

results. The differences between the threshold estimates of the SITA and Full Threshold strategies were largest (≈3 dB) for midrange sensitivities (≈15 dB). The threshold distributions of SITA were considerably different from those of the Full Threshold strategy. The differences remained of similar magnitude when the analysis was repeated on a subset of 20 locations that are examined early during the course of a Full Threshold examination. With sensitivities above 25 dB, both SITA strategies exhibited lower test–retest variability than the Full Threshold strategy. Below 25 dB, the retest intervals of SITA Standard were slightly smaller than those of the Full Threshold strategy, whereas those of SITA Fast were larger.

conclusions. SITA Standard may be superior to the Full Threshold strategy for monitoring patients with visual field loss. The greater test–retest variability of SITA Fast in areas of low sensitivity is likely to offset the benefit of even shorter test durations with this strategy. The sensitivity differences between the SITA and Full Threshold strategies may relate to factors other than reduced fatigue. They are, however, small in comparison to the test–retest variability.

The Full Threshold strategy is currently regarded as a standard technique in static threshold perimetry and is used in most glaucoma-related clinical trials. The principles of this staircase algorithm are simple. Stimuli are presented at a predetermined matrix of test locations, starting at intensity levels determined from a normative data set. The stimulus intensity is varied in steps of 4 dB until the first response reversal occurs and subsequently in steps of 2 dB. With the Humphrey Field Analyzer (Zeiss-Humphrey Instruments, Dublin, CA), the stimulus intensity of the last-seen presentation is taken as the final threshold estimate, after a second response reversal has occurred at a given location. The properties of the Full Threshold strategy have been thoroughly investigated, both by computer simulation and in clinical studies, and its limitations are well understood. ^{1 2 3} At damaged locations, the test–retest variability of single threshold estimates is nearly equivalent to the dynamic range of the instrument. ³ This makes it difficult to distinguish real change from random fluctuation, hampering the detection of visual field progression. With test durations often in excess of 15 minutes, Full Threshold examinations are demanding and time-consuming procedures. 

The recently introduced Swedish Interactive Threshold Algorithms (SITA) have enabled large reductions in examination time compared with the Full Threshold algorithm. For the examination of the central 30° of the visual field, SITA Standard reduces the test time by up to 50%. ⁴ SITA Fast examinations are even briefer. ⁵ The large reductions in test time are achieved by adapting the interstimulus interval to the patient’s response speed, by alternative estimation of false-positive response rates, which does not require catch trials, and, most importantly, by reducing the number of stimulus exposures through more efficient threshold estimation based on Bayesian principles. ⁶ Whereas the stimulus intensities are varied according to simple staircase rules (similar to those of the Full Threshold strategy), thresholds are estimated by a maximum-likelihood technique, making use of information that is already available before the test is started and relying on assumptions about the shapes of the frequency-of-seeing curves and on the spatial correlation between neighboring field locations. At each test location, the patient’s responses are combined with a priori probability density functions derived from samples of normal and glaucomatous visual fields, resulting in two a posteriori functions. The largest value of either a posteriori distribution is interpreted as the most likely threshold, and the width of the distribution gives a measure of the uncertainty about this estimate at any given time throughout the visual field test. At each location, testing is stopped as soon as the uncertainty has been reduced beyond a predetermined limit referred to as the error-related factor. The final threshold estimate is computed as that stimulus intensity with the largest likelihood of being detectable during 50% of presentations. 

The large time savings achieved with the SITA strategies have motivated the trend for them to replace the Full Threshold algorithm in clinical practice and in glaucoma research. Several research groups have shown good qualitative agreement between test results obtained with SITA Standard and SITA Fast and those obtained with the Full Threshold strategy. ^{5 7 8 9} Previous publications have also reported on the average differences between the threshold estimates of the SITA algorithms and those of the Full Threshold strategy and on the lower global test–retest variability with the SITA strategies. ¹⁰ Compared with the Full Threshold strategy, however, the SITA algorithms are mathematically complex procedures. A more detailed examination of the statistical properties of their threshold estimates is necessary for a full understanding of their performance. The detection of visual field progression, for example, relies on the accuracy and reproducibility of threshold values from individual visual field locations. In this study, we investigate how the threshold differences between the strategies depend on sensitivity and whether these effects are explained by reduced patient fatigue with the briefer SITA examinations. To enable a clinically intuitive comparison of threshold reproducibility, we derived pointwise retest intervals from a set of four examinations with each of the three strategies, obtained within a 4-week period in patients with stable glaucoma. Because the visual fields of these patients are unlikely to have changed, this interval describes the range within which estimates from subsequent retest sessions are likely to fall, for any given threshold level at the initial test. An estimate outside the retest interval would therefore be interpreted as evidence of likely change. 

Methods

Patients

Visual field data were obtained from patients with glaucoma who were attending the Tajimi Municipal Hospital (Gifu, Japan). The study adhered to the tenets of the Declaration of Helsinki and was approved by the ethics committee of Tajimi Municipal Hospital. Forty-nine consecutively recruited patients (38 women, mean age, 61.6 years; range, 22–81) with glaucomatous visual field loss (mean deviation, mean, –7.1 dB; range, +1.8 to −23.9 dB; Fig. 1 ) attended four sessions at 1-week intervals, during which one randomly selected eye was examined with a Humphrey Visual Field Analyzer (HFA; model 740, Zeiss-Humphrey Instruments) program 30-2. At each session, one Full Threshold, SITA Standard, and SITA Fast test were completed in random order. All patients had prior experience with static perimetry and met the conventional criteria for reliable performance (rates of false-positive and false-negative response errors <33%, fixation error rate <20%) on each of the 12 visual field tests. 

Analysis

All threshold data were transformed to a right-eye format and analyzed with custom-designed software. Data from the blind-spot locations were excluded from the analysis. 

Comparison of Threshold Estimates

Mean sensitivities (MS, in dB) were calculated for each visual field test to establish the magnitude of any order effects (differences between the four sessions) as well as the average sensitivity differences between the three strategies. To establish whether the threshold differences between the three strategies depended on sensitivity, we compared single threshold estimates of SITA Standard and SITA Fast to the mean threshold estimate of three Full Threshold examinations (referred to as the “best available” estimate of sensitivity). For example, the SITA Standard threshold estimates of session 1 were grouped according to the average Full Threshold estimates of sessions 2, 3, and 4 at the same test locations. 

All four possible combinations of sessions were used. The best available estimate was computed from three, rather than from all four, Full Threshold tests, so that the Full Threshold strategy itself could be subjected to the same analysis to examine the impact of statistical artifacts arising from regression-to-the-mean effects (which cause the retest values of very high and very low initial estimates to lie closer to the average) and from the truncated measurement range of conventional perimetry on our analysis. Subsequently, the mean and median SITA Standard and SITA Fast estimates were derived for each level of sensitivity (0–38 dB in steps of 2 dB). 

To investigate whether fatigue explains the differences between the Full Threshold estimates and those of the SITA strategy, this analysis was repeated for a subset of 20 test locations that are examined early during the course of a Full Threshold test, including only the four seed locations (13° from the fixation point along the 45°, 135°, 225°, and 315° meridians) and their closest neighbors. 

Comparison of Test–Retest Variability

Global root-mean-square (RMS) errors were calculated to compare the test–retest variability of the three strategies in individual patients. Pointwise RMS errors were plotted against average sensitivity to investigate the relationship between sensitivity and test–retest variability with each strategy. Two-sided, empiric 90% retest limits were derived to give a clinically intuitive description of the pointwise test–retest variability between the initial and the retest sessions with each of the three strategies. The intervals were derived as the 5th and 95th percentiles of the retest threshold distribution for each level of initial threshold, in steps of 2 dB. By treating the order of the four sessions as interchangeable, all 12 possible permutations of initial and retest examinations could be evaluated. 

Results

Although MS improved slightly between the 1st and 4th sessions with the Full Threshold and SITA Standard strategies, the order effect was not statistically significant (paired Hotelling t ² test, P > 0.28) and probably too small to be of clinical importance (Fig. 2) . The order of the four sessions was therefore considered to be interchangeable in the subsequent analyses. On average, sensitivity estimates of SITA Standard were 0.9 dB higher and those of SITA Fast 1.6 dB higher than those of the Full Threshold strategy (Wilcoxon test, P < 0.001). 

In areas of high sensitivity, the means and medians of SITA Standard and SITA Fast agreed closely with the best available estimates of the Full Threshold strategy. Systematic differences appeared when sensitivity was lower. These differences were slightly larger with SITA Fast compared with SITA Standard (Figs. 3b 3c) . An equivalent analysis of the Full Threshold data showed that these findings were not due to a statistical artifact of our analysis (Fig. 3a) . The differences between the strategies persisted when the analysis was repeated on data obtained in the subset of 20 locations that are tested earliest during the Full Threshold test—that is, the seed points of each quadrant and their four closest neighbors (Figs. 3d 3e 3f) . 

Examination of the individual threshold distributions at different levels of sensitivity further highlighted the distributional differences between the Full Threshold and the SITA strategies (Fig. 4) . With best available estimates of sensitivity between 28 and 32 dB, single Full Threshold estimates averaged at 29.4 dB, whereas the means of SITA Standard and SITA Fast were 30.0 dB (Wilcoxon test, P < 0.001). At this level of sensitivity, the threshold distributions of SITA Standard and SITA Fast were similar (Kolmogorov-Smirnov [KS] two-sample test, P > 0.10) and more symmetric than that of the Full Threshold strategy (Figs. 4j 4k) . 

With the sensitivity interval centered at 20 dB (best available estimates between 18 and 22 dB), the means of the Full Threshold, SITA Standard, and SITA Fast distributions were 20.6, 22.4, and 23.2 dB, respectively. The distributional differences between the three strategies were all highly significant (KS, P < 0.001). Although negative skew was apparent in the distribution of all three strategies, the distribution of the Full Threshold strategy appeared more symmetric than those of the SITA strategies (Figs. 4g 4h 4i) . 

With best available estimates between 8 and 12 dB, the most frequent single estimates of all three strategies were 0 dB (Figs. 4d 4e 4f) . The means of the single Full Threshold estimates was 10.5 dB, whereas those of SITA Standard and SITA Fast were 11.6 and 13.1 dB, respectively. The differences between the threshold distributions were all statistically significant (KS, P < 0.05). In areas of absolute visual field loss (best available estimates of sensitivity equal to 0 dB), the distributions of the Full Threshold and SITA Standard strategies were similar (KS, P > 0.10), whereas that of SITA Fast showed a longer tail at higher sensitivities (KS, P < 0.01; Figs. 4a 4b 4c ). 

Compared with the Full Threshold strategy, SITA Standard resulted in a reduction of global test–retest variability (RMS error) in 40 (82%) of 49 patients, whereas SITA Fast resulted in lower RMS errors in 32 (65%) patients (Fig. 5) . In an individual patient comparison, SITA Standard reduced the average RMS test–retest variability by 15%, (Wilcoxon test, P < 0.001), whereas SITA Fast reduced the global RMS error, on average, by 7% (Wilcoxon test, P = 0.03), compared with the Full Threshold strategy. 

With all three strategies, the test–retest variability of individual threshold estimates varied strongly with the sensitivity of the visual field location. Locations with high sensitivities showed much lower test–retest variability than those with lower sensitivities (Figs. 6 7) . Owing to the limited dynamic range of the instrument, the RMS measures of test–retest variability decreased at sensitivities near zero. However, because of the long tails of the retest distributions, the reduction in the clinically more important 90% retest intervals was much smaller (Fig. 7) . 

At locations with sensitivities above 20 dB, SITA Standard and SITA Fast gave more repeatable threshold estimates than the Full Threshold strategy. With sensitivities below 20 dB, the test–retest variability of SITA Fast was larger than that of the Full Threshold strategy, whereas that of SITA Standard was similar, or lower (Fig. 6) . In comparison with the Full Threshold strategy, the empiric 90% retest intervals of both SITA strategies were smaller in the higher range of sensitivities. For lower levels of baseline sensitivities, SITA Fast showed poorer reproducibility, whereas the intervals of SITA Standard were similar to, or perhaps slightly smaller than, those of the Full Threshold strategy (Figs. 7a 7b) . 

Discussion

The results of our pointwise analyses show that the differences between the estimates of the Full Threshold strategy and those of the SITA strategies did not vary in a simple, linear fashion with sensitivity (Fig. 3) . Although the differences were close to zero at both extremes of the dynamic range, they reached a maximum of approximately 1.5 dB (SITA Standard) and 2.5 dB (SITA Fast) at approximately 15 dB. At this sensitivity, the estimates of SITA Standard and SITA Fast were significantly higher than those of the Full Threshold strategy. Evidenced by the disparities between the means and medians, the shapes of the underlying threshold distributions were also different from that of the Full Threshold strategy (Fig. 4) . 

In the implementation of the Full Threshold strategy in the HFA, sensitivity is estimated as the stimulus attenuation of the last-seen presentation, whereas the SITA algorithms derive the estimate as that stimulus attenuation with the largest likelihood of corresponding to the 50%-point on the frequency-of-seeing curve. ^{6 11} The SITA strategies therefore would be expected to give estimates that are, on average, 1 dB higher than those of the Full Threshold strategy, independent of sensitivity. ⁵ When averaged across the entire dynamic range, the sensitivity difference between SITA Standard and the Full Threshold algorithm (0.9 dB) was similar to the expected value, whereas the difference between SITA Fast and Full Threshold was larger. These results (Fig. 2) agree closely with those reported by others, ¹⁰ but the differences are smaller than those reported by Sharma et al., ¹² whose analysis, based on a comparison between one Full Threshold and one SITA Standard examination, may be confounded by a statistical artifact akin to a regression-to-the-mean effect. Locations with very low sensitivity in the first session tend to produce higher (i.e., more sensitive) estimates during the second examination, owing to test–retest variability and to the truncated range of the instrument. 

To reduce the effect of such statistical artifacts, we compared single estimates of each strategy against the mean of three Full Threshold examinations (referred to as the best available estimate). Although the Full Threshold strategy is not an ideal gold standard, its properties have been thoroughly investigated, both from clinical data ³ and by computer simulation, ^{1 2 13} and may therefore be more fully understood than those of the SITA strategies. The staircases of the Full Threshold strategy, for example, commence at values determined from the sensitivity at neighboring locations, or from a normative database if estimates from neighboring locations are not yet available. Because of response variability, the resultant threshold estimates are biased toward the start value if it is remote from the true sensitivity at the given location. ^{1 13} Although a better estimate of sensitivity may be obtained from frequency-of-seeing (FOS) curves, the number of stimulus presentations required to estimate FOS curves accurately is too large to be practical in a clinical context. Computer simulations, in which the true sensitivity of the observer is known, are the method of choice for investigations relating to the accuracy of psychophysical measurements. However, such simulations require precise details of SITA’s visual field model that are not in the public domain. 

Bengtsson and Heijl ⁵ have hypothesized that the larger than expected differences in the sensitivity estimates with the briefer SITA examinations (compared with the Full Threshold strategy) are due to reduced fatigue. Reduced fatigue effects do not, however, explain the higher than expected sensitivity estimates that they reported from computer simulations of the SITA Fast strategy. Furthermore, our findings persisted when the analysis was repeated on a subset of visual field locations including only the primary seed points and their closest neighbors. These locations are examined early during the course of a Full Threshold test and would therefore be expected to show lower fatigue effects than other test points. These findings question the hypothesis that the differences between the strategies are solely due to reduction in fatigue with the briefer SITA examinations. It has been reported from computer simulations that biased threshold estimates may result from using the mode of the a posteriori probability density function (such as in the SITA strategies), whereas its mean provides a better estimator. ¹⁴  

Because the magnitude of the bias is likely to be related to the size of the error-related factor (i.e., the permitted uncertainty about the threshold estimate), it may explain the differences between the estimates of SITA Standard and SITA Fast, as well as the results reported from computer simulations of the SITA Fast strategy by Bengtsson and Heijl. ⁵ This threshold estimation bias may also contribute to the paradoxical finding of lower between-subject variability with SITA Fast compared to SITA Standard. ^{15 16}  

The global test–retest variability of SITA Standard was approximately 15% lower, and its retest intervals were generally smaller, compared with those of the Full Threshold strategy. The small systematic differences between the SITA strategies and the Full Threshold algorithm are unlikely to impact on the detection of deep and localized defects that are regarded as the hallmark of glaucomatous field loss. Because global visual field indices, such as MD and pattern standard deviation (PSD), are calculated with reference to normative values of each strategy, good agreement between the indices of these strategies would be expected, and several previous reports have confirmed this. ^{4 7 8 9 10} It is difficult, however, to estimate how the SITA strategies may represent early diffuse losses of visual field sensitivity that commonly accompany focal glaucomatous defects. ^{17 18} Computer simulations and longitudinal clinical trials need to establish whether the small systematic differences between the Full Threshold strategy and SITA Standard, and the slightly higher reproducibility of the latter, impact on the detection of visual field progression. When SITA Standard is substituted for the Full Threshold algorithm in the longitudinal follow-up of patients, it may be advantageous to establish new baseline measures. ¹⁹ SITA Fast showed higher reproducibility only for high sensitivities. Because, at test locations with sensitivity below 20 dB, its test–retest variability was higher than that of the Full Threshold strategy, SITA Fast is unlikely to be a good choice to monitor established visual field loss, in spite of its short test duration. 

 Supported by Grant MT-11357 from the Canadian Institute of Health Research (BCC).

 Submitted for publication January 22, 2002; revised March 25, 2002; accepted April 9, 2002.

 Commercial relationships policy: N.

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

Scattergram of MD versus corrected PSD (CPSD) with the first Full Threshold test, illustrating the distribution of visual field loss in the sample of 49 patients.

Figure 2.

 

MS of each of the three strategies at each of the four sessions. Error bar, SEM.

Figure 3.

 

Mean and median estimate of sensitivity versus ‘best available’ estimates for Full Threshold (a, d), SITA Standard (b, e) and SITA Fast (c, f). (a–c) Data for all test locations (excluding those of the blind spot); (d–f) results of the same analysis on data from the subset of 20 test locations that are examined early during the Full Threshold tests (seed locations and their closest neighbors).

Figure 4.

 

Distributions of estimates of the three strategies for four levels of sensitivity. Gray bars: best available estimate (BAE, mean of three Full Threshold tests). The intervals shown are centered at (a–c) 0 dB (BAE = 0 dB); (d–f) 10 dB (8 < BAE ≤ 12 dB); (g–i) 20 dB (18 < BAE ≤ 22 dB); and (j–l) 30 dB (28 < BAE ≤ 32 dB).

Figure 5.

 

Patient-averaged RMS errors of SITA Standard and SITA Fast compared with those of the Full Threshold strategy.

Figure 6.

 

Average pointwise RMS errors against average sensitivity for SITA Standard, SITA Fast, and the Full Threshold strategy. Threshold data were binned in intervals of 5 dB, and data points were horizontally staggered for improved visibility. Error bars, SEM.

Figure 7.

 

Retest limits of SITA Standard (a) and SITA Fast (b). Thin vertical lines indicate the 90% retest intervals (5th and 95th percentiles of retest values); vertical bars indicate the interquartile ranges. The 90% retest limits of the Full Threshold strategy (solid lines) are shown for comparison.