How Many Visual Fields Are Required to Precisely Predict Future Test Results in Glaucoma Patients When Using Different Trend Analyses?

Abstract

Purpose.: To evaluate the minimum number of visual field (VF) tests required to precisely predict future VF results using ordinary least squares linear regression (OLSLR), quadratic regression, exponential regression, logistic regression, and M-estimator robust regression model.

Methods.: Series of 15 VFs (Humphrey Field Analyzer 24-2 SITA standard) were analyzed from 247 eyes of 155 open-angle glaucoma patients. Future point-wise (PW) VF results and mean VF sensitivities were predicted with varying numbers of VFs in each regression method.

Results.: In PW-OLSLR, as expected, the minimum absolute prediction error was obtained using the maximum number of VFs in the regression (14 VFs); mean absolute prediction error was equal to 2.4 ± 0.9 dB. Ten VFs were required to reach the 95% confidence interval (CI) of the minimum absolute prediction error. Prediction errors associated with the exponential and quadratic regression models were significantly larger than those from PW-OLSLR, whereas errors from logistic regression were not significantly smaller than those from PW-OLSLR; however, the absolute prediction error from the M-estimator robust regression model was significantly smaller than those associated with PW-OLSLR (P < 0.01, paired Wilcoxon test). Like PW-OLSLR, 10 VFs were needed to obtain the minimum absolute prediction error of mean VF sensitivity, but there were no significant differences in errors using the different regression methods.

Conclusions.: Approximately 10 VFs, are needed to achieve an accurate prediction of PW VF sensitivity and mean sensitivity. Prediction error of PW VF sensitivity can be significantly minimized using the M-estimator robust regression model compared with conventional OLSLR.

Glaucoma is the second leading cause of blindness in the world.¹ As glaucoma can lead to irreversible blindness, it is essential to accurately measure visual field (VF) progression so that physicians can treat patients with glaucoma accordingly, especially because medical and surgical IOP reduction interventions can be accompanied with a number of ocular and general complications.^2–6 It is therefore imperative to accurately predict future VF progression when making glaucoma treatment decisions. Visual field trend analyses can be used to measure the progression of global indices, such as mean deviation (MD) and the visual field index (VFI), or point-wise (PW) sensitivity using ordinary least square linear regression (OLSLR). This type of analysis is widely used in glaucoma clinics using clinical support tools such as PROGRESSOR (Medisoft, Ltd., London, UK).⁷ 

Detection of VF progression is hampered by VF variability. Visual field sensitivity fluctuates in the short-term⁸ and long-term.⁹ The reliability of VF measurements is inherently affected by a patient's concentration, but previous reports have suggested that measurement noise is considerable even with good reliability indices.^10,11 The ability of VF trend analyses, in particular PW linear regression (PLR), to detect progression will be significantly impeded by VF variability.¹² Consequently, the minimum number of VFs required to obtain reliable PLR results has been widely discussed in previous studies.^13–16 

Furthermore, OLSLR is highly sensitive to outliers. Attempts have been made to overcome this problem; for example, Gardiner and Crabb¹⁴ reported that omitting VFs helps to identify progression, whereas Hirasawa et al.¹⁷ suggested that the accuracy of PW-OLSLR can be improved by applying it to the average sensitivities of small sectors. Alternatively, robust linear regression models can be applied to mitigate the effects of outliers and improve the detection of progression. In the robust model, the weight of each data point in the regression is dependent on the size of its residual; thus, the model is more “robust” to the influence of outliers.¹⁸ 

The first objective of the current study was to investigate the minimum number of VFs required to accurately predict VF progression using OLSLR. The second purpose of this study was to estimate the usefulness of the M-estimator robust regression model (M-robust) for VF progression prediction as well as the exponential, quadratic, and logistic models, compared with OLSLR. 

Methods

This study was approved by the Research Ethics Committee of the Graduate School of Medicine and Faculty of Medicine at The University of Tokyo. Written consent was given by patients for their information to be stored in the hospital database and used for research. This study was performed according to the tenets of the Declaration of Helsinki. 

Subjects

This retrospective study included 247 eyes of 155 patients with primary open-angle glaucoma (mean ± SD age: 53.7 ± 11.9 years) from the glaucoma clinic at the University of Tokyo Hospital since 2002. All patients had a minimum of 16 reliable VF tests. An unreliable VF was defined as having more than 20% fixation losses or more than 15% false-positive errors. Only a patient's initial 16 VFs were analyzed when a patient had more than 16 VF test results. The first VF was excluded from the analysis to reduce the learning effect,^19,20 and the remaining 15 VFs were obtained over 7.5 ± 2.1 (mean ± SD) years of follow-up. Inclusion criteria for this study were patients with a visual acuity better than or equal to 6/12, refraction less than 5 diopters ametropia, no previous ocular surgery (except for cataract extraction), and no other posterior segment eye diseases. Patients with other ocular diseases that could affect VF sensitivity, such as diabetes mellitus retinopathy, corneal opacity, and AMD were excluded. Patients with cataract other than clinically insignificant senile cataract were excluded. All VFs were recorded using the Humphrey Field Analyzer (HFA; Carl-Zeiss Meditec, CA, USA), with the 24-2 or 30-2 test pattern and the SITA standard strategy with a Goldmann size III target. When the VF was measured using the 30-2 test pattern, only the 52 test points overlapping with the 24-2 test pattern were used for the analysis. 

Statistical Analysis

Using OLSLR, the PW VF sensitivities of the first (in approximately 6 months), second (in approximately 1 year) and third future (in approximately 1 and a half years) VFs: sixth VF (VF₆), seventh VF (VF₇), and eighth VF (VF₈) were predicted from the first five VFs (VF_1–5). Absolute prediction accuracy was calculated as the absolute value of the difference between the predicted and the observed PW sensitivities. In addition, the absolute prediction errors using the exponential, quadratic, logistic, and M-robust methods were also calculated. The process was iterated to predict the PW sensitivities of the seventh, eighth and ninth VFs (first, second, third future VF: VF₇, VF₈, and VF₉) using VF_1–6, the PW sensitivities of VF₈, VF₉, and VF₁₀ using VF_1–7, and so on, up to prediction of the PW sensitivities of VF_13, VF₁₄, and VF₁₅ using VF_1–12. The minimum prediction error was calculated for each regression method and the minimum number of VFs to reach the 95% confidence interval (CI) was identified. These analyses were also performed using the mean sensitivity of the entire VF. As a subanalysis, predictive accuracy was compared in eyes with early-stage glaucoma (initial mTD [mean of total deviation of all 52 total deviation values in the 24-2 HFA VF] larger than −6 dB) and eyes with moderate to advanced-stage glaucoma (initial mTD ≤ −6 dB). In addition, accuracy was evaluated in eyes with an mTD progression rate smaller than −0.25 dB per year and the remaining eyes, which is commonly considered to be the deterioration by pressure-independent damaging factors.^21–23 

Statistical Models

The following five regression models were used to make predictions, where y represents PW or mean VF sensitivity and x represents the time from the initial VF: 

#1 OLSLR: y = ax + b 

#2 exponential regression: y = e^ax+b 

#3 quadratic regression: y = ax² + bx + c 

#4 M-estimator robust linear regression²⁴:  

 

for the ith of n observations, the general M-estimator minimizes the objective function:  

 

where the function ρ gives the contribution of each residual to the objective function.¹⁸  

#5 Logistic regression:  

 

where α, β, and ζ are model parameters to be estimated.  

All models were fit using the statistical programming language R (R version 2.15.1; The Foundation for Statistical Computing, Vienna, Austria). The robust regression model was fit using the R package MASS. To compare the prediction error of each model, a Wilcoxon rank test was carried out. Statistical significance was set as less than 0.05, where Holm's method^25,26 was used to correct P values for the problem of multiple testing. 

Results

Patient Data

Patients' demographic information is shown in Table 1. The mean ± SD MD in VF₁ was −6.6 ± 5.3 (range, 1.1 to −25.8) dB and −8.7 ± 6.3 (range, 1.3 to −27.0) dB in VF₁₅. The mean ± SD interval between the first, second, and third future VFs was 5.9 ± 0.2 months, 11.9 ± 0.2 months, and 17.8 ± 0.2 months, respectively. 

Table 1

 

Subject Demographics

 

Prediction Error

When predicting the first, second, and third future VFs using PW-OLSLR, the minimum absolute prediction error was obtained using VF_1–14 (2.4 ± 0.9 [range, 0.6–5.9] dB: see Fig. 1A), VF_1–13 (2.6 ± 1.0 [range, 0.6–6.5] dB: Fig. 1B), and VF_1–12 (2.8 ± 1.2 [range, 0.6–9.1] dB: Fig. 1C), respectively. The absolute prediction errors associated with the exponential and quadratic regressions were significantly larger then OLSLR in all comparisons (paired Wilcoxon test, P < 0.05 after correction of P values for multiple testing using Holm's method^25,26). No significant improvement was observed in the prediction error by applying logistic regression compared with OLSLR (paired Wilcoxon test, P > 0.05 after correction of P values for multiple testing using Holm's method^25,26). The absolute prediction errors (mean ± SD) associated with the M-estimator robust regression were 2.3 ± 0.9 (range, 0.6–5.8) dB using VF_1–14, 2.5 ± 0.9 (range, 0.6–6.2) dB using VF_1–13, and 2.7 ± 1.1 (range, 0.6–9.0) dB using VF_1–12. These values were significantly smaller than those in other models, in all comparisons for first future VF prediction (paired Wilcoxon test, P < 0.001 except for VF_1–5: P = 0.010, after correction of P values for multiple testing using Holm's method^25,26), except for VF_1–5 for second future VF prediction (P < 0.001, except for VF_1–6: P = 0.022, VF_1–12: P = 0.0047), and except for VF_1–5 for third future VF prediction (P < 0.001 except for VF_1–11: P = 0.0018). In PW-OLSLR, the absolute prediction error for predicting the first, second, and third future VFs reached 95% CI of the minimum prediction error using VF_1–11, VF_1–10, and VF_1–10, respectively. 

Figure 1

 

Absolute prediction error associated with each regression model in PW sensitivity trend analysis. The plot represents the mean absolute prediction error associated with future VF PW sensitivity, using OLSLR, exponential regression, quadratic regression, and M-estimator robust regression. (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs). (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). OLSLR, ordinary least squares linear regression; Exp, exponential regression; Quad, quadratic regression; Logist, logistic regression. **P < 0.01 and *P < 0.05 between OLSLR and M-robust (paired Wilcoxon test after correction of P values for multiple testing using Holm's method^25,26). Gray line, 95% CI range of minimum prediction with OLSLR.

A total of 126 eyes were suffering from early-stage glaucoma (mTD ≥ −6 dB, Table 2). Figure 2 shows the comparison between the absolute prediction errors associated with OLSLR and M-estimator robust regression. Significantly smaller prediction errors were observed with M-estimator robust regression when predicting the first future VF using VF_1–10, VF_1–11, and VF_1–13 (P = 0.0071, 0.025, and 0.010, paired Wilcoxon test, Fig. 2A), and when predicting the second future VF using VF_1–9, V_1–10, and VF_1–11 (P = 0.015, 0.044, and 0.044, paired Wilcoxon test, Fig. 2B). No significant difference was observed in predicting the third future VF using VF_1–5 to VF_1–12 (P > 0.05, paired Wilcoxon test, Fig. 2C). In eyes with more advanced glaucoma (mTD ≤ −6 dB; 121 eyes, Table 2), significantly smaller prediction errors were observed when predicting the first future VF using VF_1–5 to VF_1–14 (VF_1–5: 0.025, VF_1–6: P = 0.0010, VF_1–7: P = 0.0014, and VF_1–8 to VF_1–14: P < 0.001, paired Wilcoxon test, Fig. 3A), predicting the second future VF using VF_1–6 to VF_1–13 (VF_1–6: 0.001, VF_1–7 to VF_1–13: P < 0.001, paired Wilcoxon test, Fig. 3B), but not for predicting the third future VF using VF_1–5 to VF_1–12 (P > 0.05, paired Wilcoxon test, Fig. 3C). 

Table 2

 

Subanalysis Subject Demographics

 

Figure 2

 

Absolute prediction error associated with OLSLR and M-estimator robust regression in PW sensitivity trend analysis of patients in early-stage glaucoma group (n = 126 eyes). (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs). (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). **P < 0.01 and *P < 0.05 between OLSLR and M-robust (paired Wilcoxon test). Gray line, 95% CI range of minimum prediction with OLSLR.

Figure 3

 

Absolute prediction error associated with OLSLR and M-estimator robust regression in PW sensitivity trend analysis of patients in moderate to advanced-stage glaucoma group (n = 121 eyes). (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs. (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). **P < 0.01 and *P < 0.05 between OLSLR and M-robust (paired Wilcoxon test). Gray line, 95% CI range of minimum prediction with OLSLR.

A total of 117 eyes had an mTD progression rate faster than −0.25 dB per year (Table 2). Figure 4 shows the comparison between absolute prediction errors associated with OLSLR and M-estimator robust regression. Significantly smaller prediction errors were observed with M-estimator robust regression when predicting the first future VF using VF_1–6 and VF_1–8 to VF_1–12 (P = 0.028, 0.0080, 0.029, <0.001, <0.001, 0.0020, paired Wilcoxon test, Fig. 4A), predicting the second future VF using VF_1–7 to V_1–11 and VF_1–13 (P < 0.001 except for VF_1–8: P = 0.0013 and VF_1–13: P = 0.0068, paired Wilcoxon test, paired Wilcoxon test, Fig. 4B), and predicting the third future VF using from VF_1–7 to VF_1–12 (P = 0.029, 0.044, 0.031, 0.0085, 0.049, 0.013, paired Wilcoxon test, Fig. 4C). In the remaining eyes with mTD progression rate equal to or slower than −0.25 dB per year (130 eyes, Table 2), significantly smaller prediction errors were observed when predicting the first future VF using VF_1–7 to VF_1–14 (VF_1–7: P = 0.0015, VF_1–8: 0.013, VF_1–9: 0.014, VF_1–7 to VF_1–14: P < 0.001, paired Wilcoxon test, Fig. 5A), predicting the second future VF using VF_1–9 to VF_1–13 (P values were <0.001, 0.0024, 0.0025, 0.0015, and <0.001, respectively, paired Wilcoxon test, Fig. 5B). No significant difference was observed in predicting the third future VF using VF_1–5 to VF_1–12 (P > 0.05, paired Wilcoxon test, Fig. 5C). 

Figure 4

 

Absolute prediction error associated with OLSLR and M-estimator robust regression in PW sensitivity trend analysis in eyes with mTD progression rate faster than −0.25 dB per year (n = 117 eyes). (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs). (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). **P < 0.01 and *P < 0.05 between OLSLR and M-estimator robust regression (paired Wilcoxon test). Gray line, 95% CI range of minimum prediction with OLSLR.

Figure 5

 

Absolute prediction error associated with OLSLR and M-estimator robust regression in PW sensitivity trend analysis of eyes group with mTD progression rate equal to or slower than −0.25 dB per year (n = 130 eyes). (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs). (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). **P < 0.01 and *P < 0.05 between OLSLR and M-estimator robust regression (paired Wilcoxon test). Gray line, 95% CI range of minimum prediction with OLSLR.

As shown in Figures 6A through 6C, the minimum absolute prediction error associated with OLSLR of mean VF sensitivity was obtained using VF_1–14 (2.4 ± 0.9 dB [0.004–5.3]) for predicting the first future VF, using VF_1–13 (2.6 ± 1.0 dB [0.002–5.2]) for predicting the second future VF, and using VF_1–12 (2.8 ± 1.2 dB [0.008–9.1]) for predicting the third future VF. There were no significant differences in the absolute errors associated with OLSLR, exponential regression, and M-estimator robust regression models at any time point. The absolute prediction errors with the quadratic regression model were significantly worse than the OLSLR method (paired Wilcoxon test, P > 0.05 after correction of P values for multiple testing using Holm's method^25,26). 

Figure 6

 

Absolute prediction error associated with each regression model (mean sensitivity). The plot represents the mean absolute prediction error when future VF mean sensitivity is predicted using OLSLR, exponential regression, quadratic regression, and M-robust. (A) The first future VF (from the 6th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 14th VFs). (B) The second future VF (7th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 13th VFs). (C) Third future VF (8th VF to the 15th VF) was predicted using prior VFs (1st to 5th VFs up to 1st to 12th VFs). Gray line, 95% CI range of minimum prediction with OLSLR.

Discussion

In this study, the relationship between the number of VFs and the prediction accuracy of VF test results was investigated. As expected, the prediction accuracy improved as the number of VFs in the regression increased. A considerable number (approximately 10 on average) of VFs were needed to saturate the prediction accuracy. Importantly, we observed no merit when using the exponential, quadratic, or logistic models, both in the PW and mean sensitivity analyses, compared with OLSLR. The M-robust regression model, however, enabled significantly better predictions than OLSLR for the PW analyses, although this improvement was not observed for mean sensitivity predictions. 

Many studies have investigated the prediction performance of linear and nonlinear regression models.^13,27–29 

Caprioli et al.²⁸ recently fitted the exponential regression model, the quadratic regression model, and the OLSLR model to series of VFs and suggested that the exponential model was the best-fitting because it obtained the smallest Akaike's Information Criterion (AIC); the authors argue that the exponential method better models VF sensitivity as it approaches the floor level of VF sensitivity (0 dB).^29,30 Nevertheless, the model with the smallest AIC does not guarantee it is always the best model for prediction accuracy.³¹ Indeed, in the current study, the exponential regression model was significantly worse in terms of prediction accuracy, compared with OLSLR in the PW analysis, in agreement with a previous study.³² One possible reason for this apparently contradictory result may be the difference in datasets analyzed. In our study, the average MDs of the initial and last VFs were −6.6 and −8.7 dB, respectively, whereas in the Caprioli et al.²⁸ article, the MDs were much worse: −10.9 and −12.9 dB, respectively. As a result, many more VFs will tend to reach the floor level in the previous study. It is possible that the “best” model depends on the length of the observation period and the stage of glaucoma. Nonetheless, the clinical usefulness of applying the exponential regression model and also the logistic regression model³³ instead of OLSLR, may be limited because it requires approximately 7 or 8 years to obtain 15 VFs under an assumption that VF measurements are carried out every 6 months.³⁴ 

Moreover, accurately predicting patients' VFs in the early to moderate stages of glaucoma is perhaps clinically more important than predictions in an advanced stage. 

Attempts have been made to improve the accuracy of OLSLR for VF trend analysis. Gardiner and Crabb¹⁴ recommended applying a “Three-Omitting” rule in which the last VF is omitted and instead two future VFs are used to detect and confirm progression. Over 6 years of follow-up, they showed that the standard PW-OLSLR method was sensitive for detecting VF progression with a sensitivity value equal to 97.1%; however, the specificity of the method was very low (25.4%). The “Three-Omitting” method, on the other hand, achieved a sensitivity of 65.7% with a specificity of 87.4%. Thus, the proposed method affords a significant improvement in specificity, but nevertheless is accompanied with a notable loss in sensitivity. It should be pointed out, however, that the “Three-Omitting” approach is not a method to improve the regression model itself and, furthermore, future VFs are always needed to diagnose progression; this is in contrast to the robust regression method used in the current study. 

Bengtsson et al.²⁹ investigated the usefulness of the initial five VFs for predicting future progression, using 11 VFs captured over 8.2 years on average. They showed that linear extrapolation of the five initial VF results offered a reliable predictor of future field loss in most patients. Until now, however, it has not been investigated how many VFs are needed to saturate the prediction accuracy. In the current study, longer series of VFs were investigated and it was shown that a considerable number (approximately 10 VFs taken over 5.1 ± 1.6 years) are required for this purpose. 

The absolute prediction error was significantly smaller using PW-OLSLR versus the exponential and quadratic models. There was no significant improvement of the prediction accuracy by using the logistic regression model, instead of OLSLR. Moreover, M-estimator robust regression obtained significantly smaller prediction errors than PW-OLSLR. Interestingly, this tendency was more profound in eyes with moderate to advanced-stage glaucoma (Figs. 2, 3). This is probably because test-retest reproducibility of measured VF sensitivity is poor in these eyes³⁵ and so the merit of reducing the effect of outliers in the M-estimator robust regression is greater. On the other hand, reducing the effect of outliers could be argued to affect the ability of robust regression to identify fast-progressing eyes. The current results, however, suggest that the prediction accuracy of M-estimator robust regression tended to be significantly better than those with OLSLR in such eyes (Figs. 4, 5). When predicting mean VF sensitivity, no significant differences were observed in prediction errors among the OLSLR, exponential, quadratic, and robust regression models. This is probably because outliers are much less frequent in mean sensitivity measurement compared with PW sensitivity. Indeed, there was no significant difference between the prediction errors associated with M-estimator robust regression and OLSLR when predicting mean VF sensitivity (Fig. 6). 

Despite the smaller prediction errors of M-robust regression, the same numbers of VFs were required as OLSLR to reach the 95% CI of minimum absolute error. Both regression models assume that the distribution of VF errors is normally distributed; however, the distribution is actually somewhat bimodal (due to truncation of VF sensitivity) and skewed.³⁶ This may have contributed to improve prediction accuracy in the robust regression method. Nonetheless, the improvement of the prediction accuracy is relatively small and it might have only marginal benefit at the clinical settings. There are recent articles that succeeded to improve the prediction accuracy by implementing clinical knowledge into the prediction model, such as clustering eyes with the progression patterns,^37–40 setting a penalty term in the regression model,⁴¹ and dividing VF into small sectors.¹⁷ A future study should be carried out to further improve the diagnostic accuracy by combining the merits of these approaches. 

In conclusion, a considerable number of VFs (approximately 10), are required to obtain accurate predictions of future VF test results; moreover, prediction error can be significantly improved by using the robust regression model when PW sensitivity is analyzed. 

Acknowledgments

Supported in part by the Japan Science and Technology Agency (JST)-CREST and Grant 26462679 from the Ministry of Education, Culture, Sports, Science and Technology of Japan. 

Disclosure: Y. Taketani, None; H. Murata, None; Y. Fujino, None; C. Mayama, None; R. Asaoka, None