Artificial Neural Network Approach for Differentiating Open-Angle Glaucoma From Glaucoma Suspect Without a Visual Field Test

Abstract

Purpose.: To increase the effectiveness of treating open-angle glaucoma (OAG), we tried to find a screening method of differentiating OAG from glaucoma suspect (GS) without a visual field (VF) test.

Methods.: Data were collected from the Korean National Health and Nutrition Examination Survey (KNHANES) conducted in 2010. Of 8958 participants, 386 suspected OAG subjects underwent a VF test. For the training dataset, five OAG risk prediction models were created using multivariate logistic regression and an artificial neural network (ANN) with various clinical variables. Informative variables were selected by an algorithm of consistency subset evaluation, and cross validation was used to optimize performance. The test dataset was used subsequently to assess OAG-prediction performance using the area under the curve (AUC) of the receiver-operating characteristic.

Results.: Among five OAG risk prediction models, an ANN model with nine noncategorized factors had the greatest AUC (0.890). It predicted OAG with an accuracy of 84.0%, sensitivity of 78.3%, and specificity of 85.9%. It included four nonophthalmologic factors (sex, age, menopause, and duration of hypertension) and five ophthalmologic factors (IOP, spherical equivalent refractive errors, vertical cup-to-disc ratio, presence of superotemporal retinal nerve fiber layer [RNFL] defect, and presence of inferotemporal RNFL defect).

Conclusions.: Though VF tests are considered the most important examination to distinguish OAG from GS, they sometimes are impractical to conduct for small private eye clinics and during large scale medical check-ups. The ANN approach may be a cost-effective screening tool for differentiating OAG patients from GS subjects.

Open-angle glaucoma (OAG) is a neurodegenerative ocular disease that is predicted to affect approximately 59 million people worldwide by 2020.¹ Generally, OAG is characterized by progressive optic nerve damage, which leads to visual field (VF) defects and glaucomatous optic disc changes with loss of the retinal nerve fiber layer (RNFL).² 

Due to an ever increasing prevalence of glaucoma, hospital eye services are challenged by the large amount of work being placed on glaucoma specialists.¹ Patients with suspected glaucoma are a major cause of referrals to hospital eye services; however, 36% to 60% of these referred patients are confirmed as not having glaucoma after further detailed examinations including a VF test.³ Considering the costs of treatment and monitoring, it is imperative to distinguish OAG from glaucoma suspect (GS), which may not have clinically significant optic neuropathy. 

A standard automated VF test has a key role in diagnosing OAG from GS. According to the subjects, however, it sometimes is difficult to perform, and it consumes a significant amount of time and resources. Such a manual process performed by patients is subjective and has shown intrasubject variability as well as intersubject variability in several epidemiologic studies.⁴ Due to these problems, repeated VF tests usually are required to diagnose OAG. Furthermore, recent studies have reported that VF tests are a major part of the workload being placed on hospital eye services due to the increasing incidence of OAG and GS.⁵ This workload results in an extended waiting time for patients in routine clinical settings.⁶ Therefore, it is necessary to develop a clinically efficient and cost-effective tool for differentiating OAG from GS. If such a test was able to provide an objective assessment of OAG, it would improve clinicians' decision-making in personalizing individual management. 

Understanding biomarker patterns of patients also would facilitate in assessing and distinguishing between the risks of OAG from GS. Our study used multivariate logistic regression (MLR) and an artificial neural network (ANN) to analyze high-dimensional biomarker patterns to prompt ophthalmologists to identify OAG among patients with suspected glaucoma. A simple scoring system is an easy and convenient model to assess the risk of diseases, and it has been used in the prediction of various diseases by a number of clinicians.⁷ However, due to low performance of such scoring systems, a more accurate prediction tool is needed. The ANN, which falls within the area of artificial intelligence technology and uses mathematical systems that mimic biological neural networks, may be a solution to this problem.⁸ These networks can be trained to recognize underlying patterns of diseases. Once appropriate training is completed, the neural networks attempt to predict disease risk with greater accuracy than conventional classification analysis. Recently, there have been many advances in the methodology of ANN with the hope of automatically finding an optimal predictive model. Owing to its ability to detect complex nonlinear relationships between predictors and diseases, ANN has been used successfully in medical decision support systems.⁹ 

In this study, we developed and validated an ANN model with the aim of differentiating OAG from GS without a VF test. The objective of this study was to select patients with clinically significant optic nerve damage and recommend them for VF tests to increase the effectiveness of treating OAG. To achieve the best performance, our prediction models incorporated a wide range of information from ophthalmologic measurements and systemic factors. 

Subjects and Methods

Data Source and Subjects

This cross-sectional study investigated and analyzed various prediction models for OAG in South Korea. All analyses were conducted on data collected from the Fifth Korean National Health and Nutrition Examination Survey (KNHANES V-1, available in the public domain at http://knhanes.cdc.go.kr/knhanes) conducted in 2010. The KNHANES is an ongoing, population-based, nationwide epidemiological survey performed by the Korea Center for Disease Control and Prevention, Ministry of Health and Welfare.¹⁰ The KNHANES V-1 consists of a health interview survey, health examination survey, and nutrition survey. All 8958 participants were from 3840 households and were selected randomly from 192 survey locations using stratified sampling that considered population sex, age, regional area, and type of residential area. Given that all data were available on the web and data analysis was secondary, an additional ethical review of the study protocol was not needed for this study. The study protocol of the KNHANES V-1 adhered to the tenets of the Declaration of Helsinki, and all participants were treated accordingly. 

The health interview survey was a face-to-face interview conducted by trained interviewers. Each participant was interviewed and completed a questionnaire assessing alcohol consumption, smoking status, diabetes mellitus, hypertension, and physical activity level. All the subjects' levels of physical activity (walking, moderate, and vigorous physical activities) were calculated using the metabolic equivalents of task values based on self-reported frequency and duration of activities during the week. Height, weight, and waist circumference were measured and body mass index was calculated. Measurements of serum biomarkers, including hemoglobin, fasting glucose, total cholesterol, triglycerides, and creatinine, were collected. Estimated glomerular filtration rates were calculated from serum creatinine levels.¹¹ 

A flow diagram of inclusion and exclusion procedures in the KNHANES V-1 is shown in Figure 1. Initial candidates for this study included the 6283 participants of the ophthalmologic interview group. To reduce the number of confounding variables that might have an effect on OAG diagnosis, we excluded participants who had (1) any systemic disease that could influence VF, including mental retardation, stroke, or thyroid diseases (n = 259); (2) any ophthalmic disease that could influence VF, including age-related macular degeneration or diabetic retinopathy (n = 390); (3) a history of any ophthalmic disease that could cause amblyopia, including strabismus, ptosis, or retinal disorders (n = 595); (4) a history of surgical or medical glaucoma treatment (n = 22); (5) any evidence of intraocular inflammation of keratic precipitates or iris atrophy (n = 6); or (6) any evidence of a closed anterior chamber angle (n = 125). The remaining 4886 participants were evaluated to determine whether they had OAG or not, and a total of 3727 subjects were categorized as a “healthy” group. The other 1159 subjects were considered to have suspected OAG and were advised to perform a VF test. Among these VF-test-requiring subjects, 386 subjects who underwent a reliable VF test ultimately were included in our study population. The dataset of these 386 suspected OAG participants was separated randomly into two independent groups: a training group and a test group. The training group comprised two-thirds of the entire dataset (257 participants) and was used to construct the ANN model. The test group consisted of one-third of the entire dataset (129 participants) and was used to assess the ability to predict OAG. 

Figure 1

 

Flow diagram of the inclusion and exclusion procedure used with data from the KNHANES V-1.

Ophthalmologic Examination

Visual acuity based on the logarithm of the minimum angle of resolution (logMAR) scale was measured at a distance of 4.0 m using an international standard vision chart. An autorefractor-keratometer was used for all refraction measurements, which were converted to spherical equivalent refractive errors (SERE). A structured slit-lamp examination was performed to find any diseases in the anterior segment of the eye and to measure the IOP using Goldmann applanation tonometry. It also was used to evaluate the anterior chamber angle status; the anterior chamber depth was measured using the Van Herick method. A digital fundus camera was used to obtain the digital fundus images, which were captured under physiological mydriasis from all participants who were 19 years of age or older. A VF test using frequency-doubling technology (FDT; Humphrey Matrix; Carl Zeiss Meditec, Inc., Dublin, CA, USA) with the screening program N-30-1 was performed if the participants had suspected glaucoma. The suspected glaucoma included any of five conditions: (1) elevated IOP (>21 mm Hg), (2) vertical or horizontal cup-to-disc ratio ≥0.5, (3) presence of optic disc hemorrhage, (4) presence of RNFL defect, or (5) violation of the inferior-superior-nasal-temporal (ISNT) rule. All ophthalmologic examinations were completed by two trained ophthalmologists.¹² 

Participants were classified finally into three groups: healthy, GS (suspected OAG, yet diagnosed as not having OAG after the VF test), and OAG. The OAG patients were defined as meeting the diagnostic criteria of the International Society for Geographical and Epidemiological Ophthalmology.¹³ The specific diagnostic criteria adopted for the Korean population were: (1) presence of a reliable FDT testing result (fixation and false positive error ≤1), (2) glaucomatous optic disc (loss of neuroretinal rim with a vertical or horizontal cup-to-disc ratio ≥0.6, presence of optic disc hemorrhage or RNFL defect, or violation of the ISNT rule), (3) presence of an abnormal FDT testing result (at least one location of reduced sensitivity), and (4) presence of an open anterior chamber angle (peripheral anterior chamber depth >1/4 corneal thickness).¹² When both eyes were eligible, one eye was selected randomly and used for the analyses. 

Multivariate Logistic Regression

For OAG risk prediction model development, the association between clinical features and OAG was examined using MLR based on the data from the 386 suspected OAG participants who had a reliable VF testing result. We initially screened 66 comprehensive variables (Table 1) considered to be potentially associated with OAG. Backward elimination was performed until we reached a final model with significant covariates. The P value thresholds at which variables were entered into and removed from the model were 0.05 and 0.10, respectively. After selecting the OAG-associated factors by MLR (Table 2), these were used to develop several OAG risk prediction models (see Model Selection and Validation). When a model treated the factors as having a categorized form, the cutoff point of each variable was decided with considerable computation. 

Table 1

 

Clinical Characteristics of Participants in the KNHANES V-1 by Glaucoma Status

 

Table 2

 

Multivariate Logistic Regression Analysis to Find the Clinical Features Related to OAG

 

Artificial Neural Network

An ANN is the best known machine learning technique, and one was constructed for our study using NeuroSolutions, version 6.0 (NeuroDimension, Inc., Gainesville, FL, USA). NeuroSolutions is a professional software solution that simplifies the construction of ANNs.¹⁴ This software allows for simultaneous testing of different types of neural networks. To avoid overfitting, the prediction models were internally validated using cross-validation. Performances of the prediction models were monitored during training and cross-validation to obtain optimal algorithm parameters, such as the momentum, learning rate, and number of hidden nodes. The ANN construction was accomplished using the training group. 

Model Selection and Validation

We developed five different OAG risk prediction models (Table 3). Model 1 was based on MLR using nine categorized clinical features. Model 2 was an MLR model similar to Model 1; however, it used the same features in noncategorized form. Model 3 was an ANN model using the same nine noncategorized features (we named this model Yonsei ANN for OAG Risk Prediction). Model 4 was an ANN model using all 66 clinical features. Model 5 was another MLR model using the aforementioned five conditions of suspected glaucoma. 

Table 3

 

Risk Prediction Model Development for OAG

 

The prediction models were validated in two schemes: a 10-fold cross-validation in the training dataset and then a validation in the test dataset. The 10-fold cross-validation, which randomly divided the set of samples into 10 parts for training and validating, provided an estimation of generalized prediction performance. Validation in the test set showed explicit results when the prediction models were established based on the entire training set. The area under the curve (AUC) of the receiver operating characteristic (ROC), accuracy, sensitivity, and specificity of the prediction models were calculated. We generated the ROC curves and selected cutoff points that maximized Youden's index.¹⁵ Participants above the cutoff points were classified as being at high-risk in each prediction model. 

Statistical Analysis

We used SPSS 18.0 (SPSS, Inc., Chicago, IL, USA) for statistical analysis and MedCalc 12.3 (MedCalc, MariaKerke, Belgium) for ROC analysis. 

Results

The background characteristics of the participants in the KNHANES V-1 by glaucoma status are presented in Table 1. Among 386 suspected OAG participants of our study population, 94 subjects were confirmed to have OAG after a VF test. The other 292 subjects were confirmed to have GS rather than OAG. Several features were significantly different between the GS and OAG groups. The OAG patients seemed to be older, and more likely to have hypertension, diabetes, dyslipidemia, and ischemic heart disease than GS subjects. Data for the 3727 participants classified as healthy also are shown in Table 1. 

Among 66 putative clinical features, MLR selected nine factors that had a significant association with OAG (Table 2): sex, age, menopause, duration of hypertension, SERE, IOP, vertical cup-to-disc ratio, superotemporal RNFL defect, and inferotemporal RNFL defect. Among these nine factors, the presence of superotemporal and inferotemporal RNFL defects had the highest odds ratios (OR): 7.76 (95% confidence interval [CI], 2.30–26.19) and 7.31 (95% CI, 2.14–25.02), respectively. The ORs were 4.90 (95% CI, 1.37–17.49) for IOP greater than 21 mm Hg and 3.35 (95% CI, 1.58–7.14) for SERE less than −3.0 diopters. Conversely, a vertical cup-to-disc ratio greater than 0.7 had the lowest OR 2.27 (95% CI, 1.21–4.23). 

Table 3 presents the five risk prediction models that were able to differentiate OAG from GS without a VF test. As described above, Model 3, Yonsei ANN for OAG Risk Prediction, was an ANN model that used the same nine noncategorized factors (Fig. 2). This model was a multilayer perception neural network with a back-propagation algorithm. We found three neurons in the hidden layer. 

Figure 2

 

Schematic representation of our ANN model for differentiating OAG from GS without a VF test. C/D, cup-to-disc ratio; HTN, hypertension; RNFLD_IT, inferotemporal retinal nerve fiber layer defect; RNFLD_ST, superotemporal retinal nerve fiber layer defect; SERE, spherical equivalent refractive errors.

To assess the OAG predicting ability of our models, a 10-fold cross validation was performed in the training dataset (Table 4). As for the results, the ANN of Model 3 showed the best discriminative ability among the five OAG risk prediction models. In other words, this model differentiated OAG from GS most effectively, with an AUC of 0.870 (95% CI, 0.826–0.906). Its negative predictive value reached 97.6% (95% CI, 95.0%–99.1%), and its accuracy was 79.5% (95% CI, 74.3%–83.9%). We then applied these OAG risk prediction models to a test dataset composed of independent data from the training set (Table 4). Even in the test set, consistent validation results were observed. The ANN of Model 3 had the greatest AUC (0.890; 95% CI, 0.808–0.945) and was found to have an accuracy of 84.0% (95% CI, 74.7%–90.7%). For both datasets, Model 5 of MLR using the traditional five ophthalmic factors showed the poorest performance. The AUCs for training and test datasets were 0.620 (95% CI, 0.562–0.676) and 0.635 (95% CI, 0.529–0.732), respectively. 

Table 4

 

Performance of Risk Prediction Models for OAG

 

Figure 3 shows ROC curves of the five OAG risk prediction models used to differentiate OAG patients among subjects with suspected glaucoma in the test dataset. The ANN of Model 3 was the best discriminator between OAG and GS. Its AUC of 0.890 was significantly greater than the AUCs of Model 1 (0.756) and Model 5 (0.635, P < 0.05, Table 4). 

Figure 3

 

The ROC curves of the risk prediction models for OAG among subjects with suspected glaucoma in the test group. Model 1 was the MLR model using the nine categorized predictors. Model 2 was the MLR model established by same nine factors, but using these factors in a noncategorized form. Model 3 was the ANN model trained with the same nine noncategorized factors. Model 4 was the ANN model trained with all 66 relevant clinical features. Model 5 was another MLR model based on the five conditions of suspected glaucoma. *For Model 3, which has the greatest area under the ROC curve (0.890).

Discussion

Using population-based health records, we introduced a novel artificial intelligence approach that used an ANN to differentiate OAG from GS without a VF test. This technique significantly improved the performance of OAG risk prediction models, even when the same clinical factors were used for MLR-based models. This ANN approach may be a useful cost-effective screening tool for distinguishing real OAG patients from those with suspected glaucoma. 

Recent studies have described the importance objective OAG risk prediction and have developed several risk calculators.^16,17 A calculator developed by the Ocular Hypertension Treatment Study (OHTS) estimates the 5-year risk of an untreated individual with ocular hypertension developing OAG.¹⁶ The study identified five risk predictors, including age, IOP, central corneal thickness, vertical cup-to-disc ratio, and Humphrey VF pattern standard deviation. Another calculator developed by the New York Eye and Ear Infirmary and the Bascom Palmer Eye Institute forecasts VF outcomes in patients with treated glaucoma.¹⁷ Our study is essentially in the same vein as these researchers, who emphasize the necessity of a glaucoma risk calculator. Considering that each calculator may have its own purpose, in the present study, we focused on the initial screening of OAG rather than confirmation of diagnosis or prediction of disease progression. As conducting a VF test is the rate-limiting step for differentiating OAG from GS, we attempted to develop an OAG risk calculator without a VF test. 

Though most of the previous glaucoma risk calculators were based on the assumption that clinical outcomes are linearly associated with selected risk factors, such linear modeling is not the most suitable in achieving the best discriminative ability to predict disease outcomes.¹⁴ We believe that our proposed ANN, which is based on large amounts health data, addresses these concerns. Our results also indicated that it is possible to develop a predictive instrument using machine-learning techniques, such as ANN. Validation using ROC analysis suggests that the ANN approach offers a statistically significant improvement in differentiating OAG from GS even without a VF test. The ANN also was shown to be more effective in analyzing the epidemiological underlying patterns of OAG than MLR. These findings are consistent with previous studies comparing ANN and conventional methods in various complicated problems involving disease prediction.¹⁴ Given that an ANN has the ability to incorporate nonlinearity in high-dimensional space, it is possible to consider all factors to improve the sensitivity and specificity of prediction.¹⁸ If our prediction models sustain their accuracy after providing validation for patients in the outpatient clinic, it may be possible to use them as cost-effective screening tools to identify candidates for OAG. 

In the present study, we evaluated nonophthalmologic factors as well as ophthalmologic factors. Emerging evidence indicates that OAG is not localized but influenced by systemic metabolic status.¹⁹ Incorporating information from both ophthalmologic examinations and systemic evaluations would increase the chances of detecting OAG. When all clinical data were available, the presence of OAG was associated with nine risk factors. These factors that were useful in differentiating OAG from GS were sex, age, menopause, duration of hypertension, SERE, IOP, vertical cup-to-disc ratio, and superotemporal and inferotemporal RNFL defects. Thus, we reconfirmed the well-known glaucoma risk factors of age, IOP, vertical cup-to-disc ratio, and superotemporal and inferotemporal RNFL defects.¹⁹ In addition, we discovered the importance of sex, menopause, duration of hypertension, and SERE as glaucoma risk factors. Our data implied that men and postmenopausal women are at greater risk of contracting OAG. Though the relationship between sex and glaucoma still is a controversial issue, current evidence suggests that older women are more vulnerable to glaucoma than men.²⁰ One possible theory is that estrogen deficiency affects the optic nerve during the aging process. Thasarat et al.²¹ hypothesized that the early loss of estrogen leads to glaucomatous damage and degenerative changes in the optic nerve. Hypertension also is a known risk factor of OAG.²² In our multivariate analysis, the duration of hypertension was a predictive factor of OAG separate from GS. Our data indicated that 5 years or more of hypertension is a risk factor for OAG. Recently, the World Glaucoma Society recognized that lower perfusion pressure is a risk factor for glaucoma; however, there is no consensus about the exact systemic blood pressure characteristics or vascular changes in patients with glaucoma. It is thought that chronic systemic hypertension may have negative effects on vascular changes. Furthermore, we also found that ophthalmic analysis results may be a predictive factor for OAG. Myopia was selected by our prediction model, which has long been suggested as a risk factor for OAG.²³ There is conflicting evidence of the importance of the refractive error range for OAG. In this study, individuals with moderate to high myopia (≤−3.0 diopters) were at risk of OAG from GS. Regarding IOP, we found that 21 mm Hg was an acceptable cutoff value for predicting the risk of OAG. Suh et al.²⁴ reported that the mean IOP for the South Korean population is 14.10 ± 2.74 mm Hg. This mean value is low in comparison with IOP values reported from European or American populations, yet similar to that of the Japanese population. Generally, ocular hypertension is defined as an IOP greater than 21 mm Hg, but our calculator suggested that an IOP greater than 20 mm Hg is suitable for predicting OAG in the South Korean population. Additionally, a vertical cup-to-disc ratio of 0.7 or greater was found to be a risk factor used in identifying patients with OAG in most studies, and our study showed similar results.² 

Regarding ANN, there have been several attempts to apply it on the VF and RNFL assessments for automated glaucoma diagnosis and/or progression.^25–29 Due to its performance, it has promise of future greatness for humans. In this investigation, we applied the ANN to develop an OAG risk calculator. Although our ANN model showed good prediction performance for OAG, this study has several limitations. First, the findings reported in our study were cross-sectional and observational. Although OAG usually is considered irreversible, and the study population was nationwide and randomly sampled, the cross-sectional nature of this study precludes determination of the best prediction model. We expect that a longitudinal study design will be a more powerful prediction tool for differentiating the rate of optic nerve damage from baseline data. Second, this is an Asian-specific study performed at the level of a single country. Generally, the incidence and progression of glaucoma are influenced by ethnic differences and genetic backgrounds. The Collaborative Normal-Tension Glaucoma Study (CNTGS) and the Collaborative Initial Glaucoma Treatment Study (CIGTS) have shown that race is one of the significant risk factors for normal-tension and newly-diagnosed glaucoma.^30,31 Thus, it is uncertain whether our results will be equally applicable to other demographic populations. Third, we had no central corneal thickness (CCT) data. The CCT was known to be a strong factor in predicting the development of OAG in the OHTS study. However, a recent study reported that IOP and CCT are each equal and separate predictive factors of OAG. Moreover, Brandt et al.³² suggested that the CCT correction formula for Goldmann applanation tonometry measurements may be of little value for clinicians. Based on such reports, we expect that the IOP value will compensate for the lack of CCT data in prediction models of OAG. In a future study, we hope to include CCT data to compare its predictive accuracy against IOP. 

Our free OAG risk calculator, Yonsei ANN for OAG Risk Prediction (Fig. 4), is simple and easy to use. To fill out the nine entry fields of our calculator, a user does not need any expensive specialized instruments, such as those used for standard automated perimetry or optical coherence tomography. Thus, it may be useful for general physicians and comprehensive ophthalmologists rather than glaucoma specialists. We believe that our calculator significantly reduces the number of subjects who require a VF test to confirm the diagnosis of OAG. It also may be very helpful for people who live in underpopulated or underdeveloped areas, who may have difficulties in visiting glaucoma specialists. Another advantage of our calculator is that it becomes more accurate whenever we add new informative data, due to the intrinsic characteristics of artificial intelligence behind our ANN. 

Figure 4

 

Yonsei ANN for OAG Risk Prediction. The ANN approach for differentiating OAG from GS without a VF test.

In summary, this study supports the conclusion that machine-learning techniques using ANNs can contribute to the advancement of clinical decision-making tools that can accurately discriminate OAG even without a VF test. We hope that this study is helpful in effectively identifying patients with elevated IOP or glaucomatous optic discs who are in need of a VF test and ultimately providing earlier treatment for OAG, which is a major cause of blindness in such patients. 

Acknowledgments

Supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (No. 2014R1A1A2057875). 

Disclosure: E. Oh, None; T.K. Yoo, None; S. Hong, None