Purchase this article with an account.
Lama Al-Aswad, Rahul Kapoor, Chia.kai Chu, Stephen Walters, Dan Gong, aakriti garg, Kalashree Gopal, Vipul Patel, Sameer Trikha, Thomas Rogers, Nicolas Jaccard, C Gustavo De Moraes, Golnaz Moazami; Evaluation of the Pegasus Deep Learning System for identifying Glaucomatous Optic
Neuropathy Based on Color Fundus Photographs. Invest. Ophthalmol. Vis. Sci. 2019;60(9):1463.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
To evaluate the performance of a deep learning system for the identification of glaucomatous optic neuropathy.
A retrospective single center study in which six ophthalmologists and the deep learning system, Pegasus (Visulytix Ltd., LondonUK), graded 110 color fundus photographs. Patient images were randomly sampled from the Singapore Malay Eye Study. Ophthalmologists and Pegasus were compared to each other and to the original clinical diagnosis given by the SiMES, which was defined as the gold standard.Pegasus’ performance was compared to the “best case” consensus scenario, which was the combination of ophthalmologists whose consensus opinion most closely matched the gold standard. The performance of the ophthalmologists and Pegasus was assessed in terms ofsensitivity, specificity and the Area Under the Receiver Operating Characteristic curve (AUROC), and the intra- and inter-observer agreements were determined.
Pegasus achieved an AUROC of 92.6% compared to ophthalmologist AUROCs that ranged from 69.6% to 84.9% and the “best case” consensus scenario AUROC of 89.1%. Pegasus had a sensitivity of 83.7% and a specificity of 88.2%, whereas the ophthalmologists’sensitivity ranged from 61.3 to 81.6% and specificity ranged from 80.0% to 94.1%. The agreement between Pegasus and gold standard was 0.715, while the highest ophthalmologist agreement with the gold standard was 0.613. Intra-observer agreement ranged from 0.62 to0.97 for ophthalmologists and was perfect (1.00) for Pegasus.
Pegasus outperformed the best-case consensus scenario involving six ophthalmologists. Furthermore, the high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy. Future work will extend this study to a larger sample of patients.
This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.
Figure 1. Receiver Operating Characteristic (ROC) curves for the deep learning system (red) and the best case consensus scenario between graders (dashed blue). The sensitivity and specificity for individual graders are plotted as points at a range of confidence thresholds.
Figure 2: Inter-observer agreement matrix. Agreement measured using Cohen’s kappa coefficients.
This PDF is available to Subscribers Only