July 2019
Volume 60, Issue 9
Open Access
ARVO Annual Meeting Abstract  |   July 2019
Evaluation of the Pegasus Deep Learning System for identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs
Author Affiliations & Notes
  • Lama Al-Aswad
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Rahul Kapoor
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Chia.kai Chu
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Stephen Walters
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Dan Gong
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • aakriti garg
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Kalashree Gopal
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Vipul Patel
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Sameer Trikha
    Visulytix Ltd, United Kingdom
    King’s College Hospital NHS Foundation Trust , United Kingdom
  • Thomas Rogers
    Visulytix Ltd, United Kingdom
  • Nicolas Jaccard
    Visulytix Ltd, United Kingdom
  • C Gustavo De Moraes
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Golnaz Moazami
    Ophthalmology , Edward S. Harkness Eye Insititue, Columbia University Medical Center , New York, New York, United States
  • Footnotes
    Commercial Relationships   Lama Al-Aswad, None; Rahul Kapoor, None; Chia.kai Chu, None; Stephen Walters, None; Dan Gong, None; aakriti garg, None; Kalashree Gopal, None; Vipul Patel, None; Sameer Trikha, Visulytix (I); Thomas Rogers, Visulytix (I); Nicolas Jaccard, Visulytix (I); C Gustavo De Moraes, None; Golnaz Moazami, None
  • Footnotes
    Support  Save Vision Foundation
Investigative Ophthalmology & Visual Science July 2019, Vol.60, 1463. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Lama Al-Aswad, Rahul Kapoor, Chia.kai Chu, Stephen Walters, Dan Gong, aakriti garg, Kalashree Gopal, Vipul Patel, Sameer Trikha, Thomas Rogers, Nicolas Jaccard, C Gustavo De Moraes, Golnaz Moazami; Evaluation of the Pegasus Deep Learning System for identifying Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. Invest. Ophthalmol. Vis. Sci. 2019;60(9):1463.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To evaluate the performance of a deep learning system for the identification of glaucomatous optic neuropathy.

Methods : A retrospective single center study in which six ophthalmologists and the deep learning system, Pegasus (Visulytix Ltd., LondonUK), graded 110 color fundus photographs. Patient images were randomly sampled from the Singapore Malay Eye Study. Ophthalmologists and Pegasus were compared to each other and to the original clinical diagnosis given by the SiMES, which was defined as the gold standard.
Pegasus’ performance was compared to the “best case” consensus scenario, which was the combination of ophthalmologists whose consensus opinion most closely matched the gold standard. The performance of the ophthalmologists and Pegasus was assessed in terms of
sensitivity, specificity and the Area Under the Receiver Operating Characteristic curve (AUROC), and the intra- and inter-observer agreements were determined.

Results : Pegasus achieved an AUROC of 92.6% compared to ophthalmologist AUROCs that ranged from 69.6% to 84.9% and the “best case” consensus scenario AUROC of 89.1%. Pegasus had a sensitivity of 83.7% and a specificity of 88.2%, whereas the ophthalmologists’
sensitivity ranged from 61.3 to 81.6% and specificity ranged from 80.0% to 94.1%. The agreement between Pegasus and gold standard was 0.715, while the highest ophthalmologist agreement with the gold standard was 0.613. Intra-observer agreement ranged from 0.62 to
0.97 for ophthalmologists and was perfect (1.00) for Pegasus.

Conclusions : Pegasus outperformed the best-case consensus scenario involving six ophthalmologists. Furthermore, the high sensitivity of Pegasus makes it a valuable tool for screening patients with glaucomatous optic neuropathy. Future work will extend this study to a larger sample of patients.

This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.

 

Figure 1. Receiver Operating Characteristic (ROC) curves for the deep learning system (red) and the best case consensus scenario between graders (dashed blue). The sensitivity and specificity for individual graders are plotted as points at a range of confidence thresholds.

Figure 1. Receiver Operating Characteristic (ROC) curves for the deep learning system (red) and the best case consensus scenario between graders (dashed blue). The sensitivity and specificity for individual graders are plotted as points at a range of confidence thresholds.

 

Figure 2: Inter-observer agreement matrix. Agreement measured using Cohen’s kappa coefficients.

Figure 2: Inter-observer agreement matrix. Agreement measured using Cohen’s kappa coefficients.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×