June 2021
Volume 62, Issue 8
Open Access
ARVO Annual Meeting Abstract  |   June 2021
Uncertainty estimation for the feature agnostic glaucoma detection based on OCT volumes
Author Affiliations & Notes
  • Bahman Tahayori
    IBM Research Australia, Southbank, Victoria, Australia
  • Bhavna Josephine Antony
    IBM Research Australia, Southbank, Victoria, Australia
  • Hiroshi Ishikawa
    Department of Ophthalmology, NYU Langone Health, New York, New York, United States
  • Gadi Wollstein
    Department of Ophthalmology, NYU Langone Health, New York, New York, United States
  • Joel S Schuman
    Department of Ophthalmology, NYU Langone Health, New York, New York, United States
  • Rahil Garnavi
    IBM Research Australia, Southbank, Victoria, Australia
  • Footnotes
    Commercial Relationships   Bahman Tahayori, IBM Research (E); Bhavna Antony, IBM Research (E); Hiroshi Ishikawa, None; Gadi Wollstein, None; Joel Schuman, Zeiss (P); Rahil Garnavi, IBM Research (E)
  • Footnotes
    Support  Supported by EY013178, EY030929, unrestricted grant from Research to Prevent Blindness
Investigative Ophthalmology & Visual Science June 2021, Vol.62, 1782. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Bahman Tahayori, Bhavna Josephine Antony, Hiroshi Ishikawa, Gadi Wollstein, Joel S Schuman, Rahil Garnavi; Uncertainty estimation for the feature agnostic glaucoma detection based on OCT volumes. Invest. Ophthalmol. Vis. Sci. 2021;62(8):1782.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To improve the performance of the feature agnostic AI-based glaucoma detection algorithm by evaluating an uncertainty score for each prediction.

Methods : We previously developed a 5-layer 3D Convolutional Neural Network (CNN) in using the OCT scans from both eyes of 134 healthy, 779 glaucoma patients on a Cirrus HD-OCT scanner (200x200 ONH Cubes; Zeiss, Dublin CA). In our analysis, we excluded scans with signal strength less than 7 and downsampled the volumes to 64x64x128 voxels.

Uncertainty of AI models can be estimated by computing the effect of randomly ignoring a set of parameters within the network. We randomly zeroed 5% of each of the 5 convolutional layers and computed the entropy in the final score over 20 forward passes. The performance of the approach was assessed using a 10-fold cross validation study.

Results : Over the 10-folds, the model showed an AUC of 0.91±0.027. In analysing the uncertainty and the probabilistic scores generated by the model (Softmax function) for one fold (see Fig. 1), we observed that a threshold of 0.8 can be used to flag 75% of the false positives and false negatives for further review. On the other hand, only 25% of the healthy controls and 20% of glaucoma patients showed an uncertainty score above that threshold. Fig. 2 summarises the overall uncertainties scores and indicates that low scores are associated with the correctly identified cases while the errors show higher uncertainty scores.

Conclusions : The quantitative uncertainty measure provides supplementary information to clinicians and can be used to flag difficult cases automatically.
Given that the dataset used in this work is highly imbalanced (more positive cases compared to normal cases) the uncertainty score for true negative cases is significantly higher compared to true positive cases. We expect to achieve lower uncertainty scores for normal cases if more data for normal eyes are available.

The uncertainty analysis presented here may aid clinical interpretations of AI-based glaucoma detection outcomes. A separate study will be run to measure this improvement and compare the result with experts’ level of uncertainty.

This is a 2021 ARVO Annual Meeting abstract.

 

This figure shows the test set uncertainty result versus the softmax output. The dashed line is the selected threshold (0.8) for revising cases with high uncertainty.

This figure shows the test set uncertainty result versus the softmax output. The dashed line is the selected threshold (0.8) for revising cases with high uncertainty.

 

The average entropy based uncertainty for all 10 folds.

The average entropy based uncertainty for all 10 folds.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×