June 2022
Volume 63, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2022
Explaining Deep Learning Models for Low Vision Prognosis
Author Affiliations & Notes
  • Haiwen Gui
    Stanford University School of Medicine, Stanford, California, United States
  • Benjamin Tseng
    Byers Eye Institute, Department of Ophthalmology, Stanford University, Stanford, California, United States
  • Wendeng Hu
    Byers Eye Institute, Department of Ophthalmology, Stanford University, Stanford, California, United States
  • Sophia Y Wang
    Byers Eye Institute, Department of Ophthalmology, Stanford University, Stanford, California, United States
  • Footnotes
    Commercial Relationships   Haiwen Gui None; Benjamin Tseng None; Wendeng Hu None; Sophia Wang None
  • Footnotes
    Support  Stanford MedScholars program; National Eye Institute 1K23EY03263501; Career Development Award from Research to Prevent Blindness; unrestricted departmental grant from Research to Prevent Blindness; departmental grant National Eye Institute P30-EY026877
Investigative Ophthalmology & Visual Science June 2022, Vol.63, 4058 – F0022. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Haiwen Gui, Benjamin Tseng, Wendeng Hu, Sophia Y Wang; Explaining Deep Learning Models for Low Vision Prognosis. Invest. Ophthalmol. Vis. Sci. 2022;63(7):4058 – F0022.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : Low vision rehabilitation improves quality-of-life for visually impaired patients, but referral rates fall short of national guidelines. Automatically identifying patients with poor visual prognosis from electronic health records (EHR) could allow targeted referrals to low vision services. The purpose of this study was to build, understand, and explain deep learning artificial intelligence models that predict visual prognosis using EHR data.

Methods : We identified 5547 patients with low vision (defined as best documented visual acuity (VA) <20/40) from EHR from 2009-2018, with ≥ 1 year of follow-up. From the EHR, we extracted ophthalmology free-text notes and structured data, such as demographics, billing/procedure codes, medications, and eye exam findings. To predict whether low vision patients would still have low vision a year later, we developed deep learning models that used 1) structured inputs, 2) free-text progress notes represented by standardized clinical concepts extracted using named entity recognition (NER), and 3) a combination of the two. Performance metrics including area under the receiver operating curve (AUROC) and F1 score were evaluated on a held-out test set. We then evaluated models using Local Interpretable Model-Agnostic Explanations to determine which input features were most important to the predictions.

Results : Among the 5547 patients, 40.7% (N=2258) never improved to better than 20/40 over one year of follow-up. Our single-modality model based on structured inputs predicted low vision prognosis with AUROC of 80% and F1 score of 70%. Our model combining NER text and structured inputs achieved an AUROC of 79% and F1 score of 63%. Explainability studies revealed that important features for predicting low vision prognosis included features clinicians also rely upon, such as best visual acuity and the presence or absence of irreversible ophthalmic findings, as shown in Fig1.

Conclusions : Deep learning models often suffer from a lack of transparency. Our explainability analyses provide insight into the medical context surrounding low vision prognosis and are a vital step towards increasing clinicians’ trust in these models.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Fig1. Most important model features for predicting low vision prognosis. Global explanations show the top 15 features that contribute most to the prediction for the 3 models. We also show which factors were most important for the prediction of a single patient whose vision did not improve.

Fig1. Most important model features for predicting low vision prognosis. Global explanations show the top 15 features that contribute most to the prediction for the 3 models. We also show which factors were most important for the prediction of a single patient whose vision did not improve.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×