June 2022
Volume 63, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2022
Implementation of a Large-Scale Retinal Image Curation Workflow Using Deep Learning Framework
Author Affiliations & Notes
  • Rohit Balaji
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Jen Heathcote
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Robert Slater
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Nancy Barrett
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Rick Voland
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Vesna Tomic
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Jared McDonald
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Barbara A Blodi
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Amitha Domalpally
    Department of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Footnotes
    Commercial Relationships   Rohit Balaji None; Jen Heathcote None; Robert Slater None; Nancy Barrett None; Rick Voland None; Vesna Tomic None; Jared McDonald None; Barbara Blodi None; Amitha Domalpally None
  • Footnotes
    Support  None
Investigative Ophthalmology & Visual Science June 2022, Vol.63, 2990 – F0260. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Rohit Balaji, Jen Heathcote, Robert Slater, Nancy Barrett, Rick Voland, Vesna Tomic, Jared McDonald, Barbara A Blodi, Amitha Domalpally; Implementation of a Large-Scale Retinal Image Curation Workflow Using Deep Learning Framework. Invest. Ophthalmol. Vis. Sci. 2022;63(7):2990 – F0260.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : The development of artificial intelligence (AI) algorithms for analyzing retinal pathologies requires training based on well-organized, labeled images. The goals of this project are to develop an AI model to curate 7-field retinal photographs and to explore ways to implement the AI model at the Wisconsin Reading Center (WRC) with high-volume image submissions.

Methods : Stereoscopic 7 modified field images of the retina are used for the evaluation of diabetic retinopathy via the Early Treatment Diabetic Retinopathy Study (ETDRS) Severity Scale as an outcome in clinical trials. The imaging protocol includes 7 pairs of images of the optic disc, macula, and surrounding retinal quadrants along with an image of the anterior part of the eye (red reflex). Each field of the retina is identified by a field designation number (Figure 1). Clinical images submitted to the WRC are often inefficiently organized for the training of AI algorithms. We trained a neural network to differentiate red reflex images from retinal images and to provide the appropriate retinal field designation. Model outputs included classification of the retinal images into 8 classes (7 fields and red reflex) and a probability score (0 – 1) for each class to predict potential classification errors. The AI model was trained and internally validated on 17,529 images from multiple sites and tested on 3004 independent images. The ground truth was generated by 2 WRC graders.

Results : Exact agreement on field designations between graders and the AI model was found for 2651/3004 images (88%, kappa 0.87) (Figure 2). AI probability scores were 0.95-0.99 for labels that matched the human assessment and 0.39-0.84 for labels that did not match. Fields with non-matched labels included images with poor image quality/focus and improper localization of retinal landmarks.

Conclusions : The deep learning model provides an accurate, automated method for curating retinal images. The probability score provides a tool to flag potential errors in AI labels that can be routed for human oversight. Large-scale AI implementation systems need a tiered approach to augment workflow, increase trust in AI models, and to provide a means for continuous model development.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Figure 1. Typical 7M retinal field designations in a left eye

Figure 1. Typical 7M retinal field designations in a left eye

 

Figure 2. Designation agreement by field

Figure 2. Designation agreement by field

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×