June 2021
Volume 62, Issue 8
Open Access
ARVO Annual Meeting Abstract  |   June 2021
Democratizing AI for DR: Automated Self-Training to Address Label Scarcity for Deep Learning in Diabetic Retinopathy Classification
Author Affiliations & Notes
  • Edward Korot
    Ophthalmology, Stanford University School of Medicine, Stanford, California, United States
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Mariana Batista Goncalves
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Josef Christian Huemer
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Hagar Khalid
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Siegfried Wagner
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Xiaoxuan Liu
    University of Birmingham, Birmingham, Birmingham, United Kingdom
  • Livia Faes
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Alastair K Denniston
    University of Birmingham, Birmingham, Birmingham, United Kingdom
  • Pearse Keane
    Ophthalmology, Moorfields Eye Hospital NHS Foundation Trust, London, London, United Kingdom
  • Footnotes
    Commercial Relationships   Edward Korot, None; Mariana Goncalves, None; Josef Huemer, None; Hagar Khalid, None; Siegfried Wagner, None; Xiaoxuan Liu, None; Livia Faes, None; Alastair Denniston, None; Pearse Keane, None
  • Footnotes
    Support  Moorfields Eye Charity 190016A (EK), Career Development 190028A (PAK), UKRI Future Leaders Fellowship MR/T019050/1 (PAK), (NIHR) Clinician Scientist Award NIHR-CS–2014-12-023 (PAK)
Investigative Ophthalmology & Visual Science June 2021, Vol.62, 2132. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Edward Korot, Mariana Batista Goncalves, Josef Christian Huemer, Hagar Khalid, Siegfried Wagner, Xiaoxuan Liu, Livia Faes, Alastair K Denniston, Pearse Keane; Democratizing AI for DR: Automated Self-Training to Address Label Scarcity for Deep Learning in Diabetic Retinopathy Classification. Invest. Ophthalmol. Vis. Sci. 2021;62(8):2132.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To address the scarcity of high quality adjudicated labels for training supervised deep learning models. We used an automated sans-coding approach, developing a teacher model from a small high quality labeled dataset to subsequently assign diabetic retinopathy (DR) referral pseudolabels to a large unlabeled dataset. A student model was then trained from the combined dataset.

Methods : We utilized two publicly available fundus photo DR datasets. High quality adjudicated DR severity labels were applied to the first dataset of fundus photos (n=1744). Google Cloud Automated Machine Learning (AutoML) was next used to train a deep learning image classification model sans-coding, termed the teacher model for referable and nonreferable DR. The resulting algorithm was deployed in Google Cloud platform and made available for inference. The teacher model was used to generate DR referral predictions for an unlabeled public fundus photo dataset (n=58,689). The resulting image-label pairs were combined with the high quality teacher model’s training dataset, and a student deep learning model was trained.

Results : The teacher model area under the precision-recall curve (AUPRC), accuracy, sensitivity, specificity was 0.964, 92.0%, 73.9, 98.4 respectively, while the student model metrics improved to 0.976, 92.5%, 87.0%, 94.5% on the internal validation dataset. Teacher model external validation accuracy, sensitivity and specificity was 93.3%, 94.4%, 75.0%, while student model external validation metrics improved to 97.6%, 99.5%, 66.7% respectively.

Conclusions :
We demonstrate a sans-coding framework, which utilizes AutoML to address label scarcity for deep learning in DR. We show that self-training is an effective method to increase performance and decrease overfitting, which may simultaneously save time on expensive yet high-quality expert labeling. While the student model external validation dataset had 1 more false positive than the teacher model, clinically costly false negatives decreased by 10 as compared to the teacher model. As the tools for machine learning continue to be democratized, our methodology has potential to address the remaining disparity of expensive clinical labeling which bottlenecks small scale clinicians and researchers.

This is a 2021 ARVO Annual Meeting abstract.

 

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×