June 2020
Volume 61, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2020
Automated Machine Learning Model for Fundus Photo Gradeability and Laterality: A Public ML Research Toolkit Sans-coding
Author Affiliations & Notes
  • Siegfried Wagner
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Edward Korot
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
    Google Health, United Kingdom
  • Hagar Khalid
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Tjebo Heeren
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Daniel Ferraz
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Zeyu Guan
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Gongyu Zhang
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Josef Christian Huemer
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Konstantinos Balaskas
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Pearse Andrew Keane
    NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of Ophthalmology, London, ENGLAND, United Kingdom
  • Footnotes
    Commercial Relationships   Siegfried Wagner, None; Edward Korot, Google Health (E); Hagar Khalid, None; Tjebo Heeren, None; Daniel Ferraz, None; Zeyu Guan, None; Gongyu Zhang, None; Josef Huemer, None; Konstantinos Balaskas, Alimera (R), Allergan (R), Bayer (R), Bayer (F), Heidelberg engineering (R), Novartis (F), Novartis (R), Topcon (R); Pearse Keane, Allergan (R), Bayer (R), Carl Zeiss Meditec (R), Google Health (C), Haag-streit (R), Heidelberg Engineering (R), Novartis (R), Novartis (S), Optos (C), Topcon (R)
  • Footnotes
    Support  NIHR Academic Clinical Fellowship
Investigative Ophthalmology & Visual Science June 2020, Vol.61, 2029. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Siegfried Wagner, Edward Korot, Hagar Khalid, Tjebo Heeren, Daniel Ferraz, Zeyu Guan, Gongyu Zhang, Josef Christian Huemer, Konstantinos Balaskas, Pearse Andrew Keane; Automated Machine Learning Model for Fundus Photo Gradeability and Laterality: A Public ML Research Toolkit Sans-coding. Invest. Ophthalmol. Vis. Sci. 2020;61(7):2029.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To create a machine learning algorithm for fundus photo organization without coding. This model was shared with researchers as part of a broader research toolkit. It enables batch organization of large fundus photography datasets, and simultaneous exclusion of ungradable images in an automated fashion.

Methods : We utilized publicly available fundus photo datasets, and labeled them according to gradeability and laterality. Two retina fellows independently graded training and validation photos (n=3982), and an adjudication process was then performed with a third fellow deciding disagreements to set accurate ground truth labels. This was repeated with an external test dataset of 400 fundus photos. This test set comprised fundus photos from a different setting, country, and fundus camera. Google Cloud AutoML was then used to train a deep learning classification model through the online interface without the use of coding. An AUPRC was provided by the cloud platform, and confusion matrices were calculated. The resulting algorithm was exported as an edge model, and integrated into a framework for batch prediction. This was uploaded to GitHub for public use.

Results : Overall area under the precision-recall curve (AUPRC) was 0.96 for the internal testing dataset. Overall precision was 91.3%, and recall was 84.5%. When the threshold was adjusted to correspond with the intersect of precision and recall curves, the resulting per-label (precision, recall, [threshold]) were Left eye: (92.2%, 90.1%, [0.43]), Right eye (88.6%, 91.6%, [0.43], Ungradeable (77.4%, 81.3% [0.37]).

Conclusions : We demonstrate the use of automated machine learning sans-coding for the creation of a machine learning model to organize fundus photos. We published this for public use as part of a research toolkit. This may be used by researchers for organizing large datasets by eye laterality, and to exclude ungradable images testing their own deep learning models. Furthermore, we hope this encourages researchers without coding experience to design and publish their own automated deep learning models for the benefit of the research community.

This is a 2020 ARVO Annual Meeting abstract.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×