July 2019
Volume 60, Issue 9
Free
ARVO Annual Meeting Abstract  |   July 2019
Clinical validation of a machine-learned algorithm for detection of diabetic retinopathy (DR) and diabetic macular edema (DME) in fundus images
Author Affiliations & Notes
  • Kira Whitehouse
    Verily Life Sciences LLC, California, United States
  • Sunny Virmani
    Verily Life Sciences LLC, California, United States
  • Sunny Jansen
    Verily Life Sciences LLC, California, United States
  • Peter Wubbels
    Verily Life Sciences LLC, California, United States
  • Florence Thng
    Verily Life Sciences LLC, California, United States
  • Dorothy Kwok
    Verily Life Sciences LLC, California, United States
  • Jennifer Han
    Verily Life Sciences LLC, California, United States
  • Dave Miller
    Verily Life Sciences LLC, California, United States
  • Anil Patwardhan
    Verily Life Sciences LLC, California, United States
  • Ashley Moulton
    Verily Life Sciences LLC, California, United States
  • Alejandra Maciel
    Google, California, United States
  • Anita Misra
    Google, California, United States
  • Shirin Barez
    University of California, Berkeley, California, United States
  • Wing Li
    University of California, Berkeley, California, United States
  • Harry Green
    University of California, Berkeley, California, United States
  • Jorge Cuadros
    EyePACS LLC, California, United States
    University of California, Berkeley, California, United States
  • Footnotes
    Commercial Relationships   Kira Whitehouse, Verily Life Sciences LLC (E); Sunny Virmani, Verily Life Sciences LLC (E); Sunny Jansen, Verily Life Sciences LLC (E); Peter Wubbels, Verily Life Sciences LLC (E); Florence Thng, Verily Life Sciences LLC (E); Dorothy Kwok, Verily Life Sciences LLC (E); Jennifer Han, Verily Life Sciences LLC (E); Dave Miller, Verily Life Sciences LLC (E); Anil Patwardhan, Verily Life Sciences LLC (E); Ashley Moulton, Verily Life Sciences LLC (E); Alejandra Maciel, Google (E); Anita Misra, Google (E); Shirin Barez, EyePACS LLC (F); Wing Li, EyePACS LLC (F), Verily Life Sciences LLC (E); Harry Green, EyePACS LLC (F); Jorge Cuadros, EyePACS LLC (E), Google LLC (C), Verily Life Sciences LLC (C)
  • Footnotes
    Support  None
Investigative Ophthalmology & Visual Science July 2019, Vol.60, 1441. doi:https://doi.org/
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Kira Whitehouse, Sunny Virmani, Sunny Jansen, Peter Wubbels, Florence Thng, Dorothy Kwok, Jennifer Han, Dave Miller, Anil Patwardhan, Ashley Moulton, Alejandra Maciel, Anita Misra, Shirin Barez, Wing Li, Harry Green, Jorge Cuadros; Clinical validation of a machine-learned algorithm for detection of diabetic retinopathy (DR) and diabetic macular edema (DME) in fundus images. Invest. Ophthalmol. Vis. Sci. 2019;60(9):1441. doi: https://doi.org/.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To measure the performance of a machine-learned algorithm (MLA) in classifying DR and detecting DME in 45° fundus images.

Methods : A single 45° fundus image centered between the macula and optic disc was retrospectively collected for 850 diabetic individuals from EyePACS. Images were selected to obtain a prespecified distribution of DR and DME based on grades from EyePACS teleretinal service.

Images were prospectively reviewed by three graders, each with experience grading DR/DME in more than 10,000 images. Graders used the ICDR Disease Severity Scale, assigning a DR grade from 1 of 6 options (No, Mild, Moderate, Severe, Proliferative, Ungradable) and a DME grade from 1 of 3 options (No, Yes, Ungradable). Adjudication was performed by the same graders in case of any disagreement. The distribution of adjudicated grades was [23.5% No, 23.7% Mild, 23.7% Moderate, 13.9% Severe, 15.2% Proliferative, 0% Ungradable] for DR and [69.9% No, 28.3% Yes, 1.8% Ungradable] for DME.

To evaluate the performance of the MLA, its grades for DR and DME were compared to the adjudicated grades.

Results : The MLA classified 13 of 850 (1.5%) images as ungradable for either DME or DR. Of these, 12 were DME ungradable but DR gradable; 1 was ungradable for both. The graders classified 15 images as DME ungradable (6 overlapped with the MLA) and no images DR ungradable. The MLA and the graders were within one level of agreement for DR in 818 of 849 (96.3%) cases gradable for DR.

In each of the following metrics, ungradable classification was considered as referrable. The MLA:
● Detected referrable DR (i.e. Moderate, Severe, or Proliferative DR) with a sensitivity of 90.4% (95% CI: 87.3-93.0%) and a specificity of 97.5% (95% CI: 95.5-98.8%).
● Classified 1 of 130 (0.8%, CI: 0.02-4.21%) cases as less than Moderate DR, which graders classified as Proliferative DR.
● Classified 19 of 325 (5.8%, CI: 3.56-8.98%) cases as No DME and less than Moderate DR, which graders classified as vision-threatening DR.

Conclusions : The MLA performed well in identifying referrable DR cases and had a low ungradability rate. This strongly supports its potential usefulness in a clinical screening setting. The MLA’s ability to identify images with the most severe levels of DR and DME suggest it could be used effectively to screen for cases needing urgent referral.

This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×