July 2018
Volume 59, Issue 9
Open Access
ARVO Annual Meeting Abstract  |   July 2018
Artificial intelligence in retinopathy of prematurity (ROP): diagnostic performance of a supervised machine learning system (i-ROP ASSIST)
Author Affiliations & Notes
  • Susan Ostmo
    Casey Eye Institute, Oregon Health & Sciences University, Portland, Oregon, United States
  • Veysi Yildiz
    Northeastern University, Boston, Massachusetts, United States
  • Peng Tian
    Northeastern University, Boston, Massachusetts, United States
  • James M Brown
    Massachusetts General Hospital, Boston, Massachusetts, United States
  • J. Peter Campbell
    Casey Eye Institute, Oregon Health & Sciences University, Portland, Oregon, United States
  • Sang Jin Kim
    Casey Eye Institute, Oregon Health & Sciences University, Portland, Oregon, United States
  • Jennifer Dy
    Northeastern University, Boston, Massachusetts, United States
  • Stratis Ioannidis
    Northeastern University, Boston, Massachusetts, United States
  • Deniz Erdogmus
    Northeastern University, Boston, Massachusetts, United States
  • Robison Vernon Paul Chan
    University of Illinois, Chicago, Chicago, Illinois, United States
  • Jayashree Kalpathy-Cramer
    Massachusetts General Hospital, Boston, Massachusetts, United States
  • Michael F Chiang
    Casey Eye Institute, Oregon Health & Sciences University, Portland, Oregon, United States
  • Footnotes
    Commercial Relationships   Susan Ostmo, None; Veysi Yildiz, None; Peng Tian, None; James Brown, None; J. Peter Campbell, None; Sang Kim, None; Jennifer Dy, None; Stratis Ioannidis, None; Deniz Erdogmus, None; Robison Chan, Alcon (C), Allergan (C), Bausch & Lomb (C), Visunex (C); Jayashree Kalpathy-Cramer, INFOTECH Soft Inc (C); Michael Chiang, Clarity (S), Novartis (C)
  • Footnotes
    Support  R01 EY019474 from the National Institutes of Health, Bethesda, Maryland; 2. P30EY010572 from the National Institutes of Health, Bethesda, Maryland; 3. NSF SCH-1622679; 4. Unrestricted funding from the Research to Prevent Blindness
Investigative Ophthalmology & Visual Science July 2018, Vol.59, 2772. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Susan Ostmo, Veysi Yildiz, Peng Tian, James M Brown, J. Peter Campbell, Sang Jin Kim, Jennifer Dy, Stratis Ioannidis, Deniz Erdogmus, Robison Vernon Paul Chan, Jayashree Kalpathy-Cramer, Michael F Chiang; Artificial intelligence in retinopathy of prematurity (ROP): diagnostic performance of a supervised machine learning system (i-ROP ASSIST). Invest. Ophthalmol. Vis. Sci. 2018;59(9):2772.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : This study explores the use of a supervised machine learning (SML) algorithm (i-ROP ASSIST) to develop a quantitative severity scale in ROP using diagnostic labels (plus disease vs. pre-plus disease vs. normal) from multiple experts, as well as pairwise comparison rankings (i.e. selection of image with more severe retinal vascular findings).

Methods : We developed separate training and testing sets of 100 images with a 3-level (plus, pre-plus, or normal) reference standard diagnosis (RSD). The training set included diagnostic labels from 13 experts, as well as pairwise comparisons (each image was ranked from least severe to most severe) by 5 graders. Using previously published methods, we automatically converted each image into a binary mask (Figure 1), and extracted multiple tortuosity and dilation related features from the masks. Using a SML approach, with 5 fold (80:20) cross validation, we trained a logistic regression model combining both expert diagnostic labels and pairwise comparison rankings from our training set to identify: (1) plus disease based on the RSD, (2) pre-plus or worse disease based on the RSD, (3) a continuous severity score (1-100). We calculated the area under the receiver operating characteristic curve (AUC) to evaluate the performance of our method in the independent test set.

Results : In the test set, the algorithm achieved an AUC of for detection of plus disease, and for detection of pre-plus or worse. Figure 2 displays the severity score (1-100) compared to the expert ordered ranking of disease severity for the training set (the only dataset with pairwise rankings to sort). Using a cut off severity score of 40, the sensitivity for detection of pre-plus or worse disease was 93.9% with a specificity of 98.2%.

Conclusions : The i-ROP ASSIST SML algorithm was able to identify plus disease with high sensitivity and specificity, and compute a quantitative severity score that correlated well with expert ranking of disease severity. Future work will us the output from the SML algorithm to develop a more refined quantitative scale that could be used in the screening, diagnosis, and management of ROP.

This is an abstract that was submitted for the 2018 ARVO Annual Meeting, held in Honolulu, Hawaii, April 29 - May 3, 2018.

 

 

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×