Artificial intelligence in retinopathy of prematurity: development of a fully automated deep convolutional neural network (DeepROP) for plus disease diagnosis

Abstract

Purpose : We propose that an image analysis system based on “deep learning”can achieve expert-level performance in an entirely automated fashion.

Methods : A multi-institutional dataset of approximately 6,000 posterior retinal photographs was collected as part of the ongoing “Imaging and Informatics in ROP” (i-ROP) study. Each image was assigned a reference standard diagnosis (RSD) using previously published methods. Images were excluded if two or more image graders labeled them as being “unacceptable for diagnosis”, or if there was evidence of retinal detachment. Retinal photographs were pre-processed using a vessel segmentation algorithm that eliminates variations in image contrast and coloration. For automated diagnosis, the dataset was then split in an 80:20 ratio into five training (n = 4299 ± 70, mean ± standard deviation) and test (n = 1113±70) sets. A deep convolutional neural network (CNN) was trained on each of the training datasets to predict whether an image is ‘normal’, ‘pre-plus’ or ‘plus’. Each of the trained CNNs was evaluated on its corresponding test dataset, with performance measured using receiver operating characteristic (ROC) analysis and areas under the ROC curve (AUC). Image features learned automatically during training were visualized using t-distributed stochastic neighbor embedding (t-SNE) as a two-dimensional scatter plot. The distance between pairs of points on the scatter plot is proportional to their “feature similarity”.

Results : The mean ± standard deviation AUCs for the five models were 0.94±0.01 for diagnosis of ‘pre-plus or worse, and 0.98±0.01 for diagnosis of “plus.” Analysis of image features learned by the CNN using t-SNE shows separation of ‘normal’, ‘pre-plus’ and ‘plus’ images.

Conclusions : The DeepROP CNN can diagnose plus disease from retinal photographs with almost perfect sensitivity and specificity compared to the RSD. t-SNE analysis of image features learned automatically by the CNN suggest a phenotypic continuum of disease severity that could explain inter-observer discrepancies and provide a more granular score for monitoring disease progression.

This is an abstract that was submitted for the 2018 ARVO Annual Meeting, held in Honolulu, Hawaii, April 29 - May 3, 2018.

 

Areas under the ROC curve for classification of normal (vs. pre-plus and plus) and plus (vs. normal and pre-plus).

 

t-SNE plot of retinal images along the disease spectrum, using features extracted from the convolutional neural network.