Deep learning for automated diagnosis of plus disease in Indian ROP patients

Abstract

Purpose : India has the highest rates of premature birth globally, with ROP currently in its third epidemic. Diagnosis and management of ROP in India is challenging due to the low number of practicing ROP experts, logistical challenges of rural screening programmes, and low interrater agreement on the diagnosis of plus disease. The purpose of this study was to evaluate the performance of a fully-automated algorithm for plus disease diagnosis from fundus photographs acquired by the Aravind Eye Hospital (AEH) network in India.

Methods : We performed our analysis retrospectively on a cohort of 366 patients from AEH, comprising 8,811 images in total. We applied a “quality control” (QC) algorithm to automatically exclude images that failed to meet the following criteria: (1) does not contain an optic disk in the central square region of the image, and (2) are not acceptable for ROP diagnosis. The previously published Imaging and Informatics in ROP (i-ROP) DL algorithm was used to automatically diagnose plus disease. Sensitivity, specificity and receiver operator characteristic (ROC) curves were calculated with respect to the clinical diagnosis, averaging over all valid images in each exam. The area under the ROC curve (AUC) was calculated for two binary diagnoses; normal vs. pre-plus or worse, and plus vs. normal or pre-plus.

Results : The table shows the demographics of the AEH cohort before and after QC. In the Figure, (A) shows the confusion matrix produced by the i-ROP DL (columns) on each image after QC, with respect to the clinical diagnosis (rows). Overall, the results show very few misclassifications of ‘normal’ images as ‘plus’ images, and vice versa. (B) shows the ROC curves calculated at the exam level, with AUCs of 0.83 and 0.90 for “normal” and “plus”, respectively. The sensitivity and specificity for diagnosis of plus disease were 0.97 and 0.71, respectively.

Conclusions : The i-ROP DL algorithm achieves good diagnostic performance on a independent dataset of Indian ROP patients, despite being trained only on data acquired in the US and Mexico. Errors in the automated diagnosis may be due to several reasons: (1) imperfections in the QC process, (2) subjectivity of the clinical diagnosis, (3) differences in the camera hardware, and (4) phenotypic differences between populations. Additional fine-tuning of the i-ROP DL algorithm on a subset of the AEH data may improve overall performance.

This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.