Comparison of the performance of a Machine-to-Machine (M2M) deep learning algorithm versus human graders for glaucoma screening in teleretinal fundus images

Abstract

Purpose : Application of deep learning (DL) algorithms may improve the efficacy of screening for glaucoma, especially if applied to fundus photos acquired by existing teleretinal screening programs. The purpose of this study was to apply and evaluate how a DL algorithm previously trained with optical coherence tomography (OCT) data to predict retinal nerve fiber layer (RNFL) thickness from photographs would perform compared to human graders in a real-world teleretinal screening program.

Methods : Cross-sectional study of 848 non-mydriatic fundus photos acquired in adult diabetics who underwent teleretinal screening at a community health center. Photographs were graded for glaucoma, no glaucoma, or deemed not gradable by two masked glaucoma specialists and disagreements were resolved by a third masked grader. A M2M DL algorithm that had been previously trained to predict OCT average retinal nerve fiber layer (RNFL) thickness from fundus photography was applied to the new set of photographs from teleretinal screening. The performance of the algorithm was compared to that of human graders. Generalized estimating equations were used to compare the predicted RNFL thickness between eyes that were classified with glaucoma or no glaucoma. The receiver operating characteristic (ROC) curve was plotted to illustrate the trade-off between sensitivity and specificity and the area under the ROC curve was calculated to summarize diagnostic accuracy.

Results : 4.1% of fundus photographs had glaucoma according to human graders, and 3.6% could not be graded. The mean predicted average RNFL thickness was 95.6 ± 10 mm for no glaucoma and 75.1 ± 8.9 mm for glaucoma (p < 0.001, GEE). The AUC for the DL algorithm’s ability to distinguish glaucoma from no glaucoma was 0.93 (95% CI: 0.90, 0.97) (Figure) if human graders were the reference standard.

Conclusions : Predictions of RNFL thickness from the M2M DL algorithm applied to fundus photographs were able to successfully discriminate glaucoma from non-glaucoma in the setting of an existing teleretinal screening program.

This is a 2020 ARVO Annual Meeting abstract.

 

Figure. Receiver operating characteristic curve to discriminate glaucoma from no-glaucoma for the machine-to-machine deep learning algorithm.