Automated Machine Learning vs. Inception for the Autonomous Detection of Disc Hemorrhage

Abstract

Purpose : To compare automated machine learning (AutoML) to transfer learning (Inception) for disc hemorrhage (DH) identification.

Methods : This study was a retrospective analysis of fundus photographs obtained from the Ocular Hypertension Treatment Study, New York Eye and Ear Infirmary of Mt. Sinai, Massachusetts Eye and Ear, Thessaloniki Eye Study, and GONE datasets. High magnification fundus photographs of different resolutions and sources centered on the optic disc were included. Ground truth was established by the consensus of grades from two masked glaucoma specialists. Images were graded independently and disagreements in grading were resolved via adjudication, ungradable images were excluded. The complete dataset was split into training (80%), validation (10%), and test (10%). An automated machine learning model (AutoML), which tunes model hyperparameters without user engagement, was trained to distinguish disc hemorrhage (DH) positive and DH negative images. After training and validation, AutoML evaluated the model against a subset of test images, providing Precision and Recall data for each confidence threshold from 0 to 1. This process yields a binary classification model that predicts DH based on the returned probability at a given confidence threshold. A hard-coded convolutional neural network model based on the widely used InceptionV3 architecture was trained on the same data set. The performance of each model on an identical test set was then evaluated.

Results : A total of 897 images were included, 314 DH positive and 583 DH negative. The AutoML model achieved an area under the precision-recall curve (AUPRC) of 0.923, a sensitivity of 85% and specificity and 86%. For the hard-coded deep learning model, AUPRC was 0.82 with a sensitivity and specificity of 83% and 60%, respectively. The Inception model attention maps help visualize what parts of an image contributed most to a prediction (Fig 1).

Conclusions : The AutoML algorithm performance was high and showed good concordance with a traditional hard-coded algorithm. AutoML offers a low clearance method for clinicians without programming expertise to develop and deploy deep learning solutions.

This is a 2021 ARVO Annual Meeting abstract.

 

Fig 1. Attention maps demonstrate the pixels of disc images that contributed most to a prediction. Applying the disc photo's attention map reveals that the most important areas for this prediction are the inferotemporal disc hemorrhage and peripapillary areas.