Evaluating the Utility of Deep Learning Using Ultra-widefield Fluorescein Angiography for Predicting Need for Anti-VEGF Therapy in Diabetic Eye Disease

Abstract

Purpose : Deep learning (DL) is a class of machine learning that utilizes neural networks to generate representations that allow for distinguishing categories of interest. Being an unsupervised feature generation method, large training sets are typically needed to learn appropriate representations to discriminate the categories of interest. While DL has been extensively explored in ophthalmology for diagnostic applications, it has not been extensively evaluated for predicting need for future treatment, arguably a more challenging problem compared to disease diagnosis and one where large imaging datasets with accompanying treatment response and outcome information may not be available. The purpose of this study was to investigate the ability of DL to predict need for anti-VEGF from ultra-widefield angiograms (UWFA) in eyes with diabetic retinopathy (DR).

Methods : A retrospective image analysis study was conducted on eyes with DR that had UWFA imaging. DL was applied to the late phase UWFA images to classify eyes as needing anti-VEGF or not. In order to evaluate the impact of sample size on DL performance, the study set was sampled into five subsets of increasing size. For each subset experiment, a class activation map (CAM) was generated per sample to identify regions of interest (ROI) that the DL model places attention on, when making its predictions.

Results : Two-hundred seventeen eyes from 189 patients were included. 141 eyes required anti-VEGF treatment. Five subsets with balanced class distributions were generated from this dataset. Subsets increase by a factor of 28 in size from 28 to 140 eyes. The 3-fold cross-validated AUROC indicates minimal correlation between dataset size and model performance. The best performing model had an average AUROC of 0.628 ± 0.087 over the subsets. Resulting CAMs frequently demonstrated inconsistent identified ROI across subsets suggesting that DL tended to be sensitive to the choice of training cases across every fold.

Conclusions : In this study, regardless of sample size, DL was unable to consistently identify image representations from limited UWFA samples to predict need for anti-VEGF therapy. These findings suggest that other handcrafted radiomic approaches, multi-modal DL, and integration of multiple features might need to be considered for predicting need for treatment.

This is a 2021 ARVO Annual Meeting abstract.