Predicting 2-year and 5-year Late AMD Progression using Deep Learning with Longitudinal Fundus Images

Abstract

Purpose : Accurately predicting a patient’s risk of progressing to late age-related macular degeneration (AMD) is difficult but crucial for personalized medicine. While current risk prediction models are useful, none utilizes a longitudinal series of images in patients’ history to estimate the risk of late AMD in a given time interval. In this work, we seek to evaluate how deep neural networks capture the sequential patterns of longitudinal fundus images and improve the prediction of 2-year and 5-year risks of progression to late AMD.

Methods : We developed a hybrid architecture that consists of a Convolutional Neural Network and a Recurrent Neural Network, popularly known as CNN-RNN (Fig 1). Specifically, we used a ResNet to extract features from the images and applied an LSTM layer on top for the prediction. Two training strategies were used: (1) we used the ResNet trained on a late AMD detection task as a fixed feature extractor, and (2) we fine-tuned the weights of ResNet together with the LSTM in an end-to-end manner. We also compared our model with a ResNet which predicts AMD progression using a single image.

Results : We trained and evaluated the models using longitudinal fundus images from the Age-Related Eye Disease Study (AREDS). The AREDS dataset includes 4,315 participants, a large majority of whom had annual visits over 10 years. The dataset was split into train/validation sets with a 0.7:0.3 ratio. We evaluated models using the area under the receiver operating characteristic curve (AUC). Table 1 shows that the CNN-RNN model outperforms the baseline on 2-year (0.926 vs. 0.892) and 5-year (0.904 vs. 0.829) predictions. In the meantime, our model’s performance increases with up to 5 visits, but drops with more visits in patient history. This might be due to the data imbalance and sparsity issues. We also observed that the end-to-end training did not show significant improvement.

Conclusions : We presented a CNN-RNN model for predicting late AMD progression for each patient in this study. Our experiments show that our model can capture dynamic sequential patterns and outperforms baselines. This study sheds light on the potential of deep learning to facilitate and support clinical decision-making on an individual basis.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Figure 1. The archietecture of (A) CNN-RNN and (B) ResNet model.

 

Table 1. AUC on predicting 2-year and 5-year late AMD progression on longitudinal fundus images.