Multi-modal Geographic Atrophy Lesion Growth Rate Prediction using Deep Learning

Abstract

Purpose : To predict geographic atrophy (GA) lesion growth rate from fundus autofluorescence (FAF) and spectral-domain optical coherence tomography (OCT) using a multi-modal multi-task deep learning approach.

Methods : This study was performed retrospectively on study eyes of patients with bilateral GA enrolled in the lampalizumab clinical trials. The macular Spectralis (Heidelberg Engineering, Inc. Heidelberg, Germany) OCT volumes of 496x1024x49 voxels and macular 30 degree FAF images of 768x768 pixels from baseline visits were used to predict baseline GA lesion area and annualized GA growth rate. For OCT volumes, each B-scan was flattened along Bruch’s membrane (BM) and three en-face maps averaged over full, sub-BM and above-BM depths were combined as a three-channel input. The growth rate (mm²/year) of GA lesion area (mm², measured from FAF images by an independent reading center and graded by two readers and an adjudicator if necessary) was estimated as the slope from a linear regression model fitted for each individual patient using all available visits. The datasets from 1279 patients were divided into 80% training and 20% test datasets. Further, the training data were randomly divided into 5 cross-validation (CV) folds. Three multi-task convolutional neural network (CNN) models were trained to simultaneously predict GA lesion area and GA growth rate: OCT only, FAF only and multi-modal (OCT and FAF). The performance was evaluated by calculating the in-sample coefficient of determination (R²) defined as the square of Pearson correlation coefficient (r) between true and predicted GA area and growth rate.

Results : Table 1 shows the CV and test set performance of the three multi-task CNN models: OCT only, FAF only and multi-modal input. The multi-modal model had the best CV performance with mean R² of 0.92 and 0.46 for GA lesion area and GA growth rate predictions, respectively. On the test set, the same model showed R² of 0.94 for GA lesion area prediction and 0.56 for GA growth rate prediction.

Conclusions :
These findings show the feasibility of using baseline FAF and/or OCT images to predict individual GA growth rate with a multi-task deep learning approach and that a multi-modal approach could improve performance. This work can potentially be useful for clinical trial adjustment, stratification and/or enrichment. Meanwhile, further validation in additional datasets is needed to confirm robust performance.

This is a 2021 ARVO Annual Meeting abstract.