Machine Learning (ML) for Predicting Visual Acuity (VA) Outcomes in the Comparison of Age-related Macular Degeneration (AMD) Treatment Trials (CATT)

Abstract

Purpose : Predicting VA can help set treatment outcomes expectations and plan for an optimal treatment course. This project is to evaluate various ML models for predicting 2-year VA responses to anti-vascular endothelial growth factor (anti-VEGF) treatment for neovascular AMD (nAMD) in the CATT.

Methods : This is a secondary analysis of CATT data publicly available at https://hyperprod.cceb.med.upenn.edu/catt/catt_index.php. Four ML models [support-vector machine (SVM), random forest (RF), extreme gradient boosting (XGBoost), and multi-layer perceptron (MLP) neural network] were evaluated for predicting four VA outcomes at 2 years (≥3-line VA gain, ≥3-line VA loss, VA, VA change from baseline). ML models using clinical and image data from baseline only and up to week 12 were assessed by the area under the receiver operating characteristic (ROC) curve (AUC) for predicting ≥3-line VA gain and loss, by R² and mean absolute error (MAE) for predicting VA change from baseline and actual VA at 2 years. The CATT data from 1029 participants were randomly split for training (n=717) and for final validation (n=312).

Results : Of the 1029 participants, the mean (SD) VA and VA change from baseline at 2 years were 67.3 (18.2) and 6.4 (16.5) letters, respectively, with 30% and 9% having ≥3-line VA gain and loss, respectively. Using baseline data, the ML models from cross-validation achieved AUCs of 0.77-0.79 and 0.57-0.61 for predicting ≥3-line VA gain and loss, respectively, with R² of 0.03-0.07 (MAE=11.1-11.8 letters) and 0.17-0.24 (MAE=11.0-12.1 letters) for predicting VA change and actual VA in 2 years. Using data up to week 12, the AUCs for predicting ≥3-line VA gain and loss increased to 0.84-0.85 and 0.58-0.73, respectively, and the R² increased to 0.24-0.27 (MAE=9.1-9.8 letters) and 0.37-0.40 (MAE=9.3-10.2 letters) for predicting VA change and actual VA, respectively (Table 1). In final validation on the test dataset using week 12 data, the models had AUCs of 0.85-0.87 and 0.67-0.79 for predicting ≥3-line VA gain and loss, respectively. The models had an R² of 0.33-0.36 (MAE=8.9-9.9 letters) and R² of 0.37-0.43 (MAE=8.8-10.2 letters) for predicting VA change and actual VA, respectively (Table 2).

Conclusions : Using the baseline data, ML models did not predict well for 2-year VA outcomes, however, including data up to 12 weeks substantially improved predictions.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.