Predicting functional outcomes for different treatment durations of faricimab in diabetic macular edema (DME)

Abstract

Purpose : To evaluate the performance of machine learning (ML) models that use baseline features to predict functional outcomes for different treatment durations (short-, mid-, and long-term) of faricimab in patients with DME.

Methods : Data from patients in the phase 3 YOSEMITE (NCT03622580) and RHINE (NCT03622593) trials who received faricimab 6.0 mg every 8 weeks were pooled and split into 70% development, 15% test, and 15% holdout sets. The development set was then split into 5 folds for cross-validation (CV). Patient data were further filtered based on the criteria shown in Fig 1. The target functional outcome was best-corrected visual acuity (BCVA) letter score. The target timepoints were week 4, week 24, and year 1 (BCVA averaged over weeks 48, 52, and 56) of faricimab treatment. Only baseline features were used to predict BCVA over time. Input features were structured into tiers to understand the contribution of different sets of features (Fig 2). ElasticNet, random forest, support vector machine, and eXtreme Gradient Boosting tree were trained and evaluated. The hyperparameters of each pair of model and input feature set were optimized via CV on the development set. The models and input feature sets that showed the best performance on the test set were selected for evaluation on the holdout set for each target timepoint. Model performance was evaluated using root mean square error (RMSE).

Results : Based on the test results, the random forest model with tier-1 input was selected for week 4 prediction, and the ElasticNet model with tier-1A input was selected for week 24 and year 1 prediction. RMSEs on the holdout set (% increase from RMSE on the test set) for week 4, week 24, and year 1 prediction were 8.26 (29%), 7.80 (22%), and 13.15 (99%), respectively. In the holdout set, the Spearman’s correlations of residuals at the 3 timepoints were statistically significant for all pairs (P<0.0001 for each pair).

Conclusions : ML models showed robust performance in predicting BCVA for week 4 and week 24. The year 1 BCVA prediction was more complex, and ML models did not generalize well to the data in the holdout set. Statistically significant correlations of residuals suggest that the observed model performance is improvable. More advanced modeling, new input features, or larger data sets may address this performance gap.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Fig 1

 

Fig 2