June 2021
Volume 62, Issue 8
Open Access
ARVO Annual Meeting Abstract  |   June 2021
A multimodal deep learning (DL) model to predict visual acuity response in the CATT study
Author Affiliations & Notes
  • Jelena Novosel
    F Hoffmann-La Roche AG, Basel, Basel-Stadt, Switzerland
  • Neha Anegondi
    Genentech Inc, South San Francisco, California, United States
  • Jian Dai
    Genentech Inc, South San Francisco, California, United States
  • Ebenezer Daniel
    University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
  • Glenn J Jaffe
    Duke University, Durham, North Carolina, United States
  • Maureen G Maguire
    University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
  • Jeffrey R Willis
    Genentech Inc, South San Francisco, California, United States
  • Footnotes
    Commercial Relationships   Jelena Novosel, F. Hoffmann-La Roche (E); Neha Anegondi, Genentech Inc. (E); Jian Dai, Genentech Inc. (E); Ebenezer Daniel, Novartis (C); Glenn Jaffe, EyePoint Pharmaceuticals (C), Iveric (C), Neurotech (C), Novartis (C), Regeneron (C); Maureen Maguire, F. Hoffmann-La Roche (C), Genentech Inc. (C); Jeffrey Willis, Genentech Inc. (E)
  • Footnotes
    Support  F. Hoffmann-La Roche Ltd., Basel, Switzerland, provided support for the study and participated in the study design; conducting the study; and data collection, management, and interpretation.
Investigative Ophthalmology & Visual Science June 2021, Vol.62, 81. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jelena Novosel, Neha Anegondi, Jian Dai, Ebenezer Daniel, Glenn J Jaffe, Maureen G Maguire, Jeffrey R Willis; A multimodal deep learning (DL) model to predict visual acuity response in the CATT study. Invest. Ophthalmol. Vis. Sci. 2021;62(8):81.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : We developed DL models to predict visual acuity response (VAR) to ranibizumab (RBZ) by using baseline (BL) characteristics and color fundus images (CFIs) of patients with neovascular age-related macular degeneration.

Methods : VAR was formulated as a classification problem with 4 classes (<5, 5–9, 10–14, and ≥15 letters); each class was assigned based on best-corrected visual acuity (BCVA) change from BL to month 12. To solve the classification problem, we designed 3 DL models that processed data from different modalities. Single models were trained to process BL characteristics, including BCVA, age, and optical coherence tomography (OCT) imaging biomarkers (Fig 1A), or CFI (Fig 1B); the third model (multimodal model; Fig 1C) fused the 2 subnetworks to produce the final classification.
This is a retrospective analysis of BL data from 284 patients receiving RBZ monthly treatment in the CATT study (NCT00593450). The distribution across the 4 classes was imbalanced, with 64, 43, 52, and 125 patients in classes 1, 2, 3, and 4, respectively. Performance was assessed based on validation (N=56) and test (N=57) data subsets using accuracy and area under the receiver operating characteristic (AUROC) curve. As accuracy and AUROC can be misleading in an imbalanced dataset, we reported F1 score (macro [m] and per-class) and area under the precision-recall (AUCPR) curve to provide a more informative assessment of model performance.

Results : Evaluation of model performance is shown in Table 1. Performance measures varied considerably among the 3 models (eg, mF1 scores of the test dataset were 0.332, 0.236, and 0.354 for BL characteristic, CFI, and multimodal models, respectively). Additionally, individual per-class results showed large variation, reflecting the presence of a strong class imbalance in the data.

Conclusions : The 3 models showed limited ability to predict VAR to RBZ, but the BL characteristics and multimodal models outperformed the CFI model. Compared with the BL characteristics model, the multimodal model showed slightly better performance. More research is required to explore whether the predictive ability of the multimodal model can be further improved.

This is a 2021 ARVO Annual Meeting abstract.

 

Fig. Illustration of architectures of the DL models

Fig. Illustration of architectures of the DL models

 

Table 1. Evaluation of model performance on validation and test datasets

Table 1. Evaluation of model performance on validation and test datasets

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×