Deep Learning Prediction of Glaucoma Progression using Multimodal Transformers and Longitudinal Clinical Data

Abstract

Purpose : There is a need for objective, reliable, approaches that can identify glaucoma progressors and facilitate therapy. One limitation of most existing deep learning glaucoma progression models is that they rely solely on single modalities, such as visual field tests or retinal imaging. Another limitation is that most models are not longitudinal, only relying on data taken from a single exam. Single modality, single timepoint approaches may not provide a comprehensive understanding of disease progression, and pose a great obstacle towards building models capable of predicting glaucoma progression. To overcome these limitations we propose a multimodal deep learning model to identify functional visual field based glaucoma progression from baseline 24-2 visual field and patient demographics, clinical measurements, Fundus photographs, OCT, and RNFL thickness captured over multiple follow-up visits.

Methods : Based on at least three 24-2 VF tests, 120 glaucoma patients (138 eyes) were classified as fast progressors (1.0 dB/year or worse mean deviation loss) and 310 glaucoma patients (380 eyes) were classified as stable (less than 1.0 dB/year of loss). Several multimodal transformer models were trained on various data modalities. In addition to baseline OCT scans, RNFL thickness maps, baseline VF, baseline fundus photographs, baseline eye measurements (IOP, SE, AL, CCT), patient demographics (age, sex, race), systemic conditions, and medication history were incorporated into model training. Longitudinal OCT and fundus imaging from follow-up visits were also incorporated.

Results : Models using baseline data achieved an AUC of 0.74 in identifying fast progressors. Adding imaging from one or two follow-up visits increased AUC to 0.83 and 0.86, respectively. Attention vectors reveal relatively equal attention on all data modalities provided as input to the model. Attention vectors on OCT scans reveal model attention focused on the RNFL and choroid. Attention vectors on fundus reveal model attention focused on the optic nerve head. False positive and false negative examples reveal greater attention focused outside the ONH of fundus images.

Conclusions :
Multimodal transformer models identified fast progressors with strong accuracy and could help clinicians identify patients before unmanageable vision loss occurs.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Model performance with only baseline data vs with follow-up data.

 

Model attention maps.