A Unified Framework for Visual Field Test Estimation and Forecasting using Convolution and Attention Networks

Abstract

Purpose : Different methods have been reported for visual field (VF) estimation from optical coherence tomography (OCT) or forecasting future VF using prior VFs. These methods are modality-specific and hard to compare with each other. Our goal is to test the efficacy of our unified framework to estimate and forecast VF using different input data (2D or 3D OCT, and VF).

Methods : From our longitudinal glaucoma cohort, we collected 8,390 pairs of 2 consecutive VFs (Humphrey, 24-2 SITA Standard, Zeiss, Dublin, CA) and their corresponding 3D OCT images (Cirrus HD-OCT, 200x200 ONH Scan, Zeiss). The average number of days between sessions was 342 days (range 90-2400). The dataset was split to perform 10-fold cross-validation without patient overlap. We utilized a hybridized convolution and transformer network (CoTrNet) architecture (Table 1) composed of inverted residual convolution and transformer (relative self-attention and a fully connected layer) blocks to capture local and global patterns. In VF estimation, either a 2D (e.g. en face image, layer thickness map, etc.) or a down-sampled 3D OCT image can be used to estimate the corresponding VF (52 out of 54 values, excluding 2 blind spots). In VF forecasting, the input is made up of the current VF and the time difference (between the current and future VFs).

Results : In VF estimation with enface images, 2D ResNet and CoTrNet achieved the global mean absolute error (MAE) of 3.91 ± 0.24, and 3.52 ± 0.26 dBs, respectively. However, the overall performance was improved by using 3D OCT images. In Figure 1, MAE and its pointwise heatmap are reported for the compared methods. It shows that 3D ResNet’s error was 3.39 ± 0.21 while CoTrNet’s error was 3.10 ± 0.15. In VF forecasting, CoTrNet (MAE=2.10 ± 0.11) significantly outperformed the identity function (2.70 ± 0.15), recurrent neural network (RNN) (2.54 ± 0.21), and CascadeNet-5 (2.27 ± 0.13) methods. Our analysis showed that VF forecasting performance stayed stable until the time interval hit 4 years.

Conclusions : Our unified framework supported the use of 2D and 3D OCT, as well as VF, as inputs. The proposed model outperformed problem- and modality-specific methods for both VF estimation and forecasting by harnessing the power of local and global processing via the integration of convolutions and transformers in the network.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.