Meta-learning approach to automatically register multivendor retinal images

Abstract

Purpose : A new, automated approach to register multivendor retinal images.

Methods : The dataset consisted of 85 eyes enrolled in an international, multi-center clinical trial (NCT03146078) characterizing the natural progression of USH2A-related retinal degeneration (Usher syndrome type 2A or non-syndromic retinitis pigmentosa). The training set included scanning laser ophthalmoscopic (SLO) images simultaneously acquired with OCT where the SLO images were manually registered to microperimetry SLO images. We used 42 pairs of images for training and 43 for testing. We trained a convolutional neural network (CNN) to predict the transformation matrix for an affine transformation using a spatial transformer network (M. Jaderberg et al., NeurIPS. 2015) and trained an additional CNN as a discriminator to estimate the quality of the registration. Joint training was achieved using the same approach as described in the generative adversarial network approach of M. Arjovsky et al. (arXiv:1701.07875 2017). We finally used an iterative optimization algorithm based on model agnostic meta-learning (C. Finn et al. ICML, 2017) to refine the transformation based on the discriminator network.

Results : The performance of the proposed method was first compared to a corresponding non-iterative deep learning method. For a fair comparison on multivendor image data, we modified the approach by (G. Balakrishnan et al. IEEE TMI 2019) to predict an affine transformation and to supervise the training. We also compared our results to the state of the art non-deep learning registration method (GFEMR) described by (J. Wang et al. Signal Processing 157 2019). We considered the root mean squared error (RMSE) between the manually marked points on the moving OCT SLO image and the fixed microperimetry SLO image as our criteria for success. The median RMSE for the 43 test images was 106.8 ± 19.7 before registration, 34.7 ± 10.5 using GFEMR, 17.1 ± 5.6 using the noniterative network, and 13.9 ± 4.5 for our method.

Conclusions : We demonstrate a novel method for multivendor retinal image registration. Following the encouraging results in this pilot experiment, more work is needed to achieve manual level accuracy for automatic registration.

This is a 2020 ARVO Annual Meeting abstract.

 

Figure 1: Qualitative comparison of the registered overlay of different registration methods compared to median performance of our method. The magenta image is the moving image, and the green image is the fixed image.