June 2020
Volume 61, Issue 7
Free
ARVO Annual Meeting Abstract  |   June 2020
Meta-learning approach to automatically register multivendor retinal images
Author Affiliations & Notes
  • Ali Hasan
    Biomedical Engineering, Duke University, Durham, North Carolina, United States
  • Zengtian Deng
    Biomedical Engineering, Duke University, Durham, North Carolina, United States
  • Jessica Loo
    Biomedical Engineering, Duke University, Durham, North Carolina, United States
  • Dibyendu Mukherjee
    Biomedical Engineering, Duke University, Durham, North Carolina, United States
  • Jacque L Duncan
    Opthalmology, University of California San Francisco, San Francisco, California, United States
  • David G Birch
    Retina Foundation, Dallas, Texas, United States
  • Glenn J Jaffe
    Opthalmology, Duke University, Durham, North Carolina, United States
  • Sina Farsiu
    Biomedical Engineering, Duke University, Durham, North Carolina, United States
  • Footnotes
    Commercial Relationships   Ali Hasan, None; Zengtian Deng, None; Jessica Loo, None; Dibyendu Mukherjee, None; Jacque Duncan, 4D Therapeutics (C), Acucela (F), AGTC (C), Allergan (F), Biogen/Nightstar (F), Biogen/Nightstar Therapeutics (C), Editas Medicine (C), Eloxx (C), Foundation Fighting Blindness (C), ProQR Therapeutics (C), SparingVision (C), Spark Therapeutics (C), Vedere Bio (C); David Birch, None; Glenn Jaffe, None; Sina Farsiu, Google (F)
  • Footnotes
    Support  The source of the data is the FFB Consortium, but the analyses, content and conclusions presented herein are solely the responsibility of the authors and have not been reviewed or approved by the Consortium and may not reflect the view of FFB.
Investigative Ophthalmology & Visual Science June 2020, Vol.61, 1634. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Ali Hasan, Zengtian Deng, Jessica Loo, Dibyendu Mukherjee, Jacque L Duncan, David G Birch, Glenn J Jaffe, Sina Farsiu; Meta-learning approach to automatically register multivendor retinal images. Invest. Ophthalmol. Vis. Sci. 2020;61(7):1634.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : A new, automated approach to register multivendor retinal images.

Methods : The dataset consisted of 85 eyes enrolled in an international, multi-center clinical trial (NCT03146078) characterizing the natural progression of USH2A-related retinal degeneration (Usher syndrome type 2A or non-syndromic retinitis pigmentosa). The training set included scanning laser ophthalmoscopic (SLO) images simultaneously acquired with OCT where the SLO images were manually registered to microperimetry SLO images. We used 42 pairs of images for training and 43 for testing. We trained a convolutional neural network (CNN) to predict the transformation matrix for an affine transformation using a spatial transformer network (M. Jaderberg et al., NeurIPS. 2015) and trained an additional CNN as a discriminator to estimate the quality of the registration. Joint training was achieved using the same approach as described in the generative adversarial network approach of M. Arjovsky et al. (arXiv:1701.07875 2017). We finally used an iterative optimization algorithm based on model agnostic meta-learning (C. Finn et al. ICML, 2017) to refine the transformation based on the discriminator network.

Results : The performance of the proposed method was first compared to a corresponding non-iterative deep learning method. For a fair comparison on multivendor image data, we modified the approach by (G. Balakrishnan et al. IEEE TMI 2019) to predict an affine transformation and to supervise the training. We also compared our results to the state of the art non-deep learning registration method (GFEMR) described by (J. Wang et al. Signal Processing 157 2019). We considered the root mean squared error (RMSE) between the manually marked points on the moving OCT SLO image and the fixed microperimetry SLO image as our criteria for success. The median RMSE for the 43 test images was 106.8 ± 19.7 before registration, 34.7 ± 10.5 using GFEMR, 17.1 ± 5.6 using the noniterative network, and 13.9 ± 4.5 for our method.

Conclusions : We demonstrate a novel method for multivendor retinal image registration. Following the encouraging results in this pilot experiment, more work is needed to achieve manual level accuracy for automatic registration.

This is a 2020 ARVO Annual Meeting abstract.

 

Figure 1: Qualitative comparison of the registered overlay of different registration methods compared to median performance of our method. The magenta image is the moving image, and the green image is the fixed image.

Figure 1: Qualitative comparison of the registered overlay of different registration methods compared to median performance of our method. The magenta image is the moving image, and the green image is the fixed image.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×