June 2023
Volume 64, Issue 8
Open Access
ARVO Annual Meeting Abstract  |   June 2023
Multimodal Ophthalmic Image Registration: A Generalizable Framework Based on Image Synthesis using Cycle Generative Adversarial Networks
Author Affiliations & Notes
  • Sandeep Chandra Bollepalli
    University of Pittsburgh, Pittsburgh, Pennsylvania, United States
  • Adarsh Gadari
    UNC Greensboro, Greensboro, North Carolina, United States
  • Raveena Arasikere
    UNC Greensboro, Greensboro, North Carolina, United States
  • Aditi Darandale
    UNC Greensboro, Greensboro, North Carolina, United States
  • Shan Suthaharan
    UNC Greensboro, Greensboro, North Carolina, United States
  • Kunal K Dansingani
    University of Pittsburgh, Pittsburgh, Pennsylvania, United States
  • Jose Sahel
    University of Pittsburgh, Pittsburgh, Pennsylvania, United States
  • Jay Chhablani
    University of Pittsburgh, Pittsburgh, Pennsylvania, United States
  • Kiran Kumar Vupparaboina
    University of Pittsburgh, Pittsburgh, Pennsylvania, United States
  • Footnotes
    Commercial Relationships   Sandeep Chandra Bollepalli None; Adarsh Gadari None; Raveena Arasikere None; Aditi Darandale None; Shan Suthaharan None; Kunal Dansingani None; Jose Sahel Avista RX, Code C (Consultant/Contractor), GenSight Biologics, Sparing Vision, Prophesee, Chronolife, Tilak Healthcare, VegaVect Inc., Avista, Tenpoint, SharpEye, Code I (Personal Financial Interest), Unpaid censor on the board of GenSight Biologics and SparingVision; Censor on the board of Avista, Chair advisory board of SparingVision, Tenpoint, Institute of Ophthalmology Basel (IOB); President of the Fondation Voir & Entendre; Director board of trustees RD Fund (Foundation Fighting Blindness), Gilbert Foundation advisory board, Code S (non-remunerative); Jay Chhablani None; Kiran Vupparaboina None
  • Footnotes
    Support  The work was supported by the NIH CORE Grant P30 EY08098 to the Dept. of Ophthalmology, the Eye and Ear Foundation of Pittsburgh; the Shear Family Foundation Grant to the University of Pittsburgh Department of Ophthalmology; and an unrestricted grant from Research to Prevent Blindness, New York, NY; and partly by Grant BT/PR16582/BID/7/667/2016,
Investigative Ophthalmology & Visual Science June 2023, Vol.64, 5451. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Sandeep Chandra Bollepalli, Adarsh Gadari, Raveena Arasikere, Aditi Darandale, Shan Suthaharan, Kunal K Dansingani, Jose Sahel, Jay Chhablani, Kiran Kumar Vupparaboina; Multimodal Ophthalmic Image Registration: A Generalizable Framework Based on Image Synthesis using Cycle Generative Adversarial Networks. Invest. Ophthalmol. Vis. Sci. 2023;64(8):5451.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : Multimodal imaging including color fundus (CF), fundus autofluorescence (FAF), and fluorescein angiography (FA) is crucial in the accurate diagnosis of various posterior segment eye ailments such as diabetic retinopathy. Registering such multimodal data assumes significance in quantitative comparison and monitor disease progression across different modalities. However, owing to inherent differences pertinent to each specific modality, such as color representation, tissue reflectivity profile, and resolution, multimodal registration remains challenging. In this regard, we propose a novel cycle generative adversarial networks (cycle-GAN) approach. In particular, we transform the images of current modality to the desired targeted modality and register the real and synthesized images in the same feature space. We demonstrate the proposed method by registering CF and FAF images.

Methods : This study uses retrospectively obtained 108 FAF and 123 CF images which includes both healthy and diseased subjects. To register FAF with CF, we first transformed the CF images to an equivalent FAF representation. To this end, we adopted cycle-GAN which demonstrated the photo-realistic transformation of natural images from one domain to the desired target domain. We used residual encoder-decoder architecture in the generator module of the cycle-GAN. Data augmentation is performed to increase training data. Synthesized FAF representation from the CF image and the corresponding paired FAF images are registered using affine shift-invariant feature transform (ASIFT) corresponding points and geometric transformation (Figures 1 and 2). The accuracy of the synthesis is evaluated based on subjective grading on a scale of 0-100 and the performance of registration is evaluated using mean pixel error (MPE) of manually selected corresponding points of CF-FAF pair using MATLAB ‘cpselect’ tool (Figure 2).

Results : The proposed cycle-GAN method achieved a mean subjective grading score 88% and the MPE for registration is less than 10 pixels.

Conclusions : The proposed method demonstrated accurate multi-modal image registration of CF and FAF images, and is generalizable to other modalities.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.

 

(a) ASIFT based feature matching for automated registration. (b) Manual feature matching for quantifying registration performance.

(a) ASIFT based feature matching for automated registration. (b) Manual feature matching for quantifying registration performance.

 

CF-FAF registration using Cycle-GAN.

CF-FAF registration using Cycle-GAN.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×