Quantification of Hard Exudates Using Color Fundus Photographs based on Deep Learning Pix2Pix GAN Image Translation Model

Abstract

Purpose : Diabetic retinopathy is one of the leading causes of blindness. Hard exudates (HEs) are one of the important clinical indicators of the presence of DR and they can be visualized using color fundus (CF) photographs. Accurate segmentation of hard exudates assumes significance in the quantification and monitoring of DR progression. In this study, we attempted a novel image translation method based on Pix2Pix generative adversarial networks (GAN) deep learning model to segment HEs in CF.

Methods : This study is performed based on a retrospective dataset of 150 color fundus images taken from subjects diagnosed with DR. We adopted Pix2Pix GAN model which is developed for translating an image pixel-by-pixel into any targeted image. Here we plan to translate CF image to the corresponding HE-segmented CF image (Figure 1). For the generator, we used the residual encoder-decoder (ResUNet) model. Ground-truth segmentations required for training were obtained based on our previously validated semi-automated method. In segmented images for training, HEs were marked in blue and Pix2Pix learns the blue regions. Both generator and discriminator are optimized for mean squared error (MSE) loss functions. Train-test split was 120:30. Segmentation performance is evaluated using Dice coefficient (DC) between the ground-truth and the Pix2Pix GAN segmentations.

Results : On the 30 eyes, the proposed method achieves an average Dice score of 91.47% against ground truth segmentation. Figure 2 depicts the comparison between ground-truth and algorithmic segmentation on representative CF images.

Conclusions : The proposed Pix2Pix-GAN-based approach demonstrated close agreement with ground-truth segmentation. This method remains generalizable and can be adapted to other segmentation tasks.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.

 

Figure 1: Schematic of proposed HE segmentation using CF images based on Pix2Pix GAN.

 

Figure 2: Comparison of Pix2Pix based HE segmentation against ground-truth on representative CF images.