Evaluation of Robustness of Disc/Cup Segmentation in Different Fundus Imaging Conditions

Abstract

Purpose : Deep learning (DL) methods have become popular for segmenting the optic disc (OD) and cup (OC) from fundus images and are essential for many tasks including glaucoma diagnosis. In clinical practice, it is unlikely that fundus images captured with varying camera conditions will be of similar quality. Thus, it is crucial that segmentation models are robust to different imaging conditions. Here, we evaluate the volatility of different segmentation models with such perturbations.

Methods : We implement three segmentation DL architectures: UNet++ (Zhou et al., 2018), DeepLabV3+ (Chen et al., 2018), and CE-Net (Gu et al., 2019). The former two are popular models, whereas CE-Net is well-known for fundus segmentation. We combined 4 public datasets and split them into train and test sets of 1,802 and 289 images, respectively. The methods were re-implemented and made available to other investigators. All models were trained using Dice loss and evaluated using Intersection over Union (IoU). For imaging condition simulation, we perturbed test images with de-illumination and de-spot from Shen et al., 2021, and the perturbations used in ImageNet-C (Hendrycks & Dietterich, 2019) with five severity levels such as varieties of blurriness, noise, and saturation, etc.

Results : After applying perturbations to the test set, we made 24,558 perturbed images. The achieved IoUs are shown in Fig. 1. While all models showed decreases in performance on the perturbed images, CE-Net (IoU ± standard deviation – OC: 0.857 ± 0.089 before, 0.846 ± 0.093 after perturbations; OC: 0.782 ± 0.127 before, 0.770 ± 0.131 after) achieved the greatest IoU while also being the most resistant to perturbations and severity changes (Fig. 1C). Figs. 2A and 2B show examples of strong perturbations altering the predicted mask. Overall, the most impactful perturbations were brightness, contrast, de-illumination, and hue (Fig. 2C).

Conclusions : Our evaluation of deep learning models for disc/cup segmentation confirms the negative impact of different fundus imaging conditions on performance, though fundus-specific models such as CE-Net were more robust than general models. We also found that certain perturbations were more impactful than others. Due to this volatility, we suggest further exploration of imaging conditions and considering robustness in future segmentation model evaluation. For this reason and for more convenience, we share our source codes.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.