Evaluation of Landmark Localization Models for Fundus Imaging Conditions

Abstract

Purpose : This study aims to investigate and evaluate the volatility of deep learning methods for the localization of the fovea and disc in fundus images under various imaging conditions. Such a benchmark is essential for the development of new landmark localization models that can work in different real imaging conditions including brightness, contrast, lens effects, etc.

Methods : In this study, we created a large, robust dataset combining 8 publicly available datasets of fundus images. This dataset was split into a train and test set, consisting respectively of 2328 and 732 fundus images, with no subject overlap. The test set images are also perturbed to 56048 fundus images by various imaging perturbations, including brightness, focus, contrast, noise, and illumination spots, of 5 levels of severity. We test two heat-map based localization models, the HBA UNET (Tang. et al), the current state of the art, and the UNET, the most commonly used model for landmark localization. Euclidean Distance was used to evaluate model performance.

Results : The results obtained by the HBA UNET for fovea detection were consistent with the results of the original study, considering the differences in image sizes. Fig. 1 shows box plots for the performance of the models on fundus images with and without perturbation. While both models experienced a decline in performance when exposed to perturbation, the HBA UNET generally performed better on average for both the fovea and the disc (ED Fovea 8.97 +/- 23.77 declined to 11.62 +/- 31.81) than the UNET (ED Fovea 10.56 +/- 26.45 declined to 15.61 +/- 29.25), as demonstrated in Fig. 2 B. Furthermore, it was observed that the localization of the fovea (ED 8.97 +/- 23.77 vs 10.56 +/- 26.45), was more challenging for both the HBA UNET and the UNET compared to the localization of the disc (ED 2.19 +/- 1.81 vs 6.27 +/- 2.51), as illustrated in Fig. 2.

Conclusions : The evaluation of models confirmed the performance observed in previous studies and established new benchmarks for disk and fovea localization. Both models displayed poor performance on perturbed images, with the HBA UNET showing better performance across most types and severities of perturbations. Due to this volatility and sensitivity to imaging conditions, it is recommended to explore the development of perturbation-resistant models in the future. We share the source codes and trained models to the public.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.