Generalizability of foundation models for geographic atrophy (GA) lesion segmentation in fundus autofluorescence (FAF)

Abstract

Purpose : High-capacity, transformer-based, deep-learning models trained in a self-supervised manner on large datasets, referred to as foundation models (FMs), may offer generalizability to unseen domains/tasks, suggesting the potential to accelerate algorithm development, particularly with limited data. We evaluated the generalizability of 2 FMs—DINOv2¹ and SAM²—to segment GA lesions in FAF images, a task and imaging modality not included in the training corpus of these FMs.

Methods : Generalizability of the selected FMs was tested by adding a small, trainable, multilayer perceptron on top of the pretrained, frozen encoder of the large FMs. As a comparative baseline, a UNet³ model was trained from scratch. To test the efficiency of the FMs in small data sets, models were trained on subsamples of the full training set across 4 orders of magnitude, with 15 replications. The dataset consisted of 1879 ground-truth segmentations from 298 eyes from the Proxima B (NCT02399072) clinical trial. Images were divided 70/20/10 into training/validation/test sets, with all images from a given patient assigned to the same split. Hyperparameter tuning was performed on the validation set. Segmentation quality was evaluated using the Dice metric on the test set.

Results : Fig. 1 visualizes the test set Dice values for each model as the number of training samples varies. The FMs demonstrate strong performance relative to the UNet, particularly with limited dataset sizes. The performance of the DINOv2-based model is comparable, if not better than, the UNet model trained with 10 times more data at every scale tested.

Conclusions : The results support the hypothesis that the tested FMs exhibit strong generalizability to the task of GA lesion segmentation, despite not being pretrained on ophthalmology images. The benefit is particularly pronounced when the number of training samples is limited. This suggests FMs may be useful to accelerate the development of algorithms for which labeled data are scarce or expensive. Additional studies are required to confirm findings on other ophthalmology tasks and datasets.
References
Oquab M, et al. ArXiv 2023:2304.07193
Kirillov A, et al. IEEE/CVF 2023:4015-4026
Ronneberger O, et al. MICCAI 2015, Part III, LNCS 2015;9351:234-241

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Fig. 1 Dice score vs number of training samples by model