Evaluating the practicability of natural-domain and domain specific foundation models for ophthalmic image classification.

Abstract

Purpose : Deploying deep learning models for medical imaging in clinical settings is often challenging due to their limited generalizability. In this study, we aim to evaluate the fine-tuning of two pre-trained foundation models (natural domain: DINOv2 and domain specific: RETFound) as potential solutions to this problem. Our research focuses on investigating the generalizability, quantity of images needed, and computational requirements of these adapted models for the downstream tasks of ophthalmologic disease classification from color fundus photographs (CFPs).

Methods : We utilized 5922 CFPs from publicly available datasets with corresponding DR-stage labels, using a training, validation and test split of 70:15:15. For this study, we evaluated the four adapted models from DINOv2 and RETFound showing the most promising performance in our previous research. A detailed tabulation of these models can be found in Table 1.
In order to evaluate image quantity requirements for downstream model adaptation, a few shot study was performed. Computational requirements for each model were reported. Cross-evaluation for generalizability estimation was performed by training models on DR datasets distinct from the testing datasets and reported as average QKappa in Table 1.

Results : Amongst all models investigated, our CFP-BE-DINOv2 model showed the best transfer across datasets (average QKappa of 0.605). Adapting DINOv2 resulted in smaller models compared to RETFound. All models evaluated can be trained on consumer-grade GPUs. The computational requirements for training each model on our largest dataset (APTOS, 3992 images) is depicted in Table 1. As depicted in Figure 1, all models reached QKappa thresholds of 0.7 when trained with only 32 samples per class on APTOS. Among all models, the frozen DINOv2 model yielded the best performance when trained on 2-64 sample images, outperforming the unfrozen RETFound model (QKappa of 0.814 and 0.751 respectively, with 32 samples per class).

Conclusions : Our CFP-BE-DINOv2 model resulted in the best generalizability across distinct datasets. Our few shot study suggests that training the linear layer of DINOv2 (frozen DINOv2) may suffice, facilitating model development with minimal computational requirements.

This abstract was presented at the 2024 ARVO Imaging in the Eye Conference, held in Seattle, WA, May 4, 2024.

 

Figure 1: Few shot study results.

 

Table 1: Depicting model information, computational requirements and cross-evaluation performance.