EyeSAM: Unveiling the Potential of Segment Anything Model in Ophthalmic Image Segmentation

Abstract

Purpose : This study evaluates the recently published foundational Segment Anything Model (SAM) and its variants for precise segmentation in Optical Coherence Tomography (OCT) and Fundus Autofluorescence (FAF) images. In addition, we introduce EyeSAM, an unpublished CLI tool facilitating SAM's execution and fine-tuning for custom datasets across various image modalities.

Methods : Retinal and corneal OCT, along with FAF data, were sourced from Moorfields Eye Hospital patients. Expert graders meticulously annotated key features like intraretinal fluid, whole retina, corneal thickness, optic disc, and hyper/hypofluorescence. SAM and MedSAM served as untuned (or vanilla) baseline models, subsequently fine-tuned (retrained) on for specific ophthalmic datasets.
Data division included random patient ID-based splits into training, validation, and test sets. SAM's inferences necessitated prompts, for which we employed box coordinates extracted from graded annotations, as text data alone proved inadequate for the segmentation challenge.
Initial assessments on the test split involved Vanilla SAM and MedSAM. Following this, fine-tuning of these algorithms transpired using the train and validation data splits.

Results : Table 1 illustrates our main results. Fine-tuned SAM excelled in Corneal AS-OCT feature segmentation (0.96 vs. 0.89 for Vanilla SAM). Vanilla MedSAM performed poorly (0.29), with no improvement post fine-tuning. Hypo autofluorescence and hyper-autofluorescence ring features improved with fine-tuning, and similar outcomes were observed for the optic disc in both Vanilla and fine-tuned models. SD-OCT exhibited comparable performance between vanilla and fine-tuned models.

Conclusions : Our findings highlight Vanilla SAM's effectiveness in segmenting specific ophthalmic features. EyeSAM emerges as a valuable tool, facilitating SAM's application and fine-tuning for custom datasets, enhancing performance in AS-OCT and certain FAF features. However, SD-OCT features showed no improvement post fine-tuning. Ongoing efforts aim to refine the fine-tuning process for Vanilla SAM and Vanilla MedSAM in ophthalmic datasets, fostering advancements in image analysis for eye health.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Table 1. Comparative analysis of Vanilla SAM, Vanilla MedSAM, and their fine-tuned versions. All trained for at least 20 epochs. In bold are the cases where fine-tuning improved performance. IRF=intraretinal fluid, AF=autofluorescence.