Abstract
Purpose :
Trustworthy Artificial Intelligence systems for retinal OCT classification must simultaneously maintain their performance for different acquisition settings and identify cases outside their expected repertoire (outliers). In this study we assess if a state-of-the-art retinal foundation model is better suited than a common model for robust Age-related macular degeneration (AMD) screening.
Methods :
The public RETFound foundation model, pretrained with OCTs, was fine-tuned for AMD classification using Heidelberg Spectralis (HS) OCT central B-scans from a multi-center private dataset (1653 eyes) with target classes: healthy, intermediate AMD (iAMD), neovascular AMD (nAMD) and geographic atrophy (GA). The model was evaluated in: 1) a private dataset (1590 eyes), with the target classes and outliers: diabetic macular edema (DME), retinal vein occlusion (RVO), Stargardt disease and central serous chorioretinopathy (CSC), and in 2) OCTID, a public external dataset of 572 Cirrus images from classes: healthy, central serous retinopathy (CSR), diabetic retinopathy (DR) and macular hole (MH). The tested outlier scores were the Cosine and the Mahalanobis distances on the features of the penultimate layer. A ResNet-18 pretrained on ImageNet was fine-tuned for comparison. Each training was repeated 10 times to ensure result reliability.
Results :
The balanced accuracy for AMD classification on the private dataset was 0.92±0.01 for RETFound and 0.91±0.01 for ResNet. On OCTID, RETFound distinguished healthy from AMD cases with 0.99±0.00 balanced accuracy vs 0.66±0.16 from ResNet. For outlier detection (Fig. 1), on the private dataset, the AUCs for RETFound were 0.82±0.07/0.74±0.06 and 0.57±0.05/0.82±0.01 for ResNet using Mahalanobis/Cosine scores, respectively. For OCTID these AUCs were 0.87±0.09/0.55±0.07 for RETFound, and only 0.30±0.10/0.66±0.08 for ResNet. RETFound with Mahalanobis was the best system when considering both AMD staging and outlier detection. The feature space (Fig. 2) also shows that RETFound is better at separating outliers from the target classes.
Conclusions :
The RETFound model shows good generalization capability, successfully classifying OCTs and detecting unrelated pathologies in the context of AMD screening and staging. Retinal foundation models constitute promising and reliable approaches for trustworthy assisted diagnosis.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.