Abstract
Purpose :
To analyze the inter- and intra-reader agreement for subretinal drusenoid deposits (SDD) quantification on optical coherence tomography (OCT) imaging in intermediate age-related macular degeneration (iAMD).
Methods :
10 eyes of 10 patients with iAMD were imaged on a Spectral-Domain OCT (Spectralis, Heidelberg Engineering, Germany) using a 20x20° scan pattern with 97 B-Scans per OCT volume. 10 B-scans from 10 OCT volumes were selected predominantly from the superior retina considering the preferred localization of SDD (B-Scans 2, 5, 10, 15, 20, 30, 49, 70, 80, 90). Therefore, a total of 100 B-scans were included in the reading data set. Three experienced readers were asked to delineate stage 2 and 3 SDD lesions on the selected B-scans twice, with a break of two weeks between the annotation runs. The inter- and intra-reader agreement was calculated using Intersection over Unit (IoU), Dice Coefficient (DC) and Intraclass Correlation Coefficient (ICC).
Results :
A total of 4,828 SDD were annotated by the three readers. Regarding the number of SDD, we found excellent agreement between the readers in each reading session. The ICC was 0.953 (95%CI 0.934 - 0.967) in the first run and 0.948 (95%CI 0.928 - 0.964) in the second. We also found an excellent agreement between both annotation runs (ICC 0.976, 95%CI 0.968 - 0.983). There was moderate inter-reader agreement determined by IoU (mean IoU = 41.94 - 50.49%) and substantial inter-reader agreement using DC (mean DC = 58.48 - 66.75%). For intra-reader agreement, mean IoU values ranged from 48.98 to 57.80% and the mean DC values ranged from 65.43 to 73.75%, which we considered as moderate and substantial, respectively.
Conclusions :
In general, we found excellent agreement for the number of stage 2 and stage 3 SDD lesions on OCT B-scans, demonstrating the reliability of establishing a ground truth for validating machine learning frameworks. The moderate to substantial inter-reader agreement values underscore the difficulties of quantifying SDD pixel-wise. The accurate identification of SDD is as challenging as it is important, since SDD has been considered a risk factor not only for AMD progression but also for systemic diseases. Therefore, a validated machine learning framework would be a step forward on consistent and rapid SDD image analysis.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.