System Performance Validation of a Multi-Model Machine Learning-Augmented Multi-Layer SD-OCT Segmentation System for Ellipsoid Zone Integrity and Outer Retinal Features with and without Certified Reader Review

Abstract

Purpose : Enhanced SD-OCT characterization of the outer retina has provided new opportunities for structure/function correlation and new clinical trial endpoints (e.g., ellipsoid zone (EZ) integrity). Accurate, efficient, and reliable segmentation is critical for output reproducibility. The purpose of this study was to validate the reproducibility and agreement of multiple segmentation measures using a multi-model machine learning (ML) enhanced segmentation system with and without certified reader (CR) corrections for outer retinal assessments.

Methods : In this IRB-approved retrospective analysis, SD-OCT scans from four disease groups were analyzed: diabetic macular edema (N=60), dry age-related macular degeneration (AMD, N=60), wet AMD (N=60), and normal (N=33). To evaluate system generalizability, scans were balanced between Zeiss Cirrus (N=108) and Heidelberg Spectrails (N=105). The multi-layer segmentation system consisted of ML models that were trained on a total of 486,155 annotated OCT B-scans across 1303 patients, using a combination of U-Net and Transformer architectures. All scans were initially segmented with the ML system with integrated logic and then evaluated through four methods: semi-automated with sequential dual expert read for use as ground truth, semi-automated with 2 independent CRs, and fully-automated. Intraclass correlation coefficients (ICCs) were calculated for both agreement and consistency.

Results : For evaluation of system performance, 213 patients were included in this analysis across the 4 disease groups. The CR approach and fully automated outputs exhibited outstanding agreement and consistency across numerous parameters and diseases, including highly complex anatomic segmentation, such as EZ integrity. ICCs for outer retinal measures were 0.98-0.99 for ONL-RPE volume, 0.96-0.98 for EZ integrity measures, and 0.94-0.97 for sub-RPE compartment metrics.

Conclusions : This study demonstrates that both CR assessment and fully-automated performance were robust and reliable for outer retinal layer parameters across multiple diseases and OCT systems. Performance was consistent even for historically challenging features to segment, such as EZ integrity, providing validation of this platform and approach for high reliability analysis.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.