Feasibility and clinical efficacy of an OCT B-scan of interest tool

Abstract

Purpose : An OCT B-scan of interest (BSOI) tool was created using deep learning to highlight B-scans within an OCT volume scan which contain pathology (B-scan of interest) and B-scans of poor image quality (IQ) to improve workflow efficiency. While intrinsic test metrics, e.g. sensitivity and specificity, are valuable, predictive values (PVs) and likelihood ratios (LRs) may provide additional clinical utility. In this retrospective study, the diagnostic efficacy of the BSOI tool was evaluated.

Methods : The BSOI tool (ZEISS CIRRUS PathFinder) was previously trained using 76,544 B-scans from 598 macular cubes (512x128) acquired from 598 subjects with SD-OCT (CIRRUS™ HD-OCT 4000 and 5000, ZEISS, Dublin, CA) [1]. The BSOI tool first runs an IQ algorithm which flags B-scans with poor IQ, then flags good IQ scans “of interest” if one or more retinal pathology labels are found. Two hold-out test sets were used for validation: i) TestSet1: 25,600 B-scans (200 cubes, 200 subjects) acquired from CIRRUS 5000, labeled by at least 1 retina expert with specialist adjudication; ii) TestSet2: 28,800 B-scans (225 cubes, 225 subjects) acquired from CIRRUS^® OCT 6000 (ZEISS, Dublin, CA), labeled by at least 2 optometrists.

Sensitivity, specificity, accuracy, positive PV (PPV), negative PV (NPV), positive LR (LR+), and negative LR (LR-) were calculated from good IQ scans combined from both test sets (TestSet3). 95% confidence intervals were calculated via clustered bootstrapping.

Results : 53.9% of the B-scans in TestSet3 were of interest. Sensitivity, specificity, accuracy, study PPV, and study NPV for TestSet3 were 87.8%, 92.9%, 90.2%, 93.5%, and 86.7%, respectively (see Table 1). LR+ and LR- of 12.3 and 7.6 also suggest the tool provides a strong increase in the likelihood (or not) of pathology. Fig. 1 estimates PPV and NPV as functions of LRs and BSOI pre-test probabilities.

Conclusions : The study PPV and NPV indicate good utility in ruling in/out pathologies of interest for a given B-scan when the fraction of BSOI is high, which may be reflective of a retinal referral setting. Furthermore, in a typical healthy cohort where the BSOI prevalence is low (< ~5%) [2], Fig. 1 suggests NPV > 99% with a reasonable PPV up to ~39%. Overall, the PathFinder tool may provide additional guidance for a specific patient by helping clinicians take additional evidence into account.

[1] Talcott et al. IOC 2024; Article in Press.
[2] Yu et al. IOVS 2020; 61(9): PB0085.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.