SLIViT: a general AI framework for accurate clinical-feature diagnosis from limited 3D medical-imaging data

Abstract

Purpose : Medical imaging analysis is a critical component of clinical care. For instance, analyzing optical coherence tomography (OCT) images allows ophthalmologists to diagnose and follow up on ocular diseases, such as age-related macular degeneration (AMD), and tailor appropriate and personalized interventions to delay the progression of retinal atrophy and irreversible vision loss. AI models could potentially automate these analyses and improve healthcare by reducing costs and treatment burden. However, procuring thousands of annotated medical-imaging samples, specifically volumetric modalities (e.g., 3D OCT), to train standard AI models is cost- and expert-time-prohibitive, impeding their full optimization.

Methods : We developed the SLice Integration by Vision Transformer (SLIViT), a uniform 3D-based AI framework that leverages an exclusive combination of deep-learning modules and prior 'knowledge’ obtained from 2D OCT scans. This unique combination enables it to overcome the annotation bottleneck and excel in various 3D-OCT-related learning tasks. Interestingly, SLIViT’s prior 'retinal knowledge’ can be translated to other 3D-medical-imaging modalities, such as echocardiogram videos and 3D magnetic resonance imaging (MRI) scans.

Results : We compared the performance of SLIViT in four binary-classification tasks of AMD biomarkers, including large drusen volume, intraretinal hyperreflective foci, subretinal drusen deposits, and hyporeflective drusen cores, using less than 700 annotated training samples. We also compared its performance in regression tasks of cardiac function diagnosis in echocardiogram videos and severity assessment of a hepatic-disease from 3D MRI scans. SLIViT consistently and significantly outperformed domain-specific state-of-the-art AI models, typically improving the performance score (AUC or R²) by 0.1-0.4 (p-value<0.001). SLIViT’s accuracy on the binary-classification tasks was also compared to retina specialists’ assessment and found to be on par, 5,000x faster.

Conclusions : SLIViT is adept at 3D-medical-imaging learning tasks, in which the number of annotated training samples is typically very limited. It significantly transcends strong AI models while achieving on-par performance to clinical specialists’ manual annotation. Thus it can be used to save resources, reduce the burden on clinicians, expedite ongoing research, and consequently, improve patient healthcare.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.