Bi-Modal Deep Learning for Recognizing Multiple Retinal Diseases based on Color Fundus Photos and OCT Images

Abstract

Purpose : Artificial intelligence (AI)-based diagnosis of retinal diseases using a single imaging method of color fundus photography (FP) or optical coherence tomography (OCT) has been fully investigated.
Although bi-modal imaging examinations using both FP and OCT could provide more comprehensive retinal information than their single-modal counterparts, which might improve the accuracy of AI to detect retinal diseases, the feasibility and performance of bi-modal imaging-based AI diagnosis of retinal diseases have not been extensively investigated.
The purpose of this study is to evaluate the performance of a bi-modal imaging-based AI system in detecting multiple retinal diseases using both FP and OCT images.

Methods : The AI-based system was trained using 573 scans (from 400 patients) centered on macula, validated using 192 scans (from 140 patients), and tested using 209 scans (from another 146 patients). Each scan that captured with Topcon 3D OCT-1 Maestro (Topcon Corp., Japan) consists of one FP and one corresponding radial OCT scan, which further consists of 12 OCT B-scans. Using both images and additional summarized clinical data, at least two ophthalmologists confirmed the diagnosis of each scan, including diabetic retinopathy (DR), dry age-related macular degeneration (AMD), wet AMD, epiretinal membrane (ERM), pathologic myopia (PM), macular oedema (ME), and normal (Fig 1). Performance of this system was measured in terms of sensitivity and specificity, and average precision (AP) per diagnosis.

Results : The bi-modal image-based AI system had a mean AP for the detection of multiple retinal diseases of 0.84 (95% confidence interval [CI], 0.80 - 0.87), a sensitivity of 0.88 (95% CI, 0.84-0.93), and a specificity of 0.91 (95% CI, 0.88-0.95) (Fig 2). Compared with the bi-modal image-based AI system, the mean AP was 0.74 when the AI system was trained with only FP images (95% CI 0.69-0.78), and 0.81 when with only OCT images (95% CI, 0.77-0.84) (Fig 2).

Conclusions : AI-assisted diagnostic system based on bi-modal imaging method showed significantly better performance than that based on FP, and relatively better performance than that based on OCT. The additional information provided by bi-modal imaging method could be used for not only diagnosis, but also perhaps therapeutic decision, which needs further study.

This is a 2021 ARVO Annual Meeting abstract.

 

Fig 1. Schematic diagram of the present study.

 

Fig 2. AP per diagnosis.