Efficacy of smartphone-based retinopathy of prematurity tele-screening with and without artificial intelligence

Abstract

Purpose : Assess the efficacy of two smartphone-based fundus imaging (SBFI) systems used by technicians for retinopathy of prematurity (ROP) screening both by human graders, and also by artificial intelligence-based systems.

Methods : 156 babies were screened as part of an existing ROP tele-ophthalmology program in India. This prospective study compared Retcam images taken in parallel with either the Keeler MIO or MII Retcam systems using two masked readers evaluated zone, stage, plus, and vascular severity scores (VSS, from 1-9) in all images. Then, the MII and MIO images were combined, stratified by patient into training, validation, and test datasets (70%/10%/20%), and used to train a ResNet18 deep learning architecture for binary classification of normal versus pre-plus or plus disease, which were used as patient-level predictions of referral warranted(RW)- and treatment requiring(TR)-ROP. Sensitivity and specificity of detection of RW-ROP, and TR-ROP by both human graders and an AI system, and area under the receiver operating characteristic curve (AUROC) of grader-assigned VSS were calculated compared to diagnosis by Retcam images.

Results : Both SBFI devices were effective at detecting TR-ROP with a sensitivity [95% confidence interval] of 100% [54.1%,100%] for MII Retcam, and 100% [29.2%,100%] for MIO, with specificity of 85.4% [75.0%,91.5%] for MII Retcam and 83.0% [70.2%,91.9%] for MIO. AUROC with grader-assigned VSS only was 0.946 [0.907,0.985] and 0.961 [0.931,0.991] for RW- and TR-ROP, respectively. For the AI system, the sensitivity of detecting TR-ROP sensitivity was 100% [29.2%, 100.0%] and specificity was 58.6% [38.9%, 76.5%], and RW-ROP sensitivity was 80.0% [28.4%, 99.5%] and specificity was 59.3% [38.8%, 77.6%].

Conclusions : Technician-led ROP screening using SBFI devices was 100% sensitive using both human graders and an AI system for treatment-requiring ROP. Using SBFI systems with AI may be a cost- effective method to improve the global capacity for ROP screening.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.