Abstract
Purpose :
Plus disease denotes a severe vascular abnormality in cases of ROP, which may portend to requiring treatment to prevent blindness but its assessment is subjective. Bringing in Artificial Intelligence (AI) automation can not only improve diagnostic consistencies but can also help scale up ROP screening services to remote and underserved regions of the world. We developed and assessed the performance of an AI algorithm to automatically detect the presence of Plus disease in retinal images of premature babies.
Methods :
We trained a deep learning (DL) algorithm with 42,641 disc and macula centered images from a tele-ROP screening program in India. The model was trained to indicate the presence of Plus in the images. Since the dataset contained 0.5% of Plus images, it was trained using metric learning, a technique improving DL performance under datasets with strong class imbalance. Pre-Plus images were not presented to the AI during training. The algorithm was tested on two distinct datasets. Test set A consists of 10,976 images, with 169 pre-plus, 70 plus images and rest no plus images. Test set B consists of 108 images, with 39 pre-Plus, 45 Plus images and rest no plus images. The reference standard for training and test sets was the interpretation of ROP specialists.
Results :
The sensitivity on test set A was 95.7% (95% CI: 88.0% to 99.1%), with 3 Plus images being misclassified. Specificity with pre-plus as non-referable was 99.6% (95% CI: 99.4% to 99.7%), and 99.9% (95% CI: 99.8% to 100%) if pre-plus images were excluded. The sensitivity on test set B was 97.8% (95% CI: 88.2% to 99.9%) with one Plus image being misclassified. The specificity was 68.3% (95% CI: 55.3% to 79.4%) with pre-plus, and 100% (95% CI: 85.8% to 100%) without.
Conclusions :
The DL tool for ROP Plus detection has excellent sensitivity in picking up Plus disease and thus can potentially be used as a triaging tool for infants with ROP requiring immediate treatment. A prospective clinical validation in a real-world setting is under consideration.
This is a 2021 ARVO Annual Meeting abstract.