Abstract
Purpose :
Artificial intelligence (AI) has the potential to augment the clinical utility of ophthalmic imaging. Input data used in training these AI algorithms are often required to meet strict inclusion criteria to maximize accuracy on high quality images. However, in clinical settings, an AI trained on pristine images without ocular co-morbidities may have limited utility. This study aims to evaluate the accuracy of an artificial intelligence algorithm applied to color fundus photos (CFPs) with simulated cataracts in detecting diabetic retinopathy (DR).
Methods :
A database of 3662 CFPs (from the Asia Pacific Tele-Ophthalmology Society (APTOS) 2019 data) was used, with 80% of images used for training and 20% for testing. Using transfer learning, a convolutional neural network (Inception-ResNet-v2) was trained to classify the training images as either DR or non-DR. The CNN was then applied to classify the testing images four times, once each with mild simulated cataract, moderate simulated cataract, severe simulated cataract, and no simulated cataract. Cataracts were simulated by applying varying degrees of gaussian blur corresponding with distorting an image to appear as it would to an eye that is 20/40 (mild), 20/100 (moderate) and 20/200 (severe). Accuracy was compared by confusion matrix, including sensitivity (Sn) and specificity (Sp), and receiver operator curves (ROC).
Results :
The CNN was able to classify the dataset without any simulated cataract with an accuracy of 97.0%, Sn 95.7%, Sp 98.3%. On the mild cataract dataset, the CNN had an accuracy of 93.1%, Sn 91.8%, Sp 94.3%. For moderate cataract, accuracy was 62.8%, Sn 31.4%, Sp 95.2%. For severe cataract, accuracy was 53.5%, Sn 11.8%, Sp 96.5%.
Conclusions :
Artificial intelligence algorithms are often trained on pristine datasets, where variability is controlled to allow for optimum performance. However, real world data often has significant noise. This study shows that the accuracy of an AI algorithm trained to detect DR is significantly diminished when a simulated cataract is superimposed on the image. To prepare AI for clinical use, cataract and other real-world clinical challenges causing poor image quality must be accounted for.
This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.