Abstract
Purpose :
Artificial intelligence (AI) has substantial promise in medicine, but real-world healthcare implementation examples are lacking. We describe the development, implementation and evaluation of an AI-system in a nation-wide screening program for diabetic retinopathy (DR), a major cause of blindness.
Methods :
Our multi-year program (2015-2019) entailed 3 phases: 1) the development and validation of an AI-system; 2) the formation of inter-ministry national taskforce to integrate the AI-system into the DR screening program and 3) the conduct of a prospective study prior to final deployment. In the prospective study, we compared AI-system using two models - Full-AI and Semi-AI (first stage AI and second stage human graders) with conventional screening (two-stage human graders) on consecutive patients with diabetes from 6 primary care clinics over 18 weeks. The performance of AI-system was calculated using area under receivers’ operating curve (AUC), sensitivity, and specificity in detecting referable DR; vision-threatening DR (VTDR) and diabetic macular edema [DME]). Cost models were based on real costs.
Results :
A total of 12,931 patients (25,830 eyes, 49,743 images) were evaluated. The Full-AI model had AUC, sensitivity and specificity of 0.938, 94.5%, and 84.1%, in detecting referable DR. The sensitivity in detection of VTDR and DME were 99.3% and 97.2%, respectively; the Semi-AI model had improved AUC and specificity of 0.973 and 100%, while retaining same sensitivity in detection of referable DR, VTDR and DME. The Semi-AI model reduced human workload by 76.9% and cost by 43% compared to conventional screening.
Conclusions :
The study provides an outline for the design and implementation of a clinically effective and cost saving AI algorithm addressing a major clinical problem. This process may be a roadmap for other countries, individualized to specific patient populations, clinical needs and healthcare systems
This is a 2020 ARVO Annual Meeting abstract.