Comparison of automated and expert human grading of diabetic retinopathy using smartphone-based retinal photography

Abstract

Purpose : Diabetic retinopathy (DR) is the leading cause of vision loss and blindness in working-age adults. Retinal photography is a well-validated screening tool for DR but is expensive with limited portability. Smartphone-based photography addresses these limitations. We combine smartphone-based retinal photography with automated image analysis to detect referral-warranted DR (RWDR).

Methods : A mydriatic smartphone-based retinal camera (Cellscope Retina) was used to image diabetic patients at the University of Michigan Kellogg Eye Center Retina Clinic. Images were analyzed with cloud-based EyeApp software to generate screening recommendations (refer/no refer) based on presence of moderate non-proliferative DR or higher and/or markers for clinically significant macular edema (CSME). Images were independently evaluated by two masked readers for severity of DR and/or presence of CSME, and similarly categorized as refer/no refer. Results from EyeApp and masked readers were compared against clinical diagnosis made with slit-lamp biomicroscopy to determine sensitivity and specificity, at both eye-level and patient-level (RWDR defined at the patient-level if present in at least 1 eye of a subject).

Results : 72 patients (144 eyes) were imaged. RWDR was present in 101 eyes (77.1%) and absent in 30 eyes (22.9%) by gold standard clinical diagnosis. For detecting RWDR at the eye-level, EyeApp had a sensitivity of 77.4% and specificity of 70.4%; grader 1 had a sensitivity of 93.9% and specificity of 51.9%; grader 2 had a sensitivity of 88.8% and specificity of 63.0%. At the patient-level, RWDR was present in 55 subjects (76.4%) and absent in 12 subjects (16.7%). For detecting RWDR at patient-level, EyeApp had a sensitivity of 93.9% and specificity of 75.0%; grader 1 had a sensitivity of 98.1% and specificity of 41.7%; and grader 2 had a sensitivity of 96.0% and specificity of 41.7% (Figure 1).

Conclusions : CellScope Retina combined with EyeApp software achieves reasonable sensitivity and specificity in detection of RWDR at the person-level, with lower sensitivity but higher specificity than human graders. As the patient population studied had high prevalence of DR, additional study of a more typical screening population of diabetics in the community is needed.

This is an abstract that was submitted for the 2017 ARVO Annual Meeting, held in Baltimore, MD, May 7-11, 2017.

 

Sensitivity & specificity for referral-warranted diabetic retinopathy from CellScope Retina images using EyeApp and expert human grading