Statistical Validation Of An Automatic Algorithm For Diabetic Retinopathy Screening

Abstract

Purpose: : To present the results of a population-based statistical validation of an automatic computer system that screens retinal photographs for the presence of lesions related to diabetic retinopathy (DR).

Methods: : Digital fundus photographs from N=924 subjects were collected in three different diabetic retinopathy screening centers: Project HOPE in Albuquerque, NM; Communicare Clinics, and the Retina Institute of South Texas, in San Antonio, TX. Macula-centered, non-mydriatic images of both eyes were captured with a Canon CR1 Mark II camera. Ground truth was provided by two optometrists, with discrepancies being adjudicated by an ophthalmologist. The graders inspected the images for the presence of pathologies including: microaneurysms, hemorrhages, exudates, neovascularization on the optic disc (NVD) and elsewhere (NVE).The statistical study was a single arm trial to assess the diagnostic accuracy of the algorithm to detect DR relative to normal controls. The sample size was selected to achieve 90% statistical power with 5% significance level.A computer algorithm based in amplitude modulation-frequency modulation (AM-FM) and partial least squares (PLS) was used to produce binary results on the presence or absence of pathology. The algorithm was trained using images from N=450 subjects. The resulting system was then independently evaluated on the remaining N=474 subjects. Of those, N=312 subjects were normal controls, while the remaining N=162 presented signs of diabetic retinopathy.

Results: : The system achieved a sensitivity of 0.90 and specificity of 0.63. Based on these values we rejected the null hypotheses of 0.80 sensitivity and 0.50 specificity. When analyzing the N=30 cases of sight threatening DR (NVD, NVE, or probable CSME), accuracy was 97%. Analysis of inter-grader variability between the human graders resulted in a Cohen’s Kappa value of 0.58 (moderate agreement).

Conclusions: : A computer-aided detection algorithm based on AM-FM and PLS was trained to detect the presence of diabetic retinopathy in digital fundus images. We validated this algorithm in an independent set of images obtaining sensitivity and specificity values of 0.90 and 0.63, respectively. Thus, our algorithm of DR screening has been demonstrated to be a clinically significant test for detection of diabetic retinopathy.

Keywords: diabetic retinopathy • image processing • imaging/image analysis: clinical