Sociodemographic differences in performance and perceptions of Artificial Intelligence automated retinal image analysis systems (ARIAS) for the detection of diabetic retinopathy within the English NHS Diabetic Eye Screening Programme (DESP)

Abstract

Purpose : Diabetes mellitus is an increasing public health problem. The main microvascular complication is diabetic retinopathy (DR). Early detection of DR by retinal image grading to avoid sight loss is costly and labour intensive. ARIAS could provide a clinical and cost-effective solution. We describe a transferable, independent evaluation platform of ARIAS in real-life settings, which allows on-going surveillance of equity and bias in performance to be examined.

Methods : We prospectively curated screening episodes from the North East London DESP from October 1^st 2021 to September 30^th 2022. ARIAS that met inclusion criteria were invited (n=22) to participate in the evaluation. We co-developed surveys with health care professionals and people living with diabetes to assess their perceptions, concerns and potential impact if ARIAS were to be deployed.

Results : The curated data set of 100K screening episodes (including ~600K images) encompassed a wide spectrum of DR and age range, with 40% South Asians, 30% white and 20% of black ethnic origin. We developed a trusted research environment that can operate ARIAS simultaneously to quantify test performance overall, by ethnic and age groups and examine interactions between camera systems and sociodemographic factors with ARIAS test performance. The co-developed surveys around perceptions included general questions about AI for eye screening, efficiency, data regulation, trust, on-going responsibility/security, impact on workforce and screening experience.

Conclusions : AI approaches for DR detection are continually evolving, camera systems change, and there is a need to build in safeguarding mechanisms (impartial from any commercial influence) which ensure these systems continue to meet pre-defined standards for all population subgroups, while remaining flexible enough to identify potential improvements in performance. Through independent and transparent evaluation of commercial systems, we provide a platform that is sustainable in the longer term and equitable so that all population subgroups may benefit. Patient and practitioner involvement for AI deployment is crucial to the successful implementation and on-going use of these systems within healthcare settings.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.