Evaluation of real world efficacy of Artificial Intelligence assisted image co-localization

Abstract

Purpose : We wished to compare the ability to human observers (retinal doctors) to co-localize pathological structures from different platforms. We compared traditional co-localization techniques (side-by-side with human observers) to an AI technique which overlaid the images with AI algorithms and then alternated the images using the previously described automated alternation flicker method. We determined time to localize the lesions and accuracy of both methods.

Methods : Images taken with a Topcon color fundus camera showing diabetic retinopathy (4 eyes), AMD (5 eyes) and central vein occlusion (2 eyes) were analyzed for a masked clinician, lesions of interest were identified and marked with a circle. Infrared SLO images of the same fundus showing the corresponding area were then also selected. For the side-by-side (SBS) method, two retina specialists identified the corresponding lesion on the unmarked IR image and drew it digitally with the color image in the side. For the AI overlaid images, the same specialists identified the lesion on the unmarked IR and drew it digitally as it was being presented by automated alternating flicker along with the color image. A total of 53 lesions from 11 fundus image pairs were analyzed 2 times, one time for each method, in different days, for each specialist. Image analysis was done in random order. Time to find each lesion in each method was measured and recorded for the same person in all attempts.

Results : The time to find the lesions was faster in the overlay method (p<0.01). Out of 106 (combined total between both observers) gradings 19 lesions were missed in the SBS method and zero missed lesion in the overlay method. The average decentration (difference in lesion center normalized to lesion radius) was 22% and 27% for observers 1 and 2 in the overlay study. No significant difference between the graders (p=0.127). The average decentration was 59% and 56% for observers in the SBS method. No significant difference between the graders (p=0.265). However, there is a statistic difference when we compare SBS and overlay methods pooled for both graders (p=0.0000607).

Conclusions : The overlay method permits more rapid and accurate co-localization of lesions from different imaging platforms. It is easier and more accurate than the side-by-side method, which is closer to the “normal” approach to analyze different images in clinical and image reading settings.

This is a 2021 ARVO Annual Meeting abstract.