Data Labeling for Artificial Intelligence (AI) Algorithms for Measurement of Geographic Atrophy

Abstract

Purpose : AI algorithms have shown impressive performance in segmenting geographic atrophy (GA) from fundus autofluorescence (FAF) images. Curating training datasets for AI algorithms requires pixel level annotation of GA on a large quantity of images, which is both expensive and time consuming. The purpose of this project is to understand the labeling requirements for training AI models in automated measurement of GA lesions.

Methods : Areas of GA were outlined on the entire FAF image by certified human graders providing an area measurement (mm²) and an annotated image. We trained two convolutional neural networks to identify and quantify GA areas from these FAF images using two types of labels: (1) a “weak” label consisting of numerical area measurements only and (2) a “strong” label, where human annotated images were used to train the model. Both AI models provided numerical areas of GA as an output. The model trained with the strongly labeled data also provided a segmented FAF image. For validation, the GA area measurements predicted by the AI model were compared to those measured by human graders from the same FAF images for both models. Dice coefficient was used to measure the degree of overlap between AI segmentation of GA and human annotation when the strongly labeled dataset was used.

Results : The training dataset included 416 FAF images and validation included 104 FAF images. Mean area of GA in the validation dataset for the weakly labeled images was 6.58 mm² (SD 5.84). The mean difference between grader measured areas and AI predicted areas using the weakly labeled training data was -0.34 mm² (95% CI [-4.86, 4.18], R = 0.95).

With the strongly labeled data, the mean GA area for the validation dataset was 6.87 mm² (SD 7.04). The mean difference in area between the grader measurements and AI predictions was 0.28 mm² (95% CI [-1.20, 1.76], R= 0.99, Dice coefficient= 0.73).

Intergrader agreement was comparable with mean difference between graders of 0.33 mm² (95% CI [-1.73, 2.39], R= 0.99).

Conclusions : The AI models provide a reliable method to both identify and quantify areas of GA in eyes with advanced AMD. Automated segmentation of GA using AI is an efficient and reproducible method of GA quantification. Training AI models for semantic segmentation of GA lesions may not require laborious annotation by skilled humans.

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.