June 2022
Volume 63, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2022
Influence of Race on Training Data Quality for Artificial Intelligence (AI) Algorithms
Author Affiliations & Notes
  • Amitha Domalpally
    A-EYE Unit, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Rick Voland
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Robert Slater
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Ellie Corkery
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Pamela Vargo
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Becky Kuhtz
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • James Reimers
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Roomasa Channa
    A-EYE Unit, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Barbara A Blodi
    Wisconsin Reading Center, Dept of Ophthalmology and Visual Sciences, University of Wisconsin-Madison, Madison, Wisconsin, United States
  • Footnotes
    Commercial Relationships   Amitha Domalpally None; Rick Voland None; Robert Slater None; Ellie Corkery None; Pamela Vargo None; Becky Kuhtz None; James Reimers None; Roomasa Channa None; Barbara Blodi None
  • Footnotes
    Support  Research to Prevent Blindness, U01DK048489
Investigative Ophthalmology & Visual Science June 2022, Vol.63, 2999 – F0269. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Amitha Domalpally, Rick Voland, Robert Slater, Ellie Corkery, Pamela Vargo, Becky Kuhtz, James Reimers, Roomasa Channa, Barbara A Blodi; Influence of Race on Training Data Quality for Artificial Intelligence (AI) Algorithms. Invest. Ophthalmol. Vis. Sci. 2022;63(7):2999 – F0269.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : Diabetic retinopathy (DR) severity level is evaluated from stereoscopic 7-field color photographs by masked graders at the Wisconsin Reading Center and used as a reference standard for training and validation of AI algorithms. Training data quality influenced by race is understudied relative to standards established for White subjects. Retinal pigmentation is greater in individuals with darker skin tones, and the reduced contrast may affect the detection of DR features by graders. We explored the effect of race-related fundus pigmentation on the grader’s ability to document DR.

Methods : All images were acquired by a certified photographer at a single site using the same camera. and evaluated for DR by two experienced graders masked to all demographics. Following our standard protocol for quality assessment, graders determined a confidence score (CS) of high or low (borderline and ungradable) for each image. Graders were permitted to use digital enhancement tools for better visualization. The Red, Green, and Blue (RGB) channel values were also obtained for a representative image of each subject.

Results : Of 183 subjects, 37 (20.2%) were identified as White, 11 (6%) Black and 135 (73.8%) Other with Hispanic ethnicity. DR prevalence was 10.4% in the full cohort (366 eyes), mostly with mild-moderate non-proliferative DR. DR prevalence across the 3 racial/ethnic groups was 12.2%, 0% and 12.2% respectively. CS was high in 93%, 77% and 82%, borderline in 5%, 23% and 16% and ungradable in 1.4%, 0% and 1.5% respectively. There was significant difference between high and low CS across the three groups (p = 0.029) and also for the red channel of RGB values: 129 (95% CI 95,105), 85 (77,104), and 94(88,99) (p < 0.001).

Conclusions : Grader confidence for evaluating DR features was lower in pigmented retinas which could affect data accuracy from images obtained from darker skinned individuals. It is possible that DR was under called in Black population due to difficulty in detecting early features. This bias can transfer into AI models via training data. Photographer education in image capture techniques and grader training in racially diverse datasets is needed to obtain high quality, categorically delineated data.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Retinal images taken from a darker skinned subject with low contrast retinal features (top) versus a lighter skinned subject with high contrast retinal features (bottom) using the same camera.

Retinal images taken from a darker skinned subject with low contrast retinal features (top) versus a lighter skinned subject with high contrast retinal features (bottom) using the same camera.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×