Understanding Equity in Vision-and-Language Learning for Glaucoma Diagnosis with Deep Learning

Abstract

Purpose : This study investigated equity issues in vision-language artificial intelligence (AI) models for glaucoma detection.

Methods : We randomly selected 10,000 pairs of scanning laser ophthalmoscopy (SLO) fundus and visual field (VF) with accompanying clinical notes within the timeframe of 30 days from 10,000 patients tested between 2015 and 2022 from Massachusetts Eye and Ear (average age 60.9 ± 16.2 years). We chose the last test for each eye and included one eye per patient randomly. The patients were categorized into non-glaucoma (VF mean deviation ≥ -1 dB and normal VF glaucoma hemifield test and pattern standard deviation (PSD) results) and glaucoma categories (VF mean deviation < -3 dB and abnormal VF glaucoma hemifield test and PSD results). 7,000, 1,000, and 2,000 samples were used for training, validation, and testing, respectively. The patients were from three racial groups: Asian (8%), Black (15%), and White (77%). We developed fair contrastive language-image pre-training (FairCLIP, Figure 1) to enhance equity in glaucoma detection. T-test with bootstrapping sampling was used to compare the performance of different models measured by the area under the receiver operating characteristic curve (AUC).

Results : Within the setting of learning with both notes and diagnostic labels, FairCLIP consistently outperformed CLIP in terms of AUC across all racial groups (Figure 2), with Asian (0.85 vs. 0.82 with p < 0.001), Black (0.79 vs. 0.76 with p < 0.001), and White (0.82 vs. 0.81 with p < 0.001) populations, indicating improved equity across demographic groups. Similarly, FairCLIP enhanced equity with higher AUC values compared to CLIP for both Non-Hispanic (0.82 vs. 0.80 with p < 0.001) and Hispanic (0.79 vs. 0.75 with p < 0.001) ethnicities. Additionally, FairCLIP achieved slightly higher AUC values for females compared to CLIP (0.66 vs. 0.64 with p < 0.001) and consistently outperformed CLIP across various language preferences, emphasizing its superior fairness in diagnosing glaucoma across diverse language groups. Within the setting of learning with only clinical notes, FairCLIP also improved performance from 0.71 to 0.73 (p < 0.001) for the Black group.

Conclusions : Our experiments revealed significant demographic disparities in glaucoma detection using SLO fundus photos and clinical notes. FairCLIP, our proposed approach, significantly enhanced both accuracy and equity.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Figure 1

 

Figure 2