A New Deep Learning Technique Termed Fair Identity Scaling to Improve Model Equity for Diabetic Retinopathy Screening

Abstract

Purpose : Proposing fair identity scaling (FIS) technique to improve fairness in artificial intelligence for diabetic retinopathy (DR) screening.

Methods : We developed fair identity scaling to integrate learnable group weights and individual loss information in training to improve model equity. VGG-derived 3D models were chosen as the backbone model to illustrate the impact of fairness identity normalization on model performance equity. We compared fair scaling with a widely used technique for improving model equity called adversarial (Adv) training. We used 10,000 3D OCT scans from 10,000 patients from Massachusetts Eye and Ear tested between 2015 and 2023. 6,000, 1,000, and 3,000 subjects were used for training, validation, and testing, respectively. 9.1% of patients were identified as having vision-threatening DR (moderate and severe non-proliferative DR [NPDR] plus PDR). The overall and group-wise areas under the receiver operating characteristic curve (AUC) were used for model assessment. A new fairness metric named performance-scaled disparity (PSD) is introduced to assess the model equity. Mean and max PSDs are computed as the standard deviation of group performance and the absolute maximum difference in group performance divided by overall performance, respectively.

Results : The average age is 64.1 ± 17.0 years. 57.1% are female. 78.6%, 13.7% and 7.7% of the patients are White, Black, and Asian, respectively. 3.8% are Hispanic and 96.2% are non-Hispanic. Concerning gender, the overall AUC on OCT-B scans for DR patients with fair identity scaling, increased from 0.92 to 0.94. Specifically for males, the AUC increased from 0.91 to 0.93. Regarding race, the overall AUC on OCT-B scans for DR patients with FIS increased from 0.92 to 0.93, with the AUC for Black people increasing from 0.85 to 0.89. Regarding Hispanic ethnicity, the overall AUC for DR patients with fair identity scaling increased from 0.92 to 0.93. Significantly, the mean PSD and max PSD for race groups decreased from 5.09 to 3.14 and 11.72 to 7.46, respectively. All statistical differences mentioned were significant with p < 0.001. Our fair scaling model consistently outperformed the adversarial training approach.

Conclusions : The fair identity scaling method mitigates group disparities while preserving overall performance.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

 

Methods comparison using OCT scans for DR detection across different identity groups.