June 2022
Volume 63, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2022
Detecting patients with ocular comorbidities when screening for diabetic retinopathy (DR): an out-of-distribution (OOD) perspective
Author Affiliations & Notes
  • Jelena Novosel
    F. Hoffmann-La Roche Ltd., Basel, Switzerland
  • Jian Dai
    Genentech Inc, South San Francisco, California, United States
  • Daniela Ferrara
    Genentech Inc, South San Francisco, California, United States
  • Fethallah Benmansour
    F. Hoffmann-La Roche Ltd., Basel, Switzerland
  • Footnotes
    Commercial Relationships   Jelena Novosel F. Hoffmann-La Roche Ltd., Code E (Employment); Jian Dai Genentech, Inc., Code E (Employment); Daniela Ferrara Genentech, Inc., Code E (Employment); Fethallah Benmansour F. Hoffmann-La Roche Ltd., Code E (Employment)
  • Footnotes
    Support  Yes, Genentech, Inc., South San Francisco, CA, provided support for the study and participated in the study design; conducting the study; and data collection, management, and interpretation
Investigative Ophthalmology & Visual Science June 2022, Vol.63, 2987 – F0257. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jelena Novosel, Jian Dai, Daniela Ferrara, Fethallah Benmansour; Detecting patients with ocular comorbidities when screening for diabetic retinopathy (DR): an out-of-distribution (OOD) perspective. Invest. Ophthalmol. Vis. Sci. 2022;63(7):2987 – F0257.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : When screening for DR in a patient population with diabetes, occurrence of ocular comorbidities, such as glaucoma, neuropathy, or age-related macular degeneration, remains possible. We frame the detection of patients with such ocular comorbidities as an OOD detection problem. Here we evaluated 3 state-of-the-art OOD methods for DR screening by using deep learning (DL) models trained from 7-field color fundus photos (7F-CFP).

Methods : Data, including images, from eyes of 37,358 patients with diabetes were analyzed (Inoveon Corporation). Early Treatment Diabetic Retinopathy Study Diabetic Retinopathy Severity Scale and the presence of referable ocular comorbidities (OOD samples) were assessed from 7F-CFP by professional graders. The prevalence of OOD patients was 1%. The data were split into train:tune:test sets (80:10:10).
A DL ResNet-50 model with transfer learning was trained at the image level using all 7F-CFP for either being more than mild DR (mtmDR) or not. Then, we assessed the ability of ResNet-50 to detect OOD samples by 2 methods: (1) uncertainty estimation (UE) using Monte Carlo Dropout (Gal et al, NIPS 2016), and (2) confidence score (CS) using class-conditional Gaussian distributed feature maps and Mahalanobis distance (Lee et al, NIPS 2018). We also trained an OOD binary classification (BC) model to classify patients as having referable ocular comorbidities or not. Performance was assessed with area under the receiver operating characteristic (AUROC) curve, specificity, sensitivity, and precision.

Results : The OOD BC evaluation results are listed in Table 1 and the method shows high sensitivity and specificity. The UE and CS results are illustrated in Figure 1 and both scores show overlapping values for patients with and without referable ocular comorbidities, making the 2 groups inseparable.

Conclusions : UE and CS render limited ability to identify patients with referable ocular comorbidities when the primary DL model is trained for detecting patients with mtmDR. In contrast, the OOD BC model performed better in detecting OOD samples. This indicates that real-world deployment of DL models for mtmDR detection might require an additional model, such as our OOD BC model, that detects patients with referable ocular comorbidities.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Table 1. mtmDR and OOD BC Results

Table 1. mtmDR and OOD BC Results

 

Figure 1. UE (Top) and CS (Bottom)

Figure 1. UE (Top) and CS (Bottom)

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×