Investigative Ophthalmology & Visual Science Cover Image for Volume 65, Issue 7
June 2024
Volume 65, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2024
Evaluating GPT-4 Diagnostic Proficiency in Glaucoma Detection through Fundus Image Analysis
JALIL JALILI; Anuwat Jiravarnsirikul; Christopher Bowd; Akram Belghith; Michael Henry Goldbaum; Sally L. Baxter; Linda M Zangwill; Mark Christopher
Author Affiliations & Notes
  • JALIL JALILI
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Anuwat Jiravarnsirikul
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Department of Ophthalmology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand, Thailand
  • Christopher Bowd
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Akram Belghith
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Michael Henry Goldbaum
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Sally L. Baxter
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Division of Biomedical Informatics, Department of Medicine, University of California San Diego, California, United States
  • Linda M Zangwill
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Mark Christopher
    Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
    Hamilton Glaucoma Center, Viterbi Family Department of Ophthalmology, Shiley Eye Institute, University of California, San Diego, La Jolla, CA, USA, California, United States
  • Footnotes
    Commercial Relationships   JALIL JALILI None; Anuwat Jiravarnsirikul None; Christopher Bowd None; Akram Belghith None; Michael Goldbaum None; Sally L. Baxter Optomed, Topcon, Code F (Financial Support); Linda Zangwill Abbvie Inc., Topcon , Code C (Consultant/Contractor), National Eye Institute, Carl Zeiss Meditec Inc., Heidelberg Engineering GmbH, Optovue Inc., Topcon Medical Systems Inc. , Code F (Financial Support), Zeiss Meditec AISight Health (cofounder and board member), Code P (Patent); Mark Christopher NEI, The Glaucoma Foundation, Code F (Financial Support), AISight Health, Code O (Owner)
  • Footnotes
    Support  NEI: R00EY030942, R01EY034146, R01EY029058, R01EY11008, R01EY19869, R01EY027510, R01EY026574, EY018926, P30EY022589, DP5OD029610, OT2OD032644. Research grant from The Glaucoma Foundation. Unrestricted grant from Research to Prevent Blindness (New York, NY).
Investigative Ophthalmology & Visual Science June 2024, Vol.65, 2832. doi:
  • Views
  • Share
  • Tools
    • Alerts
      User Alerts
      You are adding an alert for:
      Evaluating GPT-4 Diagnostic Proficiency in Glaucoma Detection through Fundus Image Analysis
      You will receive an email whenever this article is corrected, updated, or cited in the literature. You can manage this and all other alerts in My Account
      The alert will be sent to:
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation
      Citation

      JALIL JALILI, Anuwat Jiravarnsirikul, Christopher Bowd, Akram Belghith, Michael Henry Goldbaum, Sally L. Baxter, Linda M Zangwill, Mark Christopher; Evaluating GPT-4 Diagnostic Proficiency in Glaucoma Detection through Fundus Image Analysis. Invest. Ophthalmol. Vis. Sci. 2024;65(7):2832.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

      ×
    • Get Permissions
  • Supplements
Abstract

Purpose : To investigate the feasibility of employing GPT-4 (OpenAI, San Francisco, CA) for fundus image analysis in glaucoma detection, and to validate its diagnostic accuracy and capability in identifying key features compared to expert evaluations.

Methods : 100 randomly-selected fundus images from the public ACRIMA dataset (50 healthy, 50 glaucoma) were processed by the GPT-4 Vision Preview model through OpenAI's API. For each image, we submitted a standardized text prompt along with fundus image focused to the optic disc region. The prompt asked GPT-4 to evaluate each fundus image based on several criteria: image quality (good, moderate, or poor), image gradeability (gradeable vs. ungradeable), cup-to-disc ratio (normal vs. enlarged), presence of peripapillary atrophy, presence of disc hemorrhages, presence and location of rim thinning (by quadrant and clock hour), glaucoma status (glaucoma vs. non-glaucoma), and an estimated probability of glaucoma status. The images were also reviewed by two expert graders using the same criteria. Expert reviewers were blind to each other and GPT-4 grades. GPT-4 predictions were compared with expert grader evaluations and ACRIMA ground truth to determine agreement and accuracy metrics.

Results : The GPT-4 model after three submissions with the same prompt successfully provided descriptions for all 100 fundus images through the API. Using the ACRIMA ground truth, GPT-4 performance in glaucoma detection had an accuracy of 68.2% and a sensitivity of 70.7%, closely matching the 2 graders’ accuracy (77.9% and 71.7%), and sensitivity (both 68.9%). GPT-4 specificity (65.9%) was lower than the 2 graders (86.0% and 74.5%). Agreement between GPT-4 and grader 1 (76% agreement, kappa = 0.52), is higher than the agreement between the 2 graders (72% agreement, kappa = 0.42).

Conclusions : GPT-4 shows potential for providing clinical decision support and drafting clinical notes on glaucoma detection through fundus image analysis, with a substantial alignment with expert evaluations in describing diagnostic features. Fine-tuning the model for this task could increase accuracy of glaucoma detection by GPT-4.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Figure 1: Fundus images alongside GPT-4’s API text responses for both accurate and incorrect glaucoma diagnoses
View OriginalDownload Slide

Figure 1: Fundus images alongside GPT-4’s API text responses for both accurate and incorrect glaucoma diagnoses

Figure 1: Fundus images alongside GPT-4’s API text responses for both accurate and incorrect glaucoma diagnoses

Figure 1: Fundus images alongside GPT-4’s API text responses for both accurate and incorrect glaucoma diagnoses
View OriginalDownload Slide
 
Figure 2: Confusion matrices comparing the glaucoma grades of GPT-4, expert graders, and ACRIMA ground truth.
View OriginalDownload Slide

Figure 2: Confusion matrices comparing the glaucoma grades of GPT-4, expert graders, and ACRIMA ground truth.

Figure 2: Confusion matrices comparing the glaucoma grades of GPT-4, expert graders, and ACRIMA ground truth.

Figure 2: Confusion matrices comparing the glaucoma grades of GPT-4, expert graders, and ACRIMA ground truth.
View OriginalDownload Slide

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Attribution-NonCommercial-NoDerivatives
Copyright © 2025 Association for Research in Vision and Ophthalmology.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×