Abstract
Purpose :
To assess the performance of “Bard”, one of ChatGPT’s competitors, in answering practice questions for the ophthalmology board certification exam.
Methods :
In December 2023, 250 multiple-choice questions from the “BoardVitals” ophthalmology exam question bank were randomly selected and inputted into Bard to assess the artificial intelligence chatbot’s ability to comprehend, process, and answer complex scientific and clinical ophthalmic questions. A random mix of text-only and image-and-text questions were selected from 10 subsections. Each subsection included 25 questions. The percentage of correct responses was calculated per section and an overall assessment score was determined.
Results :
On average, Bard answered 62.4% of questions correctly. The worst performance was 24% correct on the topic of “Retina and Vitreous”, and the best performance was on “Oculoplastics” with a score of 84% correct. While the majority of questions were inputted with minimal difficulty, not all questions were able to be processed by Bard, particularly questions that included human images and multiple visual files. Some vignette-style questions were also not understood by Bard and were therefore omitted. Future investigations will focus on including more questions per subsection to increase available data points.
Conclusions :
While Bard answered 62.4% of questions correctly and is capable of analyzing vast amounts of medical data, it ultimately lacks the holistic understanding and clinical experience of an ophthalmologist. Physicians’ judgment in making a medical diagnosis remains irreplaceable and artificial intelligence should be employed as a valuable tool for supplementing, rather than replacing, medical diagnosis.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.