Abstract
Purpose :
AI chatbots are being increasingly used in healthcare settings for patient education and improving patient care. While access to healthcare information is extensively available online, this healthcare information is often too complex to understand for patients, thus limiting health literacy in our patient populations. With the rise of popularity in AI chatbots, our goal is to determine if ChatGPT Version 3.5 can mitigate text complexity for existing patient educational materials and can be reliably recommended by physicians to simplify text from the American Academy of Ophthalmology (AAO) in English and Spanish.
Methods :
Fifteen articles were chosen from AAO in both English and Spanish and were simplified with ChatGPT 10 times each. Readability of the original and simplified articles was assessed with Flesch-Reading-Ease(FRE) and Gunning-Fog-Index for English; and Fernández-Huerta, Gutiérrez, Szigriszt-Pazo, INFLESZ, and Legibilidad-µ for Spanish. Grade levels to assess readability were calculated with Flesch-Kincaid-Grade-Level and Crawford Nivel-de-Grado.
Mean, standard deviation, and two-tailed t-tests were performed to assess differences before and after simplification.
Results :
English
The average grade level of the 15 original English articles was 8.43±1.17 when analyzed with the FKGL formula. Following simplification on ChatGPT, the grade level increased to 8.80±1.88 (p=0.53). None of the readability formulas showed statistically significant differences between original and simplified articles in English.
Spanish
The average grade level of the 15 original Spanish articles was 5.3±0.34 using the CNG formula. This was reduced to 4.15±1.10 (p=0.0001) after simplification. Notably, all texts were rated as normal reading difficulty with scores around 40. The average score on the Lµ scale was 55.57±5.02, which increased to 62.34±8.23 after simplification (p=0.003). None of the other reading scales showed statistical significance before and after simplification.
Conclusions :
Readability of AAO articles in English worsened without significance but was significantly improved in Spanish. This may result from simpler syllable structure and lesser overall vocabulary in Spanish. With increased testing, physicians can recommend ChatGPT for Spanish-speaking patients to improve health literacy.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.