Abstract
Purpose :
Generative Language Models (GLMs), such as OpenAI's GPT-4, are finding growing utility in the simplification of complex medical material to enhance comprehension, particularly within populations with low health literacy (LHL). This research explores the viability of using GPT-4 in transforming research abstracts, patient education materials (PEMs) and creating De Novo information prompted by patients.
Methods :
62 Abstracts and 9 PEMs were obtained from the three glaucoma journals and the American Glaucoma Society. The analysis involved the assessment of these texts using Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) scores. Subsequently, we employed GPT-4 to transform these abstracts to a 5th-grade reading level, reevaluated the readability of the abstracts, and ensured content similarity using Latent Semantic Analysis (LSA). Furthermore, GPT-4 was prompted to transform a standardized patient prompt regarding glaucoma management to 6 different education (n=5 for each education level). After assessing outputs using FKRE and FKG, unidirectional ANOVA, Brown-Forsythe, and Welch’s post-hoc analyses were performed to confirm differences in readability.
Results :
In the study of 62 abstracts, the initial average FKGL was 10.82 ± 10.82, with an FKRE of 43.64 ± 11.85. Post-transformation, these values changed to FKGL 7.61 ± 1.78 and FKRE 71.24 ± 9.00. For the 9 PEMs, FKGL changed from an original 8.47 ± 2.63 to 6.09 ± 1.31, and FKRE from 65.19 ± 14.44 to 77.34 ± 8.66. The abstracts saw an average FKGL reduction of 30%, (p < 0.0001) and FKRE increase of 66%, (p < 0.0001). Patient articles had an FKGL reduction of 28%, (p = 0.0272) and FKRE increase of 19% (p = 0.0459). The cosine similarity for abstracts was 0.86 ± 0.14 and 0.94 ± 0.10 for patient articles. De Novo FKGL and FKRE score differences were statistically significant (p < 0.01) among all reading levels except those between 5th and 8th grades, and Bachelor’s and Doctorate Degrees.
Conclusions :
ChatGPT's potential in simplifying medical texts, as evidenced in this study, underscores the importance of leveraging technology to enhance patient comprehension and accessibility of medical literature. This shows the potential of GLMs in making complex medical information more accessible to broader audiences, including those with LHL. Furthermore, the findings demonstrate ChatGPT’s ability to produce outputs at varying education level reading levels.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.