Abstract
Purpose :
Artificial Intelligence (AI) is becoming increasingly popular in the scientific field, as it allows to analyze extensive datasets and summarize results of academic papers. This study investigates the role of AI in Systematic Literature Review (SLR), focusing on its contributions and limitations in article selection and data organization.
Methods :
We tested Scite and Elicit AIs to research articles on “Glaucoma, pseudoexfoliation and Hearing Loss” and compared the results with our previously human-conducted PRISMA-based SLR. Then we used Elicit and ChatPDF to assess their capability to extract and organize key information from scientific articles.
Results :
Articles research using Scite provided 375 results including 10 of the 32 articles selected in our original SLR, while Elicit produced 270 results which included 15 out of 32. Elicit allows to upload PDFs and organize information into customizable tables based on criteria of interest. We uploaded the 32 PDFs of our SLR and verified information’s accuracy in the presented tables. Information on glaucoma type was correct for all articles except for 2. Patients' age was reliable in 18 cases (56.2%) whereas it was not reported in 1 (3.1%) and inaccurate in 13 (40.6%). Results for sex distribution were correct in 17 cases (53.1%), imprecise in 11 (34.4%), missing in 4 (12.5%). Number of patients were properly identified in 26 papers (81.3%), not reported in 2 (6.2%) and inaccurate in 4 (12.5%). Measured outcomes were precise in 17 cases (53.1%), lacking details in 14 (43.7%) and not described in 1 (3.2%). Main findings were precise in 14 cases (43.7%), not reported in 1 (3.1.%), and approximate in 17 (53.1%). ChatPDF allows to upload PDF articles and ask questions about their content. We searched the same information as we did for Elicit obtaining detailed and accurate data in 93.7% of cases.
Conclusions :
Our research highlighted the potentials and limitations of AI in SLR development. Neither Scite nor Elicit were able to provide the totality of the results found using the human-based PRISMA method. Elicit's ability to process and summarize information in tables is time-saving, though its accuracy may not always be assured. ChatPDF gave overall reliable and precise information, proving to be beneficial for SLR writing. Nevertheless, the active participation of human researchers remains crucial to mantain control over the quality, accuracy, and objectivity of their work.
This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.