Abstract
Purpose :
A steady increase in the use of Ophthalmology services over the past 2 decades combined with a forecasted shortfall of trained ophthalmologists, make the optimisation of patient triage vital if delivery of care is to be maintained or improved.
The purpose of this project is to develop a Machine learning-driven algorithm to risk stratification support in Ophthalmologic Emergency department triage and evaluate its performance when compared with the performance of experienced emergency triage nurses.
Methods :
A dataset of structured presentation of 9,226 cases (with demographics, signs/symptoms, and discharge diagnoses) seen at Moorfields Eye Hospital A&E (October/2021-Jun/2022) were collected at the triage, analysed and merged with the outcomes discharged diagnoses and treatment. Discharge diagnoses were risk-stratified as elective, urgent and emergency by the consensus of three experienced A&E consultants. The dataset was divided into training (n=7856|85.2%), validation (n=685|7.4%), and test (n=685|7.4%) sets. We evaluated three ML models (decision tree|random forest|XGBoost) predicting the risk ss from presenting clinical data. The best model (maximum specificity with sensitivity>95%) on the validation set was selected for testing and comparison with triages performed by nurses in current practice.
Results :
XGBoost showed superior performance (validation sensitivity=95.1%|specificity=33.1%|ROCAUC=0.776), and the features that increased predicted urgency by order of importance were short duration, red/injected eye, male gender, unilateral presentation, age<50, photophobia, and floaters.
In the test set, the model had the sensitivity of 94.9% (92.5%-96.7%), specificity of 28.0% (22.0%-34.5%), ROCAUC of 0.758.
Nurses had sensitivity of 95.6% (93.3%-97.2%, p=0.65 compared to model) and specificity 15.2% (10.6%-20.7%, p=0.004). The model reduced urgent referrals by 4.8%(3.2%-6.7%) compared to nurses.
Conclusions :
Our model had comparable sensitivity and superior specificity to triage nurses, suggesting that the use of ML could enhance emergency care service utilisation. Further research is required to determine if ML could support triage nurses in their decision-making in the clinical setting.
This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.