Abstract
Purpose :
To externally validate and assess the generalization performance of a deep learning model originally developed for the differentiation of etiologies of pediatric retinal hemorrhages, leveraging a multicenter dataset and open-source data.
Methods :
A new dataset comprising 641 fundus photographs with annotated retinal hemorrhages (309 traumatic, 332 medical) was collated from two different hospitals and four open-source datasets. The original deep learning models (ResNet18 and FastViT-SA12) from a previous study were evaluated on this external dataset using established performance metrics, including sensitivity, specificity, and overall accuracy. We also conducted subgroup analyses to investigate the model's performance for traumatic and medical etiologies.
Results :
The external validation demonstrated robust model performance, with both models exhibiting high accuracy and discriminative ability across the new dataset. The ResNet18 model achieved an accuracy of 90.17%, with a sensitivity of 88.86%, specificity of 91.91%, and an AUC of 0.9238. It misclassified 37 medical cases as trauma and 26 trauma cases as medical. The FastViT-SA12 model outperformed with an accuracy of 92.67%, sensitivity of 89.16%, specificity of 96.76%, and an AUC of 0.9649. The transformer model showed fewer misclassifications, with 36 medical cases misidentified as trauma and only 11 trauma cases misclassified as medical.
Conclusions :
The successful external validation of the ResNet18 and FastViT-SA12 models on a heterogeneous dataset emphasizes the potential of machine learning in accurately classifying the etiology of retinal hemorrhages. The reduced number of misclassifications by the FastViT-SA12 model, compared to ResNet18, underscores the importance of model selection based on specific clinical requirements and dataset characteristics. These findings affirm the critical role of external validation in verifying the robustness and generalizability of machine learning models, paving the way for their integration into clinical practice to enhance diagnostic accuracy and patient care in ophthalmology.
This abstract was presented at the 2024 ARVO Imaging in the Eye Conference, held in Seattle, WA, May 4, 2024.