OPTIMEyes: An Annotation and Inference Feedback Tool for Multimodal Ophthalmic Imaging

Abstract

Purpose : Annotating image datasets for training machine learning algorithms can be tedious. We introduce OPTIMEyes (Ophthalmic Photography and Tomography Inference and Marking Engine), a novel open-source viewing, annotation, and inference tool for application in ophthalmology. Leveraging the open-source MONAI label toolkit for smart labeling, OPTIMEyes improves a backend artificial intelligence (AI) model through active learning over time. Specifically, OPTIMEyes utilizes AI-assisted annotation for diverse ophthalmic image modalities to accelerate image annotation, model training and inference, creating a continuously self-improving tool for image analysis.

Methods : OPTIMEyes can handle various ophthalmic image filetypes and modalities. Users can zoom, pan, and annotate an image with a brush or polygon draw feature that outlines and fills disease specific lesions. Additionally, an erase tool is available to remove lesion predictions the user deems incorrect. Users can train a model from scratch with annotations or leverage existing pre-trained models. Whether the model is pre-trained or not, additional annotations on new images can be saved, allowing the backend AI model to learn from these annotations. The updated model can then perform inference on new images, they can be corrected and then saved to the model to create an active learning feedback loop. We qualitatively evaluated the tool's performance in terms of inference accuracy, ease of annotation, and improvement in model performance over time.

Results : Initial results demonstrate that OPTIMEyes can effectively learn from user annotations, with improvement in inference accuracy observed over iterative feedback cycles. OPTIMEyes is adept at handling different image modalities, and user feedback highlighted its ease of use and efficiency in annotation tasks. Model performance showed progressive improvement, indicating learning from annotations and adaptation to the ophthalmic imaging context.

Conclusions : OPTIMEyes demonstrates significant potential to boost the efficiency of ophthalmic image annotation, improve model performance and accelerate clinical integration of machine learning algorithms in the field of ophthalmology. Its ability to learn from user inputs decreases the amount of fully manual annotations needed from clinicians while simultaneously allowing them to correct inference errors, thereby continuously enhancing model performance.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.