Abstract
Purpose :
Automated machine learning (AutoML) is a novel tool machine learning research that does not require coding. This study assessed the performance of AutoML in classifying cataract surgery phases from surgical videos.
Methods :
Two ophthalmology trainees without coding experience designed a deep learning model in Google Cloud AutoML Video Classification for the classification of 10 different cataract surgery phases. We used two open-access publicly available datasets (total of 122 surgeries) for model training, validation and testing. Overall, 1,280 video segments were used for training and 144 segments for testing. External validation was performed on 10 surgeries issued from another dataset.
Results :
The AutoML model demonstrated excellent discriminating performance, even outperforming bespoke deep learning models handcrafter by experts. The area under the precision-recall curve was 0.855. At the 0.5 confidence threshold cut-off, the overall performance metrics were as follows: sensitivity (81.0%), recall (77.1%), accuracy (96.0%) and F1 score (0.79). The per-segment metrics varied across the surgical phases (precision 66.7–100%), recall 46.2–100% and specificity 94.1–100%. Hydrodissection and phacoemulsification were the most accurately predicted phases (100 and 92.31% correct predictions, respectively). During external validation, the average precision was 54.2% (0.00–90.0%), the recall was 61.1% (0.00-100%) and specificity was 96.2% (91.0-99.0%).
Conclusions :
A code-free AutoML model created by two ophthalmology trainees can accurately classify cataract surgery phases from surgical videos with an accuracy comparable or better than bespoke deep learning models developed by experts.
This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.