Abstract
Purpose :
Real-time imaging in vitreoretinal surgery directly impacts the handling of surgical instrumentation and the decision-making process of the surgeon. This experimental study tested the hypothesis that deep learning neural networks can concurrently track and segment instruments and target tissue landmarks within the retina when attached to a surgical microscope.
Methods :
A hundred and one (101) vitreoretinal procedures consisting of core vitrectomy, membranectomy, and endolaser application were employed to train and validate an instance segmentation convolutional neural network. Three vitreoretinal surgeons manually annotated the following features from six-hundred and six (606) fundus frames: vitrector, forceps and endolaser tooltips, optic disc, fovea, retinal tear and detachment, fibrovascular proliferation, endolaser spot, the area where endolaser was applied, and macular hole. Five-fold cross-validation was employed to assess the performance of the model.
Results :
The model detected and classified the vitrector tooltip with a mean area under the precision-recall curve (AUPR) of 0.972±0.009. Segmentation of target tissues such as optic disc, fovea, and macular hole reached mean AUPR values of 0.928±0.013, 0.844±0.039, and 0.916±0.021 respectively (Table 1 and Figure 1). The post-processed image was rendered at a resolution of 1920x1080 pixels at 38.77±1.52 frames per second when attached to an intraoperative visualization system.
Conclusions :
Deep learning neural networks can detect, classify, and segment tissues and instruments during different phases of vitrectomy surgery in real time. We propose a model that creates a framework to develop surgical guidance tools that may guide surgical decision-making in real time. Potential applications include (1) warning of unintended instrument-tissue interactions and potential intraoperative complications, (2) data extraction for the control of equipment parameters such as the vitrector cutting rate, and (3) retroactive data acquisition of instrument maneuvers for surgeon’s skills analysis.
This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.