June 2020
Volume 61, Issue 7
Free
ARVO Annual Meeting Abstract  |   June 2020
Deep-learning based automated instrument-tracking and adaptive-sampling for 4D imaging of ophthalmic surgical maneuvers
Author Affiliations & Notes
  • Eric Tang
    Vanderbilt University, Reston, Virginia, United States
  • Mohamed T El-Haddad
    Vanderbilt University, Reston, Virginia, United States
  • Joseph D Malone
    Vanderbilt University, Reston, Virginia, United States
  • Yuankai Tao
    Vanderbilt University, Reston, Virginia, United States
  • Footnotes
    Commercial Relationships   Eric Tang, None; Mohamed El-Haddad, Vanderbilt University (P); Joseph Malone, Vanderbilt University (P); Yuankai Tao, Leica Microsystems (R), Vanderbilt University (P)
  • Footnotes
    Support  None
Investigative Ophthalmology & Visual Science June 2020, Vol.61, 2544. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Eric Tang, Mohamed T El-Haddad, Joseph D Malone, Yuankai Tao; Deep-learning based automated instrument-tracking and adaptive-sampling for 4D imaging of ophthalmic surgical maneuvers. Invest. Ophthalmol. Vis. Sci. 2020;61(7):2544.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : Intraoperative optical coherence tomography (iOCT) benefits clinical decision-making and allows for verification of surgical goals by providing high-resolution volumetric images of tissue microstructure. Current microscope-integrated OCT systems are limited by slow scan speeds and static OCT fields-of-view (FOVs), which preclude real-time 4D visualization of surgical maneuvers and require periodic adjustment of the OCT field, respectively. Here, we present deep-leaning based automated instrument-tracking and adaptive-sampling using spectrally encoded coherence tomography and reflectometry (SECTR) for 4D imaging of ophthalmic surgical maneuvers.

Methods : A custom 400 kHz swept-source OCT engine was used to perform SECTR imaging in ex vivo porcine eyes. En face spectrally encoded reflectometry (SER) and cross-sectional OCT images were simultaneously acquired at 1664x1000x200 pix. (spectral x lateral x lateral). A convolutional neural network (CNN) was trained to detect 25G ILM forceps using 2808 SER images. Bounding box outputs produced by the CNN were transferred to an FPGA to offset scan waveforms in real-time to localize and densely sample surgical instruments within the OCT FOV (Fig. 1).

Results : Automated instrument-tracking was performed in intact ex vivo porcine eyes. Benchtop imaging was performed at 2 volumes per second with a tracking rate of 18 Hz. SER images show dense sampling of the instrument tip despite lateral movement. Corresponding OCT B-scans and OCT volumes show tracking of the instrument following out-of-plane motion (Fig. 2).

Conclusions : SECTR imaging allows for real-time localization of surgical instruments and dynamic adjustment of OCT scanning. Adaptive-sampling enhances visualization of instrument-tissue interactions by increasing sampling density without sacrificing speed. The proposed method overcomes limitations of conventional iOCT by providing 4D visualization of surgical maneuvers without the need for manual OCT aiming.

This is a 2020 ARVO Annual Meeting abstract.

 

(a) SER images (blue) are passed through the CNN. An FPGA is used to offset X (green) and Y (orange) galvanometer positions to densely sample the instrument tip and adjust OCT scanning position.

(a) SER images (blue) are passed through the CNN. An FPGA is used to offset X (green) and Y (orange) galvanometer positions to densely sample the instrument tip and adjust OCT scanning position.

 

(a), (b) SER images (blue) show adaptive-sampling in an ex vivo porcine eye following out-of-plane motion. (c)-(f) OCT B-scans and volumes (red) demonstrate automated localization of the instrument (yellow asterisk) within the OCT FOV.

(a), (b) SER images (blue) show adaptive-sampling in an ex vivo porcine eye following out-of-plane motion. (c)-(f) OCT B-scans and volumes (red) demonstrate automated localization of the instrument (yellow asterisk) within the OCT FOV.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×