Feature-Guided Image Fusion of Intrasurgical Optical Coherence Tomography and Digital Surgical Microscopy

Abstract

Purpose : Microscope-integrated optical coherence tomography (MIOCT) is an emerging imaging technology in which live volumetric OCT (“4D-OCT”) is displayed simultaneous with standard stereo zoom color microscopy. 4D-OCT provides ophthalmic surgeons with many visual cues not available in microscopy, but it cannot serve as a replacement due to lack of color features. In this work, we propose a unified solution fusing data from both modalities, guided by segmented 3D features, yielding a more efficient visualization of cues from each.

Methods : Posterior segment 4D-OCT images were manually segmented into the anterior retinal surface, retinal pigmented epithelium (RPE) surface, retinal volume, and non-retinal volume. Projection of shadows on the RPE surface was processed to obtain an image of the retinal vessels. Anterior segment 4D-OCT was manually segmented into anterior and posterior corneal surfaces, corneal volume, and sub-iris volume.
To render volumes, the sub-iris and non-retinal volume emissions were computed as the product of the OCT intensity with the RGB value of the projected microscopy image. Volume densities were proportional to OCT intensity. Retinal volume emission was the product of OCT intensity, the vessel image projection, and a preset RGB value. Densities were computed likewise aside from the preset RGB. Corneal emission and density were constant-valued.
Smoothed surfaces were rendered with compositions of hard and soft specular shading lit by a set of distant light sources. As a preliminary study, only saved imagery from prior surgeries was used and computation time was not optimized.

Results : Figures 1 and 2 demonstrate how this fusion technique combines detail from microscopy with 4D-OCT (blood vessels, tools, pigmentation). The addition of this information does not compromise the 3D context provided by the OCT data, with specular surface highlights communicating surface topology while permitting visualization of colored subsurface features.

Conclusions : 4D-OCT images and stereo zoom color microscopy can be fused to produce a single novel visualization integrating information from each in an effective, constructive manner.

This is a 2021 Imaging in the Eye Conference abstract.

 

3D-OCT and 2-D microscopy are fused into a single comprehensive visualization. Examples include a soft-tip tool in contact with the retina, retinal macular hole, and partially collapsed cornea.

 

Alternative views were rendered to further demonstrate 3D context.