Multimodal ophthalmic image registration using Hessian feature spaces

Abstract

Purpose : The adoption of big data analytics and machine learning to multimodal ophthalmic images incurs a need to develop automated systems for image registration across time and across modalities with disparate spatial scales and resolutions. We hypothesize that there exists a unique Hessian feature space that facilitates multimodal image registration. We present a computational framework for testing this hypothesis and performing multimodal image registration efficiently and accurately, with minimal computational cost.

Methods : For each of 2 healthy subjects and 6 patients with age-related macular degeneration, we collected images of 3 modalities: color fundus photography, blue autofluorescence scanning laser ophthalmoscopy (SLO), and near-infrared (IR) reflectance SLO. We applied image processing techniques to enhance perceptual details. We then extracted gradient, Hessian, and Laplacian features in Riemannian geometry and constructed a Hessian feature space, within which we alienated the conformist and nonconformist features using Otsu’s binarization, while removing weakly connected features. We applied phase correlation to the feature space and devised a geometric transformation for image registration. The accuracy of registration by our approach was compared with that of SURF-KAZE-FREAK (as a benchmark), using Kullback-Leibler divergence. A Kullback-Leibler value close to 0 signifies distribution similarity.

Results : Subjective examination of the resulting images showed excellent registration (Figure 1). Our method significantly outperformed the benchmark. Kullback-Leibler divergence ranged 0.001–0.037 for our approach and 0.080–0.201 for the benchmark, confirming that robust registration had been achieved.

Conclusions : We found a parametric feature space between ophthalmic imaging modalities that is highly useful for image registration, not merely for visual presentation but also for use in multimodal machine learning requiring accurate spatial alignment. Our computational framework detects and extracts the Hessian feature space and registers the images within that feature space. Our future research will expand this approach to include additional ophthalmic imaging modalities and implement the computational framework as a fully automated learning device for better scalability.

This is a 2020 ARVO Annual Meeting abstract.

 

Top row: Color fundus photo (CFP); autofluorescence (AF); registration result. Bottom row: Feature representations of CFP and AF; registration result.