Imageability and Registration of Multimodal Imaging using Machine Learning

Abstract

Purpose : Advances in different imaging techniques allow for high-quality multimodal acquisition of the retinal structure. Analyzing its changes over time gives crucial insights into the progression of retinal diseases. We propose artificial intelligence (AI) methods that allow for such analyses by automatically (1) detecting low-quality images (imageability) and (2) aligning all follow-up scans of different modalities (spatio-temporal registration).

Methods : Given 378 optical coherence tomography (OCT), 1282 color fundus (CF) and 900 fluorescein angiography (FA) scans and their manual quality labels in the imaging database of Vienna Reading Center (VRC), we trained and validated different methods of machine learning. Figure 1 shows how our registration algorithm aligns scans from different modalities or from the same modality over different time points. We first apply a deep learning method and extract the vessel structure from the en-face view. Then we apply image registration techniques to find the affine transformation that overlaps the major vessels of each pair of scans. For automated imageability assessment (Figure 2), we first apply image-processing techniques and extract 31 objective quality features from the scans. Then we apply a classic machine learning method (binary decision tree) to classify the scans in good and bad quality, with respect to different quality criteria.

Results : A certified grader of the VRC evaluated the registration method. From 863 pairs of scans %80.3 were correctly aligned. We also found that the manual qualitative evaluation correlates well with an automated quantitative evaluation that measures the false positive rate of the vessel overlap. The imageability method was evaluated based on 10-fold cross-validation and achieved an AUC of 0.919 to 0.998 for different quality criteria.

Conclusions : Image quality of OCT, CF and FA modalities can be rated accurately and fully automatically within seconds after acquisition. Furthermore, the automated image alignment allows for transferring differently captured retina of a patient into one coordinate system to facilitate multimodal longitudinal analysis over both time and modality.

This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.

 

Figure 1: The processing pipeline of automated image alignment (registration)

 

Figure 2: the result of automated image quality assessment (imageability) in each relevant category, as the area under the receiver operating characteristic curve (AUC).