EyeLiner: Longitudinal retinal fundus image alignment through clinically guided keypoint detection

Abstract

Purpose : Detecting and measuring changes in longitudinal imaging is key to monitoring disease progression in chronic ocular diseases like glaucoma and age-related macular degeneration. We introduce our AI-based pipeline, “EyeLiner,” for registering, or aligning, 2D color fundus photographs. This will enable clinicians to better assess change in disease status over time.

Methods : EyeLiner utilizes blood vessel segmentation masks for registering a "moving" image to a "fixed" image (Fig.1). The segmentations were generated using a Unet model. We focus only on the peripheral vessels and exclude the optic disk vessels (indicated by “+” and “-” in Fig.1) as these are generally static. A smart AI-based keypoint detection algorithm then detects corresponding keypoint matches between fixed and moving images. Lastly, the matches are passed to an alignment algorithm to compute the spatial transformation between the image pair. We evaluate EyeLiner on three datasets: Fundus Image REgistration (FIRE, 134 pairs), Sequential Fundus for Glaucoma Forecast (SIGF, 31 pairs), and our internal glaucoma dataset from the Colorado Ophthalmology Research Information System (CORIS, 10 pairs). Performance is computed as the mean Euclidean distance between clinician-annotated keypoints on the fixed and the registered moving images. Defining a successful registration as one below a distance threshold, we plot the success rate over varying thresholds. This gives a monotonic curve from which we compute the area under curve (AUC) statistic to summarize model performance.

Results : Image registrations from FIRE, SIGF, and CORIS are visualized as checkerboards with the distance values (Fig.2a). Our pipeline effectively aligns the images, as seen by the continuity in the vessels. Quantitative results show that our best model obtains AUCs of 0.85, 0.91 and 0.89 on FIRE, CORIS and SIGF, respectively, beating the current state-of-the-art model SuperRetina (Fig.2b).

Conclusions : Our novel EyeLiner pipeline aligns image pairs despite substantial physiological and camera-related changes. While our pipeline was primarily tested on fundus photographs, we anticipate that our approach is extendable to any domain with longitudinal imaging data.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Figure 1: EyeLiner pipeline for registration of 2D ocular images.

 

Figure 2: EyeLiner registrations results on FIRE, CORIS, and SIGF datasets.