Real-time pupil detection for fundus imager using single-shot detector and hard-negative training.

Abstract

Purpose : Pupil detection is integral to alignment guidance during fundus image acquisition and automated fundus image capture. In this retrospective study, we developed a deep learning algorithm for real-time tracking of pupils at greater than 25 frames per second (fps).

Methods : We collected 13,674 eye images that provide off-axis views of patients’ pupils using prototype software on CLARUS^TM 500 (ZEISS, Dublin, CA). This dataset is divided into 3 parts: Dataset1) Annotated training set containing 4,890 images with manual boundaries marked by at least one of five graders; Dataset2) Unannotated training set with 7,000 images; and Dataset3) Hold-out annotated test set with 784 images from 32 mydriatic and 29 non-mydriatic subjects. Accuracy of the algorithm was measured assuming a successful result meant localization within 400 µm of the manual annotations.
To reduce operation time, a constringed single-shot detector inspired by single-shot multi-box detector is used, comprising of 3 feature extraction and 3 candidate box prediction layers. The confidence score was used to predict at most one box out of 944 candidate output boxes.
The multi-step process is shown in Figure1, specifically, in step I, the SSD is trained using annotated Dataset1. In step II, the trained algorithm is applied to unannotated Dataset2. From the results, severely misidentified images are manually chosen as hard negatives and annotated (692 images). In step III, the SSD trained in step I is transfer-trained using the annotated hard negatives from Dataset2.

Results : The final algorithm achieves accuracies of 95.1% and 98.3% on mydriatic and non-mydriatic images of Dataset3. The algorithm developed in step I without hard-negative training achieved accuracies of 91.7% and 95.6%. Figure 2 shows some of the examples of the pupil detection results. Average execution time of the algorithm is 7.57ms (132 fps) in Macbook Pro i5-7360U CPU, 34.4ms (29 fps) using an Intel® Core^TM i7-6920HQ CPUand 36.2 ms (27 fps) in NVIDIA nano with ARM A57.

Conclusions : The proposed algorithm provides a robust real-time pupil detection for alignment guidance, with accuracy of greater than 95% in detecting the correct pupil location within 400 µm of manual annotations while also operating at a frame rate greater than the camera acquisition.

This is a 2020 Imaging in the Eye Conference abstract.

 

Figure 1: Hard negative training for SSD6.

 

Figure 2: Sample results showing detected center of pupil and confidence metric.