Automated periorbital segmentation for gaze measurement in oculoplastic patients

Abstract

Purpose : Eye mobility is a general indicator of eye health and is presently measured on a coarse scale in clinical settings. We develop a model to segment periorbital features in images of oculoplastic patients for the purpose of potentially extracting gaze measurements from the segmentations. We generalize a previous model that works on primary gaze photos to work on non-primary gazes.

Methods : We start with a previously-published PSPNet-based network with a ResNet-50 backbone pre-trained to segment sclera, irises, and eyebrows on primary gazes of oculoplastics patients photographed in a clinical setting. We replace the eyebrow class with a pupil class to aid in tracking, and fine-tune the network on 9-gaze RGB images of 20 different oculoplastics patients whose sclera, irises, and pupils are manually segmented by a clinician. Standard affine and color jitter augmentation is used to mitigate overfitting.

Results : The IoU averaged over all images in the validation set of 8 subjects is 0.84[95%CI:0.83,0.86], 0.81[0.80,0.83], 0.60[0.56,0.64] for the sclera, iris, and pupil classes, respectively. Comparing the original mode to the fine-tuned model, the IoU improved from 0.40[0.33,0.47] to 0.83[0.81,0.85] for the sclera and from 0.62[0.55,0.69] to 0.84[0.82,0.85] for the iris. The fine-tuned model achieved higher IoUs than the original model for all gazes except for the primary gaze: 0.77[0.73,0.81], 0.90[0.88,0.92] (original) vs 0.77[0.72,0.82], 0.86[0.88,0.92] (fine-tuned) for sclera and iris respectively. The most improvement was for the three down gazes, where the IoU range was 0.80-0.85 compared to 0.08-0.41 of the original model.

Conclusions : The model we developed segments periorbital eye features in non-primary gazes of faces to an accuracy that may be suitable for eye tracking in a video sequence. Future work for this involves accounting for head movements between frames and generalizing the model to work with images or videos taken by phone cameras.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.

 

Sclera (orange) and Iris (purple) segmentation overlays of original model trained on primary-gazes only (left) vs model fine-tuned on 9 gazes (right) on primary (top) and non-primary gaze. The original model includes an Eyebrow class while the fine-tuned model includes a Pupil class.

 

Overlays of the Sclera (orange), Iris (purple), and Pupil (yellow) segmentations for 9 gazes detected by the fine-tuned model.