Experimental Investigations of Pupil Accommodation Factors

Abstract

Purpose.: The contraction and dilation of the iris muscle that controls the amount of light entering the retina causes pupil accommodation. In this study, experiments were performed and two of the three factors that influence pupil accommodation were analyzed: lighting conditions and depth fixations. The psychological benefits were not examined, because they could not be quantified.

Methods.: A head-wearable eyeglasses-based, eye-capturing device was designed to measure pupil size. It included a near-infrared (NIR) camera and an NIR light-emitting diode. Twenty-four subjects watched two-dimensional (2D) and three-dimensional (3D) stereoscopic videos of the same content, and the changes in pupil size were measured by using the eye-capturing device and image-processing methods.

Results.: The pupil size changed with the intensity of the videos and the disparities between the left and right images of a 3D stereoscopic video. There was correlation between the pupil size and average intensity. The pupil diameter could be estimated as being contracted from approximately 5.96 to 4.25 mm as the intensity varied from 0 to 255. Further, from the changes in the depth fixation for the pupil accommodation, it was confirmed that the depth fixation also affected accommodation of pupil size.

Conclusions.: It was confirmed that the lighting condition was an even more significant factor in pupil accommodation than was depth fixation (significance ratio: approximately 3.2:1) when watching 3D stereoscopic video. Pupil accommodation was more affected by depth fixation in the real world than was the binocular convergence in the 3D stereoscopic display.

Pupil accommodation occurs as a result of the contraction and dilation of the iris muscle that controls the appropriate amount of light entering the retina; the primary cause of pupil accommodation is environmental lighting conditions. Furthermore, it has been reported that changes in pupil size have relationship with change in depth fixation. ¹ In other words, the pupil size is determined, not only by lighting conditions but also by depth directional fixations. According to previous research, in general, pupil size accommodation can be regarded as having relationship with the following three factors: (1) environmental lighting conditions ^2,3 ; (2) change of depth fixation ^1,4 ; and (3) psychological effects. ^{5 –7}  

The pupil has an opening with a diameter that changes from 2 to 8 mm with the contraction and dilation of the iris muscle. ⁸ The correlation between the amount of light and pupil size has been studied. Prior research on pupil accommodation showed that the pupil size can be represented as the function of the weighted average of the brightness within the human visual field of view. ³ Further, the pupil size is affected more by the central or foveal part of the retina than by its outer areas. ³ Figure 1 shows the rough graph of correlation between brightness and pupil size. Accurate graph of correlation is shown in the reference. ³  

Figure 1.

 

Rough graph of relationship between brightness and pupil size. ³

The second cause of pupil accommodation is change of depth directional fixation. ¹ The iris and ciliary fibers that control pupil size and lens thickness are connected through the ciliary muscle, as shown in Figure 2. ⁸ Even though this muscle is primarily responsible for controlling lens thickness through ciliary fibers, the operation of the iris muscles can be slightly affected by this anatomic structure. 

Figure 2.

 

Human eye model. ⁸

The third factor in pupil accommodation is psychological influences. The pupil dilates in response to auditory emotional (negative and positive) stimuli. ⁹ In a previous study, a task-evoked pupillary response showed a tendency of slight dilation as a result of loads on working memory, increased attention, sensory discrimination, and other cognitive loads. ⁵ One study reported that sad facial expressions with small pupils are judged more significant sadness. ⁶ Moreover, in another study, it complements the previous research introducing that processing of sadness is affected by pupil size. ⁷  

Among these three factors (lighting conditions, depth fixation, and psychological effects), in 3D stereoscopic displays, the first and second ones can be examined by analyzing the correlation between pupil size and amount of each visual stimuli's variables such as the average image intensity and left–right (L–R) disparity in stereoscopic content. The third factor cannot be validated because there are many kinds of emotions, and it is difficult to obtain ground-truth quantitative data on emotional states. Therefore, in this research, we assumed the changes in the pupil size caused by emotional states to be an unknown constant. The two correlations between pupil size and the variables in visual stimuli, such as intensity and L–R disparity, were quantitatively measured. Among the many kinds of visual stimuli, 3D video was chosen because this stimulus includes both variations in the intensity and L–R disparities. In our experiment, 24 subjects were divided into two groups. One group watched three-dimensional (3D) stereoscopic videos, and the other watched two-dimensional (2D) videos. Both the 3D and 2D videos had the same content. During the video-watching stage, eye images are continuously captured by a near-infrared (NIR) camera. Subsequently, we analyzed the correlations between the intensity or L–R disparity and the pupil size. From this research, we obtained the quantitative correlations between the depth directional fixation changes and pupil size. Moreover, the quantitative ratio of the effects of intensity and depth fixation was estimated. 

The rest of this article is organized as follows. In the Methods section, the analysis methods of the amounts of L–R disparity and the lighting conditions of the video stimuli and the processing method of pupil size are introduced. Then, the experimental setup and results are given, respectively. Finally, discussion and conclusions including future works are presented. In addition, the method and apparatus of the proposed measurement of pupil accommodation are presented in 1. 

Methods

Video and Eye Image Analyses

Calculating the Left–Right (L–R) Disparity of Stereoscopic Video.

For a visual stimulus, a 3D stereoscopic movie was chosen. ¹⁰ Figure 3 shows an example of 3D stereoscopic image captured by a binocular camera ¹¹ instead of the 3D stereoscopic movie. The L–R images are produced by the method of red- and green-channel separation. Although the polarized and active shutter type stereoscopic displays are reported to perform better, they require special display equipment including a polarized filter or synchronizing hardware for active shuttering. Therefore, they cannot be adopted for users who are viewing the general 2D displays. The stereoscopic method based on red and green channels has been used as an alternative, as shown in Figure 3. 

Figure 3.

 

Example of a 3D stereoscopic image captured by a binocular camera ¹¹ and its separated L–R component. (a) Sample image. Binarized edge image of the red (b) and green (c) channels in (a). (d) Merged images (b) and (c) (red: edge of red channel; green: edge of green channel; yellow: overlapped edge of both red and green channels).

The positional disparity of one object projected on both (left and right) eyes is binocular disparity. ¹² In accordance with the principle that a human perceives depth information by using two eyes, the depth of a stereoscopic image is determined by the disparity between the left and right images. In Figure 3, the object positioned at mid-depth does not have any L–R disparity. However, objects positioned at near and far depths have greater L–R disparity than that the mid-depth object. In other words, greater L–R disparity implies that the corresponding object is positioned at a nearer or farther position in depth. Furthermore, red-channel components are located at the right side of the green channel in the case of far-depth objects, as shown in Figure 3a. In contrast, red-channel components are located at the left side of the green channel in the case of near-depth objects, as shown at the bottom of Figure 3a. Because of the L–R disparities using red and green separations, watchers can visually perceive the depth variations, by wearing the red–green glasses shown in Figure A1. Moreover, the L–R disparities represent the depth information of the 3D stereoscopic image. 

In this research, we analyzed the correlation between the pupil size and L–R disparity. To obtain the average depth of one image frame, the average L–R disparity is calculated by subtracting the horizontal center of the binarized edge image of the green channel from its corresponding red channel one, based on Figure 3d and the following equation:  

 

where, HC _r and HC _g mean the horizontal centers of binarized edge images of red and green channels, respectively. The binarized edge images of the two channels are obtained by binarization with a threshold of 80, after the Prewitt edge detection method. ⁸ Consequently, D _L–R implies the average disparity between red and green channel components of a 3D stereoscopic video frame. This metric is used for analyzing the correlation between the pupil size and depth directional fixation. The D _L–R values are shown in Figure 4 according to frame number. 

Figure 4.

 

D _L–R of an experimental video sequence according to frame number.

Calculating the Average Intensity.

Image intensity is the cause of the contraction and dilation of a pupil. In this study, we analyzed the correlation between the image intensity and pupil size in subjects watching 2D videos. 

The pupil accommodation can be affected by both the changes of the image intensity and depth fixation. To deal with these two factors, independently, we measure the pupil accommodation data in subjects watching 2D and 3D displays of the same video. The accommodation that occurs when viewing the 2D display is caused by the change of the image intensity of the display. The accommodation that occurs when viewing the 3D display is caused by changes in the depth fixation brought about by the L–R disparity and by the changes of the image intensity of the display. By compensating the latter data (viewing the 3D display) with the former data (viewing the 2D display), we could analyze how the pupil accommodation was affected by the change of depth fixation irrespective of the change in image intensity. 

In the hue, saturation, and intensity (HSI) model, the intensity (I) of one pixel is defined from the red, green, and blue (RGB) color model by averaging the three levels of the RGB channel. ^8,13 Therefore, the average intensity of one video frame is calculated from the following equation:  

 

where r_ij , g_ij , and b_ij refer to the intensity values of red, green, and blue channels at the i, j pixel coordinates, respectively, and I is the average intensity of the video frame. This metric is used for analyzing the correlation between the pupil size and light stimulus. The average intensities of the experimental video are shown in Figure 5, according to frame number. 

Figure 5.

 

Average intensities of an experimental video sequence.

Eye Image Analysis.

In this research, we analyze the correlation between the pupil size from one eye image and its corresponding information (I and D_L–R). However, the pupil size (the number of dark pixels in the pupil region of the eye image) may be 0 or may suddenly decrease when the eye is closed. ¹⁴ Moreover, the calculated pupil size can include some noise components. These factors prevent accurate analyses of the temporal 1D signal of the pupil size. Hence, to refine the analysis, both median and average filters are adopted. It is known that the median filter is effective in case of impulse noise. ⁸ In the temporal 1D signal of pupil size, a sudden decrement in the amplitude is regarded as impulse noise. The size of a sliding window for the median filtering is defined as 11. To suppress the remaining noise elements, average filtering is adopted. Figure 6 shows a part of the raw signal and its filtered results. 

Figure 6.

 

Part of a temporal 1D signal of pupil size and its filtered results.

Experimental Design and Procedure

The research followed the tenets of the Declaration of Helsinki, and informed consent was obtained from the subjects after explanation of the nature and possible consequences of the study. 

Experimental Setup.

To present the 2D and 3D contents, a commercial LCD monitor was used. While watching the 3D contents, the subjects wore red–green glasses that were combined with an eye-image–capturing device (Figure A1). 

Example of experimental configuration is shown in Figure 7. To avoid having the pupil size change as a result of environmental lighting conditions, we maintained the illumination intensity in the test room based on the recommended levels of illumination by the Korean Standard (KS). ¹⁵ External illumination was completely blocked. In addition, auditory noise was blocked. The temperature and humidity were constantly maintained, and there were no vibrations or bad smells. The viewing distances were 60 cm and 90 cm. Our experiments were performed with 24 subjects (12 men and 12 women, aged (mean [26.6] and std. [1.7]). The experiments included the relaxation time during which the eye was closed for 270 seconds and they also included the questionaire-based subjective test. 

Figure 7.

 

Example of experimental setup.

Since the subjective test was done for measuring eyestrain on 2D and 3D displays, ¹⁶ its results are not used in this research. 

To prevent the experimental results from being affected by other factors, such as physical condition, we requested that the subjects get adequate sleep on the day before the experiment. ¹⁶  

Test Video.

To provide visual stimuli to subjects, two modes of a fantasy movie were used. ¹⁰ We purchased two versions of the movie: 2D and 3D stereoscopic DVDs. ¹⁶ The part of video clip from the beginning of the movie, was used for the experiment. Figure 8 shows the 2D and the 3D stereoscopic images, captured by a binocular camera, ¹¹ instead of the movie which was used for the experiment. 

Figure 8.

 

Sample images. Left and right images show 2D and 3D stereoscopic scenes, respectively.

When the subjects watched the 3D stereoscopic movie, they wore red–green glasses with an eye-capturing device, as shown in Figure A1. When the subjects watched the 2D movie, the same eye-capturing device was used, but the red–green glasses was removed. 

Experimental Procedure.

To comparatively evaluate two correlations (d _a vs. D _L–R and d _a vs. I of equations 1, 2, and 5), the 24 subjects were organized into two groups of 12 each. Since we included two kinds of displays, 3D and 2D, the two subject groups were group 1, viewers of the 2D display, and group 2, viewers of the 3D stereoscopic display. 

Viewing a movie more than once can cause boredom or drowsiness and thus affect pupil accommodation. Since the same movie was used for all testing, the subjects were members of only one group, to avoid such a training effect. ¹⁶  

Results

Pupil Accommodation by Lighting Conditions

To observe the effects of light on pupil accommodation, we performed an analysis to determine the correlation between pupil diameter (d _a in equation 5) and the average intensity of a video frame (I of equation 2). A 3D stereoscopic video comprises two kinds of visual stimuli, intensity and L–R disparity, and hence, only the data acquired from a 2D video were used for this analysis. 

Figure 9 shows a 2D dot graph wherein one dot implies average intensity and its corresponding pupil diameter. 

Figure 9.

 

Graph and linear regression result for the correlation between pupil diameter and average intensity.

From the results shown in Figure 9, we found that there was correlation between pupil size and average intensity. From the 1D linear regression results, the pupil diameter can be estimated as being contracted to approximately 0.0067 mm, with an average intensity change of 1. Furthermore, from this result, we estimate that the pupil diameter contracted from approximately 5.96 mm to approximately 4.25 mm as the intensity varies from 0 to 255. Based on the information on the pupil diameter of 2∼8 mm, ⁸ the light stimuli can induce approximately 28.5% of the maximum pupil accommodation: (5.96 mm − 4.25 mm)/(8 mm − 2 mm). 

Pupil Accommodation by Depth Fixation

To observe pupil accommodation according to depth fixation, we performed analysis of the correlation between the pupil size (d_a in equation 5) and L–R disparity (D _L–R in equation 1). Because the 2D video did not include depth variations, only the pupil size data acquired from the 3D stereoscopic video were used for this analysis. Figure 10 shows the 2D dot graph, in which one dot represents the pupil diameter and its corresponding L–R disparity. 

Figure 10.

 

Graph and linear regression result of the correlation between the pupil diameter and average L–R disparity.

Although the D _L–R has the greater range than −100 ∼100 as shown in Figures 4, the Figures 10 and 11 are shown with the D _L–R of the reduced range of −100 ∼100 for convenience. 

Figure 11.

 

Graph and linear regression result for correlation between normalized pupil diameter and average L–R disparity.

In Figure 10, the pupil size data that were acquired when subjects watched the 3D video includes both depth fixation and light factors. That is, the pupil size was affected by both changes in depth fixation and image brightness in the 3D video. Therefore, to measure the pupil size data affected only by depth fixation (eliminating the light factor), the pupil size when watching the 3D video (d _a ^3D) was subtracted by that when watching the 2D video (d _a ^2D). Consequently, the pupil size data affected only by depth variations could be obtained. The adjusted graph is shown in Figure 11; here, one dot represents a pupil size and its corresponding L–R disparity. 

From the results shown in Figure 11, we found that there was a small correlation between pupil size and depth fixation. From the 1D linear regression result, the pupil diameter can be estimated as being expanded by approximately 0.0021 mm as the L–R disparity is changed by 1. Based on this result, we estimate that pupil diameter changes by approximately 0.42 mm as the L–R disparity changes from −100 to 100. The minimum and maximum pupil diameters are 2 and 8 mm, respectively. ⁸  

Pupil Accommodation by Depth Fixation in the Real World

We performed additional experiments by using a real-world target that moves back and forth (from 10 to 50 cm of Z distance) as shown in Figure 12. By using a panel of uniform brightness without a change in environmental light, as shown in Figure 12, we could collect the pupil size data that were affected only by depth fixation (eliminating the light factor [the first factor of pupil accommodation]). To confirm that the correct data were collected, we measured the environmental light with an illuminometer held in front of the eye and found that it was maintained. 

Figure 12.

 

Experiments for measuring of pupil size by depth fixation in real world.

From the results of this experiment, we found that pupil accommodation was distinguished by the back and forth movements of the target, as shown in Figure 13. The line gradient (0.0419) in Figure 13 is much greater than that (0.0021) in Figure 11. The difference indicates that the correlation between pupil size and depth fixation in the real world is much greater than that in the 3D stereoscopic display. The R ² value (0.783) (which represents the quantitative measure of how well the data can be predicted by the regression model ¹⁷ ) in Figure 13 is greater than that (0.0478) in Figure 11. Based on these results, we find that the pupil size is more affected by the depth fixation in the real world than that in the 3D stereoscopic display. As is shown by the quantitative results in Figure 13, pupil size can be estimated as being changed ∼1.68 mm (5.06 mm − 3.38 mm) which is greater than the change in the 3D stereoscopic display. 

Figure 13.

 

Graph and linear regression results showing the correlation between pupil diameter and Z distance. The Z distance is the real world distance between the human eye and the panel, as shown in Figure 12.

Discussion

In this study, we performed experiments to analyze the reasons for pupil accommodation. Moreover, to supply visual stimuli with variation in light intensity and depth fixation, we used 2D and 3D stereoscopic videos with the same content. The pupil size was calculated by using digital image processing techniques. From the experimental results, we found the correlation of pupil size with average intensity or average L–R disparity. As mentioned earlier, pupil size is affected by three factors: environmental lighting conditions, depth fixation, and psychological effects. Therefore, the pupil size on viewing the 3D display can be modeled as follows:  

 

where w ₁ to w ₃ are the weights and F ₁ to F ₃ represent each factor affecting the pupil size: lighting condition (F ₁), depth fixations (F ₂), and psychological effects (F ₃). We represent F ₁ as an inverse term (1/F ₁), because pupil size is decreased when the amount of light increases. Since pupil size is increased with the longer depth fixation, F ₂ is represented in its direct form (F ₂). However, psychological factors (F ₃) including various kinds of human emotions cannot be quantitatively estimated. Therefore, w ₃ F ₃ (or w ₃/F ₃) is replaced by a constant (c) and was not considered in our analysis. Subsequently, to calculate the ratio between the two weights (w ₁:w ₂), we note the two results shown in Figures 9 and 11. In Figure 9, the pupil sizes are determined by only one factor, F ₁. In the linear regression result, the slope is −0.0067. As mentioned earlier, our light stimuli induced ∼28.5% of the maximum pupil accommodation. Because the slope of the regression line in Figure 11 was 0.0021, the ratio between two weights can be deduced to be approximately 3.2:1 (0.0067:0.0021). In other words, lighting condition has 3.2 times significance of depth fixation in affecting pupil accommodation. However, this ratio is found only by considering the slopes (gradients) of the regression lines. Actually, if the R ²s of Figures 9 and 11 are considered, the weight of lighting condition is regarded as more significant than that of depth fixation. 

In our analyses in Figures 9, 10 and 11, the R ²s (0.2242, 0.0054, and 0.0478, respectively) are not high, which implies that there are other causes of pupil accommodation—for example, the psychological factor such as visual fatigue, concentration power, and degree of interest in the video contents. Moreover, individual variations in the pupil size might be another cause for such low R ²s. R ² represents the quantitative measure of how well the data can be predicted by the regression model. A higher R ² means a better fitting result of the regression model, which represents a more reliable regression result. ¹⁷  

In future studies, we will attempt to measure the influence of psychological factors on pupil accommodation. For this purpose, we will design two kinds of visual stimuli, one 2D video will include interesting contents and the other 2D video will only include the same intensities with no content that can cause emotional variations. We expect to obtain the ratio between F ₁ and F ₃ by using these two videos and the identical analyzing method given in this article. Moreover, using the same methods as given in our previous studies on the objective measurements of visual fatigue using the pupil accommodation speed, studies on the measurement of visual fatigue under various kinds of visual stimuli and environmental conditions will be conducted. 

Footnotes

 Supported by National Institute for Mathematical Sciences (NIMS) Grant I21104 funded by the Korea government and in part by the NAP (National Agenda Project) of the Korea Research Council of Fundamental Science & Technology.

Footnotes

 Disclosure: E.C. Lee, None; J.W. Lee, None; K.R. Park, None

Appendix A

Pupil Size Measurement

Device.

To measure the pupil size, a head-wearable device was designed (Fig. A1). This eyeglasses-based device includes an NIR light camera that can capture the eye image lit by four NIR light-emitting diode (LED) illuminators attached at the four corners of the monitor. ¹⁶ In Lee et al., ¹⁶ the comparative eyestrain in 2D and 3D displays was measured and analyzed on the basis of the eye blink frequency. By adopting the NIR illuminator and the camera with NIR passing filter, our system is robust to the variation of environmental visible light. ¹⁶ Also, since we use the invisible NIR illuminator of 850 nm, the dazzling effect is reduced on the user's eye. In addition, with the NIR illuminator and the camera with NIR passing filter, the captured image has a clear boundary between iris and pupil regions irrespective of iris color. ¹⁶  

Figure A1.

 

Device for measuring pupil accommodation. Reprinted with permission from Lee EC, Heo H, Park KR. The comparative measurements of eyestrain caused by 2D and 3D displays. IEEE Trans Consumer Electron. 2010;56:1677–1683. Copyright 2010 IEEE.

The NIR rejecting filter present inside the conventional web camera is removed, and an NIR passing filter is attached to the camera. Since the camera is attached through a flexible wire frame, the problem of individual variations in the eye position can be solved. ¹⁶ The wavelength of the NIR LED is 850 nm, and it produces an image with a clear edge between the pupil and iris regions, as shown in Figure A1. In two previous studies, it was reported that the NIR light with longer wavelength makes the boundary between the pupil and iris clearer, whereas the NIR light with shorter wavelength makes the boundary between the iris and sclera clearer. ^18,19 However, the conventional sensitivity of a camera sensor is degraded as the wavelength increases, and this produces darker images. ²⁰ As a result, we use the wavelength of NIR illuminators at 850 nm, which does not cause dazzling. ¹⁶  

Algorithm.

In this section, the pupil detection method, which is a very important procedure for measuring the pupil size, is described. The pupil detection method is performed based on the idea that the pupil is considerably darker than the other regions in the NIR eye image. The method comprises the following steps. ¹⁶  

First, to detect the coarse position of the pupil center, the circular edge detection (CED) method is used; this method is based on the following equation ^14,16,21 :  

 

where, I(x, y) refers to the intensity at the x, y position, and x ₀, y ₀ and r are the center and radius of the circle template, respectively. The pupil boundary is located at the position where integro-differential values between pixels on the inner and outer circle templates are maximized while changing the center positions x ₀, y ₀ and the radius value (r) of the pupil boundary. 

Since the pupil is not a perfect circle, ²² and the specular reflection on the edge between the pupil and iris can cause detection errors, as shown in Figure A2, the radius determined by the CED cannot be used for calculating the pupil's size. Therefore, local thresholding is performed in a square region, whose center is the coarse pupil center obtained by the CED, as shown in Figure A2b. In our research, the size of the local square region is defined as 160 × 160 pixels considering the analysis of the maximum pupil diameter of 8 mm ⁸ and the magnification factor of the camera lens. ¹⁶ However, the local square area can include noise, such as eyelashes or shaded regions, as shown in Figure A2b. To eliminate the noise factors, component labeling and size filtering are adopted in the local thresholding region. ¹⁶ Subsequently, filling of the specular reflections is used to solve the problems caused by the specular reflective regions, as shown in Figure A2b. ^8,16 Consequently, an accurate pupil size is calculated as the number of black pixels of the local square region, as shown in Figure A2c. ¹⁶  

Figure A2.

 

Pupil detection method. (a) Coarse pupil detection by using circular edge detection. (b) After local thresholding. (c) After component labeling, size filtering, and filling of specular reflections.

To represent the pupil size image in millimeter units, the determined pupil size of a pixel unit (n _p) can be used for estimating the pupil diameter in millimeters (d _a), as given by the following equation and Figure A3.  

 

From the manual measurements and camera calibrations, we found that z was 5 cm and λ was 1150 pixels. Based on them, an object of 1 mm across at the Z distance of 5 cm corresponds to 23 pixels in the image. 

Figure A3.

 

Pupil perspective model.