Abstract
Abstract: :
Purpose: The McGurk effect is an audiovisual illusion produced by dubbing a voice saying one sound with a face articulating another. We have found that the occurrence of this effect is not influenced by a subject's natural eye scanning of a talker's face, indicating that the processing of high spatial frequency information afforded by direct oral foveation is not necessary to process visual speech. The purpose of this experiment is to determine the impact of larger gaze displacements away from the talker's face on the processing of visible speech information. Methods: Sixteen subjects observed three talkers centered on a 20" monitor. Subjects' gaze was directed 0, 20, 40, and 60 degrees from the talker's mouth. A total of 540 trials per subject were recorded. Each talker produced 9 stimuli: 5 of which were congruent (an auditory sound recorded with the video of the talker's face articulating the same sound) and 4 that were incongruent (auditory sound different from the visual speech presented by video). The order of gaze fixation positions was counterbalanced across subjects. The subjects were instructed to indicate what they heard through a keyboard interface. A correct response was recorded when the subject heard the auditory stimulus. An incorrect or McGurk effect was recorded when the subject heard a sound different from the auditory stimulus. Results: Data were analyzed with a three-way ANOVA with gaze fixation position, stimulus and talker as factors. For the incongruent stimuli, the effect of gaze fixation position was statistically significant [F(3,33)=195.2, p<0.001], with the percentage of correct responses at each gaze fixation position being significantly different from any other position. For all incongruent stimuli combined, the average percentage of correct responses increased with the eccentricity of the gaze fixation position, from 23.8%, 44.3%, 80.7%, to 88.3%. The eccentricity of the gaze fixation position also affected the average percentage of correct responses in the congruent stimulus condition; these decreased from 99.0%, 96.3%, 95.4%, to 90.8%. Conclusion: We found that gaze eccentricity modulates the influence exerted by visible speech on the perception of acoustic speech. The McGurk effect is reduced when visual speech is processed by extra-macular retina, but not completely abolished. Thus, the McGurk effect is somehow resilient to gaze eccentricity manipulation, and this supports the hypothesis that visual speech information is processed at a coarse-scale level.
Keywords: 511 perimetry • 586 spatial vision • 326 attention