Temporal Ventriloquism Reveals Intact Audiovisual Temporal Integration in Amblyopia

Abstract

Purpose: We have shown previously that amblyopia involves impaired detection of asynchrony between auditory and visual events. To distinguish whether this impairment represents a defect in temporal integration or nonintegrative multisensory processing (e.g., cross-modal matching), we used the temporal ventriloquism effect in which visual temporal order judgment (TOJ) is normally enhanced by a lagging auditory click.

Methods: Participants with amblyopia (n = 9) and normally sighted controls (n = 9) performed a visual TOJ task. Pairs of clicks accompanied the two lights such that the first click preceded the first light, or second click lagged the second light by 100, 200, or 450 ms. Baseline audiovisual synchrony and visual-only conditions also were tested.

Results: Within both groups, just noticeable differences for the visual TOJ task were significantly reduced compared with baseline in the 100- and 200-ms click lag conditions. Within the amblyopia group, poorer stereo acuity and poorer visual acuity in the amblyopic eye were significantly associated with greater enhancement in visual TOJ performance in the 200-ms click lag condition.

Conclusions: Audiovisual temporal integration is intact in amblyopia, as indicated by perceptual enhancement in the temporal ventriloquism effect. Furthermore, poorer stereo acuity and poorer visual acuity in the amblyopic eye are associated with a widened temporal binding window for the effect. These findings suggest that previously reported abnormalities in audiovisual multisensory processing may result from impaired cross-modal matching rather than a diminished capacity for temporal audiovisual integration.

Amblyopia, or “lazy eye,” is a visual disorder that arises from anomalous visual experience during a sensitive period of brain development in early life. It is most commonly caused by misalignment of the eyes (i.e., strabismus), inequality in refractive error between the eyes (i.e., anisometropia), or a combination of the two factors that cause noncorrespondence of the retinal images or visual blur.¹ In rare cases, it also may be induced by congenital cataracts that deprive one or both eyes of form vision. Although visual rehabilitation is possible in childhood, failures in diagnosis and treatment result in amblyopia being the number one cause of persistent monocular blindness in adulthood.^2,3 

Clinically, amblyopia is often regarded as a monocular impairment in low-level visual functions, such as spatial acuity, contrast sensitivity, and binocular vision.⁴ However, a considerable body of research shows that the deficits are not limited to vision in the amblyopic eye (see Meier and Giaschi⁵ for review). The fellow eye also shows subtle deficits in spatial vision,^6,7 contrast sensitivity,^8,9 and motion processing.^10,11 Temporal aspects of visual processing also are affected. Foveal vision in the amblyopic eye is less sensitive to asynchrony between visual elements,¹² extrafoveal regions have poorer temporal resolution,¹³ and the fellow eye in strabismic amblyopia shows impaired perception of temporal order.¹⁴ In agreement with these behavioral deficits in temporal perception, visual evoked responses in humans show increased latency jitter and decreased signal-to-noise ratios.^15,16 Physiological recordings in cats show reduced synchronization among neurons in the striate cortex¹⁷ when driven by stimulation of the amblyopic eye. Beyond vision, amblyopic abnormalities in multisensory processing and integration also are well documented. People with unilateral amblyopia show reduced integration of incongruent auditory and visual speech signals, as demonstrated by the McGurk effect.^18–20 They also show multisensory perceptual binding over an abnormally wide range of signal onset asynchronies (SOAs) for simple visual and auditory signals. For example, in audiovisual simultaneity judgment tasks, the temporal window of perceived simultaneity is broadened on both the visual-lead and auditory-lead sides for people with unilateral amblyopia.^21,22 In the sound-induced flash illusion,²³ a broader temporal window of auditory dominance over vision when the sound precedes the flash also is evident.²⁴ These multisensory perceptual abnormalities are observed under binocular viewing conditions, and so cannot be fully explained by visual anomalies in the amblyopic eye alone. 

It is important to distinguish multisensory integration from other multisensory processes. Multisensory integration involves the fusion or combination of unisensory signals to produce a new response that is significantly different from its component inputs.²⁵ Multisensory processing, on the other hand, is an umbrella term that encompasses multisensory integration and nonintegrative multisensory processes, such as cross-modal matching, that do not produce a new response, but seek featural equivalencies in time, space, or identity between unisensory inputs.²⁵ 

The fused illusory percepts in the McGurk effect and sound-induced flash illusion are examples of multisensory integration because their perceptual products are qualitatively different from their unisensory auditory or visual components.^23,26 A defect in multisensory integration would therefore reduce or eliminate perception of the fused phoneme or illusory flash. This is not the only explanation, however. The strength of integration also is affected by the level of feature matching (e.g., temporal correspondence, spatial correspondence, and phonetic identity) in the unisensory signal streams. For example, degradation of phonetic identity in the visual signal²⁷ can reduce the strength of the McGurk effect, and reduction of cross-modal temporal correspondence (i.e., greater SOAs) can reduce the strength of both the McGurk effect²⁸ and the sound-induced flash illusion.²³ The factors influencing the width of the audiovisual simultaneity window are similarly complex. In healthy individuals, a wider audiovisual simultaneity window correlates empirically with less susceptibility to the McGurk effect (i.e., weaker integration), but greater susceptibility to the sound-induced flash illusion (i.e., greater integration).²⁹ Therefore, the wider audiovisual simultaneity window observed in amblyopia may represent reduced capacity for multisensory integration, or integrative fusion over a wider range of SOAs, or both. Alternatively, a wider audiovisual simultaneity window may represent entirely nonintegrative factors, such as a criterion shift toward simultaneity,³⁰ or temporal uncertainty in the unisensory streams that feed into the neural machinery of multisensory integration.²² Indeed, some empirical data suggest that the width of the audiovisual simultaneity window may not be a function of multisensory integration, but rather of nonintegrative cross-modal matching of temporal features encoded within the unisensory streams.³¹ Finally, because testing paradigms for the McGurk effect, sound-induced flash illusion, and audiovisual simultaneity window typically involve a single interval with a “target present” versus “target not present” response, they are more susceptible to interindividual response bias than traditional 2-alternative forced choice (2AFC) paradigms that have a predictable noise floor. 

The nature of multisensory processing abnormalities in amblyopia remains an unresolved question. Two opposing mechanisms, a primary failure of integration or a primary deficiency in unisensory information, can both lead to the same perceptual outcome for many of the multisensory phenomena studied in amblyopia, as discussed above. To help resolve this ambiguity, we have examined integration in another audiovisual phenomenon: the temporal ventriloquism effect.³² The temporal ventriloquism effect is an example of audiovisual integration in which performance on a 2AFC visual temporal order judgment (TOJ) task is improved by paired auditory events. Specifically, the ability to detect the order of onset of two lights improves when spatially irrelevant clicks are presented such that the second click lags the second light by 100 to 200 ms (Fig. 1).³² In effect, the second click “pulls” or “ventriloquizes” the perceived onset of the second light forward in time, increasing the apparent interval between the two lights and making their temporal order easier to judge. 

Figure 1

 

Schematic of the audiovisual apparatus and audiovisual stimuli used in the experimental protocol. (A) The apparatus consisted of two green LEDs positioned 10 cm above and below a fixation target (red LED illuminated between trials). A central speaker was positioned immediately behind the fixation target. The apparatus was viewed from a distance of 1 m, such that the visual angle between the fixation point and each green LED was 5.7°. (B) General trial structure for audiovisual (AV) stimulus conditions. The SOA indicates the temporal interval between top and bottom LED onset. Illumination of LEDs was accompanied by clicks from the central speaker in “click lead,” “AV sync” (baseline), and “click lag” temporal configurations as shown. Previous work in normally sighted adults has shown that discrimination of visual TOJ is enhanced (i.e., the temporal ventriloquism effect is observed) in “click lag” conditions wherein the second click follows the second light by 100 to 200 ms.³²

Unlike the fused percepts of other audiovisual multisensory phenomena operating in the temporal dimension (e.g., the McGurk effect, audiovisual simultaneity judgment, and the sound-induced flash illusion), the temporal ventriloquism effect results in perceptual enhancement that cannot emerge from nonintegrative cross-modal matching or perceptual blending. In the present report, we used the temporal ventriloquism effect to determine whether the multisensory processing abnormalities observed in amblyopia are related to a primary failure of audiovisual temporal integration. If amblyopia involves a primary failure of audiovisual integration, perceptual enhancement by temporal ventriloquism will not be observed. 

Methods

Participants

Nine adults with unilateral amblyopia (eight female; mean age 28 years; age range 18–47 years) participated in this study, and nine normally sighted adults (seven female; mean age 31 years; age range 22–46 years) served as controls. All participants were naïve to the experimental task, and underwent a standard ocular and hearing screen by a certified orthoptist or an ophthalmologist. The ocular assessment measured habitual refractive correction (automatic lensmeter), distance visual acuity (Early Treatment Diabetic Retinopathy Study [ETDRS] chart with habitual correction), stereo acuity (Randot circles and Titmus fly test), foveal suppression (Worth four-dot test), ocular motility, and eye alignment (prism cover test). All participants reliably detected a standard set of suprathreshold tones (500, 1000, 2000, and 4000 Hz at 25 dBA sound pressure level) presented by a screening audiometer (model MA 27; MAICO Diagnostics, Eden Prairie, MN, USA) with circumaural headphones (model TDH 39; MAICO Diagnostics). Amblyopia was defined as a visual acuity of 0.18 logMAR (20/30) or worse in the affected eye, visual acuity of at least 0.1 logMAR (20/25) in the fellow eye, and an intraocular difference of at least 0.2 logMAR (two lines on the standard ETDRS chart). Anisometropic amblyopia was defined as amblyopia with an interocular difference of at least 1 diopter (D) in either spherical equivalent or astigmatic error. Strabismic amblyopia was defined as amblyopia accompanied by any manifest deviation on the cover test in the absence of significant anisometropia (defined above). Mixed-mechanism amblyopia was defined as amblyopia accompanied by both anisometropia and strabismus of at least 8 prism D. Normal vision was defined as visual acuity of 0.1 logMAR (20/25) or better in each eye, stereo acuity of 40 seconds of arc or better, and no manifest strabismus. Participants were excluded if they had any manifest or latent nystagmus, high ametropia (hyperopia > +5 D or myopia > −6 D), or a history of any neurologic, neurodevelopmental, or auditory disorder; any other ocular pathology; or prior intraocular surgery. 

Participants with amblyopia were recruited from April 2016 to December 2016 through flyers posted on hospital property and through advertisements posted on social media websites (Craigslist.ca and Kijiji.ca). An equal number of normally sighted participants were recruited in a similar fashion. Written informed consent was obtained from all participants. The study protocol was approved by the Research Ethics Board at The Hospital for Sick Children, and followed the tenets of the Declaration of Helsinki. 

The clinical characteristics of the normally sighted control participants were as follows: six right eye dominant; mean ± SD (range) visual acuity −0.07 ± 0.05 (−0.10 to 0.00) logMAR in the dominant eye, −0.04 ± 0.05 (−0.10 to 0.00) logMAR in the nondominant eye; mean ± SD (range) stereo acuity 28 ± 9 (20–40) seconds of arc. The clinical characteristics of the participants with amblyopia are summarized in the Table. 

Table

 

Clinical Characteristics of Participants With Amblyopia

 

Apparatus and Stimuli

The entire experiment was conducted in a carpeted acoustic chamber (internal dimensions 2.0 × 2.1 × 2.2 m) lined with 5 cm acoustic wedge foam (Foam Factory, Macomb, MI, USA) on the walls and ceiling. The audiovisual apparatus (shown in Fig. 1A) consisted of two 4-mm green light-emitting diodes (LEDs) with peak luminance of 100 cd/m² arranged 10 cm above and below a central speaker (model CMS0361KLX; CUI, Inc., Tualatin, OR, USA). A 4-mm red LED positioned over the central speaker served as a fixation target between trials. Auditory stimuli consisted of 2.5-ms square-wave clicks presented at 62 dBA sound pressure level. Relative stimulus signal timing was confirmed with an oscilloscope. Participants were seated with the head stabilized in a chinrest at a standard viewing distance of 1.0 m, and used a wireless gamepad (model F710; Logitech, Newark, CA, USA) to initiate each trial and enter responses. 

Design and Procedure

All trials were conducted in darkness with both eyes open. Each trial began with illumination of the central fixation LED for 500 ms. After a random delay of 500 to 750 ms following offset of the fixation LED, the two green LEDs were illuminated in sequence according to predetermined light SOA conditions. Clicks accompanied the lights according to predetermined click timing conditions. Participants were asked to make a visual TOJ by determining which light appeared last (“top” or “bottom”). Responses were unspeeded, and participants were told that the sounds did not predict the order of the lights. 

Twelve light SOA levels were tested: −144, −96, −72, −48, −36, −24, 24, 36, 48, 72, 96, and 144 ms, with negative values indicating that the bottom LED was illuminated first. Seven click timing conditions were tested for each light SOA: one audiovisual synchrony (“AV sync”) condition, three click “lead” conditions, and three click “lag” conditions (Fig. 1B). In the AV sync condition, clicks were synchronized with the onset of the two LEDs. In the three lead conditions, the first click preceded, or led, the first light by 450, 200, or 100 ms, and the second click and second light were synchronous. In the three lag conditions, the first click and first light were synchronous, but the second click trailed, or lagged, the second light by 100, 200, or 450 ms. A “visual-only” condition in which there were no accompanying clicks was also tested. Twenty practice trials, consisting of a random subset of all audiovisual trials, preceded the start of data collection. Twenty trials were run for each light SOA and click timing condition, yielding a total of 1920 experimental trials. All audiovisual conditions (i.e., AV sync, lead, and lag conditions) were randomly interleaved and run in four blocks. The visual-only condition was run separately in a single block. 

Data Analysis

The just noticeable difference (JND) and point of subjective simultaneity (PSS) for the visual TOJ task were determined for each participant in each of the eight click timing conditions. The JND quantifies the minimum SOA for which the temporal order of the two lights can be reliably determined, and is a measure of visual temporal resolution. The PSS is the SOA at which the perceived onset of the two lights is simultaneous, and typically approximates objective simultaneity (i.e., SOA = 0 seconds).³³ To calculate the JND and PSS, the proportion of trials in which the top LED was seen first was computed for all light SOA levels. A cumulative Gaussian curve was then fit to the psychometric data using a maximum likelihood method. The JND and PSS values were derived from the fitted curve. The JND was defined as the SOA at which the top LED was seen first 75% of the time, minus the SOA at which the top light was seen first 25% of the time, divided by two. The PSS was defined as the SOA at which the top light was seen first 50% of the time. All curve fits and parameters were computed in MATLAB version R2011b (Mathworks, Inc., Natick, MA, USA). 

The main analysis to detect a significant temporal ventriloquism effect consisted of 1-way repeated-measures ANOVAs (one for each group) comparing JNDs across the lead and lag click timing conditions. The control group data were not intended for between-group comparisons with the amblyopia group, but served to validate the adequacy of the apparatus to elicit the temporal ventriloquism effect. 

Exploratory subgroup analyses were conducted on the basis of stereo acuity and visual acuity in the amblyopic eye. Residual stereo acuity of ≤400 seconds of arc (i.e., some or all Randot circles) was classified as “fine” (n = 5), and stereo acuity of ≥3000 seconds of arc (i.e., no Randot circles, Titmus fly only, or negative stereopsis) was classified as “poor” (n = 4). Amblyopia with a monocular acuity of ≤0.6 logMAR (≤ 0/80) was classified as “mild/moderate” (n = 7), and amblyopia with a monocular acuity of >0.6 logMAR (>20/80) was classified as “severe” (n = 2).⁴ 

All statistical tests were computed in IBM SPSS Statistics, version 22 (Armonk, NY, USA). Homogeneity of variance was established by Levene's test for independent samples t-tests, and by Mauchly's test of sphericity for repeated-measures ANOVAs. Holm-Bonferroni adjustments were applied to post hoc multiple comparisons where indicated in the results. Statistical significance was defined as α = 0.05. 

Power Analyses

Power analyses were computed using G*Power 3.1.³⁴ The study by Morein-Zamir et al.,³² on which this experiment was based, did not offer an estimate of effect size. However, a related study by Parise and Spence³⁵ reported that within-subject change in JND for a variation of the temporal ventriloquism paradigm had an effect size of η² = 0.432 (equivalent to Cohen's f = 0.87, as required by G*Power 3.1).³⁶ Based on this estimate, a sample size of three is needed to provide 80% power to detect a significant temporal ventriloquism effect. For the exploratory subgroup analyses on the basis of stereo acuity and visual acuity in the amblyopic eye, the study had 80% power to detect effect sizes of d = 2.2 and d = 2.6, respectively. 

Results

Main Analysis

Variation in JNDs on the visual TOJ task across click timing conditions is illustrated in Figure 2. Individual JND values for each click timing condition were submitted to a 1-way repeated-measures ANOVA for each group. In the control group, there was a significant effect of click timing condition on JND (F_6,48 = 4.172, P = 0.002, η²_G = 0.27). Holm-Bonferroni multiple comparisons showed that JNDs were significantly reduced compared with baseline (i.e., AV sync condition) in the control group when the second click lagged the second light by 100 ms (mean JND reduction = 16 ms [26%], P < 0.001) and 200 ms (mean JND reduction = 9 ms [15%], P = 0.027), thus confirming that the apparatus can replicate the temporal ventriloquism effect in normally sighted control participants. In the amblyopia group, there was a significant effect of click timing condition on JND (F_6,48 = 4.862, P < 0.001, η²_G = 0.34), similar to that observed in the control group. Holm-Bonferroni multiple comparisons showed that JNDs were significantly reduced compared with baseline in the amblyopia group when the second click lagged the second light by 100 ms (mean JND reduction = 15 ms [24%], P < 0.001) and 200 ms (mean JND reduction = 14 ms [22%], P = 0.025), a response consistent with the temporal ventriloquism effect as previously described.³² 

Figure 2

 

The temporal ventriloquism effect. (A) Control group data (n = 9); (B) amblyopia group data (n = 9). Bars represent mean JNDs ± 95% confidence interval (CI). “Lead” conditions (leftmost three bars in each group) are those in which the first click preceded the onset of the first light, followed by a synchronous second click and second light. “Lag” conditions (rightmost three bars in each group) are those in which the first click and first light were synchronous, but the second click trailed the second light. The central black bar (“AV sync”) and horizontal dashed line in each group represent the baseline audiovisual synchrony condition against which the other click timing conditions were compared. A significant temporal ventriloquism effect was observed in the 100- and 200-ms click lag conditions for the control group and the amblyopia group (*P < 0.05).

To examine the possibility that performance in the baseline condition reflects interference from the accompanying clicks, JNDs in the baseline (AV sync) condition were compared with JNDs in a unimodal (visual-only) condition within each group. Paired samples t-tests showed no significant differences in mean JND between the visual-only condition and AV sync condition for the control group (t₈ = 1.752, P = 0.118, d = 0.58) or the amblyopia group (t₈ = −0.330, P = 0.750, d = 0.11). 

Within each group, the mean PSS value for each stimulus condition was compared with objective simultaneity (i.e., SOA of 0 ms between the top and bottom LEDs) using a one-sample t-test. No stimulus condition in either the control group or the amblyopia group showed significant deviation in the mean PSS from the expected value of 0 ms. 

Subgroup Analysis

Relations between susceptibility to the temporal ventriloquism effect and clinical measures of amblyopia were further explored for the two click lag conditions that showed significant JND reductions in the main analysis presented above. Individual values for JND reduction from baseline were computed by subtracting the JND in the baseline AV sync condition from the JND in each click lag condition. The results of the subgroup analysis by stereo acuity are illustrated in Figure 3A. In the 100-ms click lag condition, the mean JND reduction did not differ significantly between the two stereo acuity subgroups (t₇ = 0.435, P = 0.667, d = 0.29), but in the 200-ms click lag condition, the mean JND reduction was greater in the “poor” stereo acuity subgroup compared with the “fine” stereo acuity subgroup (t₇ = 3.919, P = 0.006, d = 2.56). Results of the subgroup analysis by visual acuity are illustrated in Figure 3B. In the 100-ms click lag condition, the mean JND reduction did not differ significantly between the two visual acuity subgroups (t₇ = 0.079, P = 0.939, d = 0.06), but in the 200-ms click lag condition, the mean JND reduction was greater in the “mild/moderate” amblyopia subgroup compared with the “severe” amblyopia subgroup (t₇ = 3.095, P = 0.017, d = 2.36). 

Figure 3

 

Subgroup analysis for susceptibility to the temporal ventriloquism effect in participants with amblyopia. (A) Comparison by residual stereo acuity. (B) Comparison by visual acuity deficit in the amblyopic eye. Susceptibility was defined as the reduction in JND from each participant's baseline performance in the “AV sync” condition. Open symbols represent individual participant data. Filled symbols represent group means ± 95% CI. The dotted line indicates normally sighted control performance and is included for reference. In the 200-ms click lag condition, poor stereo acuity and severe visual acuity deficit in the amblyopic eye were associated with significantly greater susceptibility to the temporal ventriloquism effect (*P < 0.05).

Discussion

We characterized and compared the effect of spatially irrelevant sounds on performance in a visual TOJ task for participants with unilateral amblyopia and normally sighted controls under binocular viewing conditions. Both the amblyopia and control groups showed a significant improvement in visual temporal precision, as measured by the JND, when the second click lagged the onset of the second light by 100 to 200 ms, consistent with the temporal ventriloquism effect described previously.³² This finding suggests that the capacity for audiovisual integration in the temporal domain remains intact in amblyopia. By extension, it lends support to the hypothesis that failed integration is not the primary source of the multisensory processing abnormalities observed in amblyopia. In the amblyopia group, we also found that poorer stereo acuity and poorer visual acuity in the amblyopic eye were associated with greater susceptibility to the temporal ventriloquism effect (i.e., greater JND reduction from baseline) when the second click lagged the onset of the second light by 200 ms. These findings suggest that the temporal binding window for the effect may be widened in people with greater deficits in stereo acuity and visual acuity. The subgroup analyses must be interpreted with caution, however, as the sample sizes are small. 

A common factor that modulates the temporal ventriloquism effect and many other multisensory phenomena (such as audiovisual simultaneity perception, the McGurk effect, and the sound-induced flash illusion) is a dependency on cross-modal temporal correspondence and asynchrony detection.^23,28,29,32 Previous work has shown that unilateral amblyopia is associated with symmetric widening of the temporal window of audiovisual simultaneity perception^21,22 and reduced susceptibility to the McGurk effect under both monocular and binocular viewing conditions.^18–20 In addition, a study of the sound-induced flash illusion in amblyopia suggested that the temporal binding window for the illusion is extended under binocular conditions when the clicks lead the flash.²⁴ How does an intact temporal ventriloquism effect with a possibly extended temporal binding window fit in the context of these prior findings? Insight comes from the work by Stevenson et al.,²⁹ who described the correlations of various indices of multisensory function in a sample of normally sighted adults. They found that the width of the audiovisual simultaneity window was negatively correlated with susceptibility to the McGurk effect, but positively correlated with susceptibility to the sound-induced flash illusion. They proposed that a narrower audiovisual simultaneity window relates directly to a superior ability to dissociate, or resolve, asynchronous unisensory components of an audiovisual stimulus pair. Because temporal correspondence is a constraint on multisensory perceptual binding, any change in the sensitivity to audiovisual asynchrony will necessarily alter the likelihood of audiovisual integration. In the case of the McGurk effect, heightened sensitivity to asynchrony means that auditory and visual stimuli perceived as synchronous are more unique, more likely to have arisen from a single event, and therefore more strongly integrated in a fused percept. In the case of the sound-induced flash illusion, diminished sensitivity to asynchrony means that the temporal constraints on integration are looser. In turn, the asynchrony inherent in the sound-induced flash illusion stimulus poses less of an impediment to integration, and therefore susceptibility to the illusory percept is increased. In their study, Stevenson et al.²⁹ did not explicitly test the width of the temporal binding window for the sound-induced flash illusion, but based on their reasoning (outlined above), one might expect perceptual binding and an illusory percept over a wider range of audiovisual SOAs; that is, a wider temporal binding window, as was previously observed in amblyopia.²⁴ Like the sound-induced flash illusion, the temporal ventriloquism effect is also dependent on perceptual binding of asynchronous auditory and visual signals in the temporal dimension. Therefore, reduced sensitivity to audiovisual asynchrony, as evidenced by a widened simultaneity window,^21,22 would likely not diminish susceptibility to the temporal ventriloquism effect, but enable integration over a wider range of SOAs. Indeed, a widened audiovisual temporal binding window in amblyopia is suggested by the significant relation between susceptibility to the temporal ventriloquism effect and the extent of the stereo acuity and visual acuity deficits (Figs. 3A, 3B). 

Taken together, the profile of multisensory processing abnormalities suggests that amblyopia involves reduced temporal resolution in unisensory perception or in the mechanism for cross-modal matching (i.e., nonintegrative comparison of unisensory features) rather than a primary deficit in audiovisual integration. Several sources of evidence point to an amblyopic deficit in temporal resolution residing within vision rather than audition. Most obviously, amblyopia is a primary disorder of the visual system, and its causative factors are ones that interfere with normal visual experience. Behaviorally, deficits in temporal processing have been demonstrated in the amblyopic and fellow eye.^12–14 Physiologically, cortical responses driven by stimulation of the amblyopic eye are less synchronized,¹⁷ and their temporal encoding is less reliable.^15,16 Studies of normally sighted people also demonstrate that audition is more temporally precise than vision,³⁷ and tends to be dominant in processing the temporal dimension of audiovisual events.^23,38–40 Given the normal dominance of audition in temporal audiovisual processing, any amblyopic deficit in auditory temporal resolution would likely have diminished the magnitude of the temporal ventriloquism effect and the sound-induced flash illusion, yet no such diminution was observed in the present study or previously.²⁴ Finally, the width of the audiovisual simultaneity window and the width of the temporal binding window for temporal ventriloquism vary with the extent of amblyopic deficits in stereo acuity and visual acuity, respectively.²² Although association does not equal causation, the relation is compelling. 

Reduced temporal resolution in amblyopic vision may arise from noisy encoding of the visual signal. Indeed, increased temporal jitter^15,16 and increased latency^41,42 in cortical signals from the amblyopic eye may alter the perceived timing of a visual event such that the probability distribution for the estimate of visual event timing is widened and shifted toward later onset. Miscalibration of visual temporal reliability, induced by amblyopic visual input and crystallized during development, may therefore provide an explanation for the weaker temporal constraints (i.e., wider temporal window) for audiovisual perceptual binding documented in the present study and in prior work on amblyopic multisensory perception.^21,22,24 

Alternatively, the temporal resolution deficit relevant to abnormal audiovisual processing in amblyopia may not lie within unisensory visual perception, but instead at the interface between auditory and visual temporal perception, at the level of cross-modal matching of temporal features. This view is supported by prior work showing that sensitivity to audiovisual asynchrony detection is equally impaired whether viewing with the amblyopic eye, the fellow eye, or with both eyes together,^21,22 and by evidence from normally sighted adults indicating that detection of audiovisual asynchrony is based on matching, rather than integration, of temporal features encoded within the unisensory streams.³¹ In amblyopia, the neural mechanism for cross-modal matching may be miscalibrated in early life under the influence of increased temporal jitter^15,17 and interocular latency differences.^41,42 If audiovisual cross-modal matching is calibrated before the end of the sensitive period for amblyopic visual recovery,⁴³ then therapy that improves vision, equalizes evoked response latencies,^44,45 and reduces internal temporal noise¹⁶ may not narrow the audiovisual temporal binding window. Such asynchronous sensitive periods for unisensory and multisensory functions may therefore explain how cross-modal matching may be selectively impaired, and why the deficit persists under binocular viewing conditions that provide high-fidelity temporal information from the fellow eye. 

Acknowledgments

Supported by grant MOP 106663 from the Canadian Institutes of Health Research, Leaders Opportunity Fund from the Canada Foundation for Innovation, the John and Melinda Thompson Endowment Fund in Vision Neurosciences at the Hospital for Sick Children, the Department of Ophthalmology and Vision Sciences at The Hospital for Sick Children, and the Vision Science Research Program at the University of Toronto. 

Disclosure: M.D. Richards, None; H.C. Goltz, None; A.M.F. Wong, None