Visual motion aftereffects arise from a cascade of two isomorphic adaptation mechanisms

Abstract

Prolonged exposure to a moving stimulus can substantially alter the perceived velocity (both speed and direction) of subsequently presented stimuli. Here, we show that these changes can be parsimoniously explained with a model that combines the effects of two isomorphic adaptation mechanisms, one nondirectional and one directional. Each produces a pattern of velocity biases that serves as an observable “signature” of the corresponding mechanism. The net effect on perceived velocity is a superposition of these two signatures. By examining human velocity judgments in the context of different adaptor velocities, we are able to separate these two signatures. The model fits the data well, successfully predicts subjects' behavior in an additional experiment using a nondirectional adaptor, and is in agreement with a variety of previous experimental results. As such, the model provides a unifying explanation for the diversity of motion aftereffects.

Introduction

Our visual system continually adapts to the recent history of visual input. Although we are generally not aware of it, this adaptation causes substantial changes in our perception. A prominent effect is that of repulsion: Adaptation exaggerates the perceived differences between stimuli and the adaptor. For example, in the well-known tilt aftereffect, adaptation to a vertical bar causes the orientation of a subsequently viewed near-vertical bar to appear further from vertical (Gibson, 1937). These changes, extensively documented in the experimental literature, provide a window into the properties of the underlying neural representations. The simplest and most accepted linking hypothesis is based on tuned populations of cells and gain changes (Blakemore, Nachmias, & Sutton, 1970; Sekuler & Pantle, 1967). Specifically, those neurons (or, more abstractly, “channels”) that are activated by the adaptor reduce their gain, which leads to a characteristic repulsive bias in the perception of subsequently viewed stimuli (Clifford, Wenderoth, & Spehkar, 2000; Maffei, Fiorentini, & Bisti, 1973; Schwartz, Hsu, & Dayan, 2007). Additional physiological changes have been observed (e.g., changes in the shape of tuning curves), which, by themselves, may lead to different effects, but altogether the net effect seems to be consistent with repulsive perceptual aftereffects (Jin, Dragoi, Sur, & Seung, 2005; Kohn, 2007; Schwartz et al., 2007; Seriès, Stocker, & Simoncelli, in press). As such, regardless of the physiological details, we can think of the repulsive pattern of adaptation-induced biases as providing an externally observable perceptual “signature” for the underlying adaptation mechanism. 

A modality in which adaptation has been particularly well studied is that of motion. Prolonged exposure to a moving stimulus leads to perceived illusory motion of static stimuli (Nishida & Johnston, 1999; Wright & Johnston, 1985; often referred to as the “waterfall illusion”, Thompson, 1880), illusory motion of dynamic random stimuli (see, e.g., Hiris & Blake, 1992), biases in the perceived velocity of moving stimuli (Clifford & Wenderoth, 1999; Ledgeway & Smith, 1997; Mather, Verstraten, & Anstis, 1998; Schrater & Simoncelli, 1998; Smith, 1985; Smith & Edgar, 1994; Thompson, 1981), and changes in speed or direction discriminability (Clifford & Wenderoth, 1999; Kristjansson, 2001; Phinney, Bowd, & Patterson, 1997). Although these results all arise from adaptation to moving stimuli, the individual studies use stimuli with different spatial structures (e.g., dots, bars, gratings), different choices for the velocities of adapting and test stimuli (especially with regard to moving versus stationary stimuli), and typically manipulate either speed or direction individually, rather than the joint two-dimensional quantity of velocity (with the exception of Schrater & Simoncelli, 1998). Furthermore, unlike adaptation-induced biases seen in other visual stimulus parameters, speed biases are asymmetric around the adaptor. As such, the published results are difficult to reconcile, and to date, there is no consistent and unified explanation for the observed motion aftereffects. 

Here, we provide a characterization of the perceptual biases arising from motion adaptation over a broad range of test and adaptor velocities. We find that the resulting perceptual changes can be parsimoniously and accurately fit using not one but a superposition of two signatures, corresponding to two distinct underlying mechanisms—one directional and one nondirectional—thus providing a unified explanation for previously noted differences in the effects of adaptation on speed and direction. 

Methods

Psychophysical experiments

Seven subjects with normal or corrected-to-normal vision participated in the psychophysical experiments. Protocols for selection of subjects and experimental procedures were approved by the human subjects committee of New York University and all subjects signed an approved consent form. All but one of the subjects (number 1) were unfamiliar with the purpose of the study. Stimuli were high contrast (0.8) spatial broadband drifting gratings (frequency range from 1/3 cycles deg ⁻¹ to 2 cycles deg ⁻¹) with phases randomized on each trial and spatial power spectrum falling as f ⁻² (see examples in Figure 1), identical to those used in one of our previous studies (Stocker & Simoncelli, 2006a). The mean luminance of gratings and background was held constant at 38 cd m⁻². Stimuli were presented on a CRT monitor running at 120-Hz refresh rate. We used the Psychophysics Toolbox for the controlled timing of the display (Brainard, 1997; Pelli, 1997). Stimulus size and eccentricity were as indicated in Figure 1a. Subjects' heads were stabilized using a chin rest. Each subject performed a primary, a control, and a validation adaptation experiment in a two-alternative-forced-choice (2AFC) paradigm (except subject 4, who did not complete the validation experiment). They were asked to fixate a central fixation mark while an adaptor stimulus was presented to the left of the fixation mark for 40 s. After this initial adaptation phase, a stimulus pair consisting of test (to the left of fixation, in the same location as the adaptor) and reference (to the right of fixation) was presented for 0.5 s, followed by a blank period of ∼1.25 s in which the subject had to indicate which stimulus was moving faster by pushing an appropriate key. After a top-up period in which the original adaptation stimuli was presented again for 5 s, the next reference/test pair was presented. This trial sequence was identical for all experiments and is summarized in Figure 1c. The data were fit with a psychometric function (cumulative Gaussian) that characterizes the probability of a subject deciding that the reference is faster than the test, as a function of the reference velocity. 

Figure 1

 

Methods. (a) Test stimuli and experimental setup. (b) Different adaptation conditions. (c) Timing diagram. (d) Two-dimensional velocity space and the locations of the different adaptors therein.

Primary experiment

The primary adaptation experiment was comprised of four separate temporal blocks, corresponding to adaptor velocities v _a = [0.0, 0.8, 4.1, 8.5] deg s ⁻¹. A period of at least 24 h was provided between blocks to allow complete recovery from the adaptation effects of previous blocks. Each block contained trials for 12 conditions, corresponding to test stimulus velocities of v _t = ±[0.8, 2.4, 4.1, 6.1, 8.5, 12] where negative velocity indicates leftward and positive indicates rightward motion (note that subject 1 was not tested at the highest speed). The reference stimulus had a variable speed v _r that was adjusted according to two interleaved adaptive staircase procedures, each starting from one end of the adaptive speed range of each condition. Staircases were of the type “one-up/one-down”. Each test condition contained 60 trials, which were used to estimate a psychometric function for that condition. Individual trials for different test conditions were randomly interleaved. 

Control experiment

The control adaptation experiment contained a single block with an adaptor stimulus that was identical in spatial frequency spectrum and contrast to the other adaptation stimuli but spatially rotated by 90 degrees and stationary (see Figures 1b and 1d). This control adaptor is completely ambiguous with regard to velocity along its grating orientation (the “aperture problem”) and, as such, does not produce motion aftereffects in that direction. However, it retains all the other characteristics (e.g., contrast and temporal and spatial frequencies) of the adaptation stimuli used in the primary experiment. Thus it serves as a control to account for all possible nonmotion adaptation effects, such as apparent loss of contrast (Bex, Bedingham, & Hammett, 1999), as well as any potential spatial asymmetry between perception at the reference and test locations. The procedural sequence of this block was identical to that of the main experiment. All perceptual biases in the primary experiment were computed relative to the biases measured in this control experiment. 

Validation experiment

The validation adaptation experiment was identical to the control experiment, with the exception that the rotated adaptor stimulus was not static but drifting upward with speed v _a = 8.5 deg s ⁻¹ (see Figures 1b and 1d). 

Perceptual space

Throughout this paper we are using a transformed representation of stimulus motion. We (Stocker & Simoncelli, 2006a) and others (Nover, Anderson, & DeAngelis, 2005) have shown that a compressive nonlinear transformation, such as a normalized logarithm, allows a more homogeneous interpretation of both physiological and psychophysical data. Here, we used a circular representation of speed

$\tilde{v}$

= 2 arctan(v/r₀). This representation is very similar to the normalized logarithm with the difference that it is a closed (compact) space. For the findings presented here, the choice is not critical and is also relatively insensitive to the exact value of the parameter r₀ (here, r₀ = 5). However, we believe that a closed velocity space has some interesting implications for perception, which are beyond the scope of this paper. 

Results

We adopt the traditional hypothesis that prolonged exposure to a single stimulus leads to a gain reduction in sensitivity of those sensory channels that are selectively responsive to the stimulus, and the strength of this gain reduction is largest for channels that respond most strongly (Blakemore et al., 1970; see Figure 2a). A simple population average scheme for decoding channel activity leads to repulsive biases as shown in Figure 2b (see Blakemore et al., 1970; Clifford et al., 2000; Schwartz et al., 2007). The precise form of the bias curve depends on many details: the number of channels, their tuning, their noise properties, the relationship between gain reduction and response level to the adaptor, and the decoding method. However, these details are not of critical importance in the present context: we assume only that the bias curve takes on a characteristic repulsive form, which is anti-symmetric about the adaptor, and of limited extent. Aftereffects in many visual modalities (e.g., orientation, Gibson, 1937; spatial frequency, Blakemore et al., 1970) exhibit this behavior. As such, we interpret this repulsive bias as an externally visible manifestation of a single underlying physiological mechanism, a so-called signature of adaptation (see Supplementary material for an extension of this signature that includes also changes in the variability of the percept). 

Figure 2

 

Signature of adaptation. (a) A generic mechanism for sensory adaptation: the response gain of channels that are selectively activated by the adaptor is reduced. The reduction is largest for channels that respond most. (b) Decoding such channel activity by taking the population average leads to a stereotypical signature in subsequent perception: the perceived value of stimulus v is repulsively biased away from the value of the adaptor, v_a. We define bias as the difference between the mean of perceived stimulus value v, before and after adaptation.

Psychophysical measurements

In order to examine the signature of motion adaptation, we measured the effect of adaptation on subsequent perception of visual velocity as described above. From the fit psychometric functions, we extracted perceptual biases, defined as the difference between mean perceived speed (point-of-subjective equality) under adapted and control conditions, which revealed a consistent pattern across all seven subjects. Three examples are shown in Figure 3, corresponding to those subjects that had complete data sets with the least trial variability. 

Figure 3

 

Perceptual estimation bias due to adaptation. Each row shows the perceptual biases of one subject, for each of four adaptation conditions, with adaptor velocities indicated by arrows and dashed vertical lines. Hollow points represent measured shifts in the point of subjective equality with respect to the control condition. Error bars represent the 5–95% quantiles of extracted biases, based on 1000 bootstrap samples of the data. Red bold lines indicate the bias of the two-mechanism model, fitted separately for each subject, using a total of 9 free parameters (see also Figure 4). Both axes represent nonlinearly transformed velocities (see Methods section), with horizontal and vertical distances scaled identically. Note that bias is a signed velocity difference; thus positive bias for negative test velocities indicates reduced perceived speed, while positive bias for positive test velocities indicates increased perceived speed. All subjects show similar behavior, with adaptation affecting subsequent perception over the whole range of test velocities, including test stimuli moving in the opposite direction.

For test stimuli moving in the same direction as the adaptor, we find that perceived speeds are typically reduced for test speeds equal to or smaller than the adaptor and are increased for higher speeds. This is in agreement with previously published motion adaptation results (see, e.g., Bex et al., 1999; Smith, 1985). However, we also find significant effects for test stimuli moving in the direction opposite to that of the adaptor, which have not been systematically examined in previous literature. For slow adaptor speeds, the perceived speed of test stimuli moving in the opposite direction is typically unchanged or slightly increased. However, as the adaptor speed increases, the perceived speed of opposite-moving tests is reduced. Bias amplitudes for test stimuli moving in either direction are seen to increase with adaptor speed. Finally, some subjects also show small but significant biases around a stationary adaptor, as has been previously reported (Ascher, Welch, & Festa, 1996). 

Two isomorphic mechanisms account for motion aftereffects

The perceptual biases shown in Figure 3 clearly do not match the signature of Figure 2b. For all nonzero adaptor speeds, the bias is typically not zero at the location of the adaptor, and more generally, it is not symmetric around the adaptor. Perhaps this is not surprising. Visual motion is computed through a somewhat elaborate hierarchical spatio-temporal processing cascade in visual cortex. It seems plausible that adaptation could occur at several levels along this processing cascade, ¹ and thus the observed perceptual changes may reflect the joint effects of multiple mechanisms (Langley & Anderson, 2007; Mather & Harris, 1998). Here, we propose that the observed adaptation effects represent a superposition of two signatures (as shown in Figure 2b) arising from a directional and a nondirectional mechanism. 

To examine our hypothesis, we fit the full data set (i.e., the changes in mean perceived velocity for all test speeds, and under all four primary adaptation conditions) with a single model consisting of a linear combination of two adaptation signatures, as illustrated in Figure 4a. We chose single-parameter functions (first derivatives of Gaussians of width s _D and s _ND, respectively) for modeling each signature of perceptual bias ( Figure 2). We have also examined other function choices (with more parameters), which, however, did not significantly alter the main results presented here. An implicit feature of a nondirectional mechanism is its symmetry about zero motion. As a consequence, we assumed that the nondirectional signature was always centered at zero motion while the directional signature was centered at the adaptor. In addition, we constrained the widths of both signatures to be fixed across all adaptation conditions while the amplitudes were allowed to vary. Fits were computed by minimizing the total squared error. 

Figure 4

 

Superposition of a directional and a nondirectional signature. (a) Perceptual bias is modeled as a weighted sum of two signatures ( Figure 2b): a nondirectional signature centered at zero motion and a directional signature centered at the adaptor. For each subject, we jointly fit the measured estimation bias gathered under all adaptation conditions. We constrained the signature widths s _D and s _ND (standard deviation of the Gaussians) to be constant across all adaptation conditions, while the fitted amplitudes w _D ⁱ and w _ND ⁱ were allowed to change with adaptor velocity v _a ⁱ. Fits are shown as bold red lines in Figure 3. Note that for the nondirectional signature, the fitted weights were predominantly negative, which flips the polarity of the signature. (b) The estimated signature widths for each of the three sample subjects and the averaged subject (data of all seven subjects) suggest that the directional mechanism is constructed from more narrowly tuned channels than the nondirectional mechanism. (c) For the tested velocity range, the amplitude of the directional signature (circles) is typically positive and monotonically increasing with adaptor velocity v _a, while the amplitudes of the nondirectional signature (squares) become increasingly negative with increasing adaptation speed. For v _a = 0, the amplitude of the nondirectional signature was set to zero because the control and the stationary adaptor stimuli are activating the nondirectional mechanism equally. Bold lines (red and blue) represent linear fits to the estimated amplitude weights that are used to compute the predictions shown in Figure 7. Error bars indicate the 5–95% quantiles over 1000 bootstrap samples of the data.

As shown in Figure 3 the model provides a good account of the data. Figures 4b and 4c show the estimated model parameters for the three sample subjects and the average subject computed over the data of all seven subjects. The width of the nondirectional signature ( s _ND) is approximately twice as large as the directional one ( s _D). The amplitude of the directional signature ( w _D) increases monotonically with adaptor speed, saturating for some subjects. The amplitude of the nondirectional signature ( w _ND) becomes increasingly negative at higher adaptor speeds, indicating an increasingly attractive bias toward zero motion. In the Discussion section, we show that the fit superposition of signatures is consistent with a more mechanistic model based on a sequential cascade of two different populations of motion processing channels. Note that for a zero motion adaptor, the signatures are both centered at zero, and aside from their different widths, there is little to distinguish their relative contributions. Because in this case both the control adaptor and the zero-motion adaptor presumably produce equal activation (and adaptation) of the nondirectional mechanism, we assume that the measured net bias (adapted minus control) is entirely due to the directional signature, and thus the weight of the nondirectional signature is zero. Note that this assumption is specific to this particular control condition: adaptation effects in the nondirectional mechanism could be revealed under different control conditions. 

Model comparison

We compare the fits of our two-mechanism model to those obtained for each of these mechanisms in isolation. The signatures of the two-mechanism model were constrained to have a fixed shape across all four adaptors (resulting in a total of 9 free parameters), but we allowed each of the partial models the flexibility to fit each condition separately, in order to give them full opportunity to explain the data. We also compared to a model proposed by Hammett, Champion, Morland, and Thompson (2005), which accounts for adaptation biases in speed estimates based on a comparison of two physiologically inspired channels. The four free parameters of this model were also optimized to fit the data of each subject. 

Figure 5a shows the performance of the different models for all seven subjects, indicated as mean squared error relative to the error of the two-mechanism model. The two-mechanism model performs best for each individual subject, which is also true when we account for the different number of degrees of freedom by performing an information criterion analysis (not shown). While each of the two partial models (directional/nondirectional) might provide acceptable fits for some of the subjects, they each fail to account for the whole range of subjects' behavior, a fact that can be readily understood by examining the fits for a single adaptation condition, as shown in Figure 5b. The figure illustrates that the directional mechanism cannot account for bias effects at the adaptor and for velocities in opposite directions, while the nondirectional mechanism alone is not capable of accounting for the asymmetry across directions. The same drawback is seen in the model proposed by Hammett et al., which is also nondirectional. Moreover, the Hammett model produces a bias curve that is discontinuous at the origin (the model output is not defined for stationary stimuli). 

Figure 5

 

Goodness of fit and comparison to alternative, single-mechanism models. (a) Mean squared error of fitted alternative models, relative to the error of our proposed two-mechanism model. Error bars indicate the 5–95% quantiles for 1000 bootstrap samples of the data. Across all subjects, the two-mechanism model consistently outperforms the alternative models. (b) Fitted models and their fitting error, illustrated for a single subject and adaptation condition (subject 1, v _a = 4.1 deg s ⁻¹). Note that the model by Hammett et al. (2005) is not defined at zero velocity.

Model validation

As a means of validating the model, we developed and ran an experiment with an adaptor designed to isolate the nondirectional mechanism. We used the same adaptor stimulus as in the control condition (grating rotated by 90 degrees relative to the original control adaptor) but drifting upward with speed equal to the fastest adaptor used in the primary adaptation experiment (see Figures 1b and 1d). According to our model hypothesis, this stimulus should drive the nondirectional mechanism as strongly as its 90-degree rotated counterpart from the original experiment. However, it should not engage the directional mechanism in the (horizontal) test direction, since its motion is completely ambiguous along this direction (aperture problem). 

We tested each subject using this nondirectional adaptor, and then compared the resulting biases to those predicted by the nondirectional signature whose parameters were extracted from the primary experiment ( Figure 4). As Figure 6 shows, the predictions are reasonably well matched to the measured biases of all three subjects shown in Figure 3, supporting the two-mechanism model. ² 

Figure 6

 

Predictions of the model for a nondirectional adaptor and experimental validation. Circles indicate measured biases after adaptation to the vertically drifting adaptor. The model predicts that this adaptor only engages the nondirectional mechanism. Bold lines represent the model prediction for the three subjects based on their individually estimated model parameters from the primary adaptation condition. The measured biases confirm the model prediction. In particular, they are all anti-symmetric about zero motion, as expected for the signature of a nondirectional mechanism. For subject 2, the prediction slightly underestimates the effect at high speeds.

Predictions

The fitted model can be used to make a number of predictions. First, we can predict the classical static motion aftereffect (sMAE) results for our subjects. We performed a linear fit to the estimated model parameters (shown in Figure 4c) in order to continuously predict the perceived speed (in the opposite direction) of a static test stimulus after adaptation, i.e., the static motion aftereffect, as well as the speed of a “nulling” test stimulus (at which the test stimulus appears stationary after adaptation). The results are shown in Figure 7a. Both the shape and the peak speed of the predicted sMAE are similar for our subjects and remarkably consistent with previously reported data, despite substantial differences in stimulus details (adaptation duration, eccentricity, spatial frequency, etc.; Nishida & Johnston, 1999; Wright & Johnston, 1985). A key property of the model that allows prediction of the sMAE is the fact that the signatures are continuously and smoothly defined over the entire velocity space, including zero motion. Perhaps more importantly, the good qualitative match to the experimental data demonstrates that effects observed for zero motion (stationary) adapting stimuli can be viewed as consistent with those observed for moving adapting stimuli, thus suggesting a continuous perceptual representation of velocity that includes zero. 

Figure 7

 

Predictions derived from estimated model parameters of individual subjects. (a) Classical motion aftereffects as a function of the adaptor speed. Solid lines indicate predicted perceived speed of a stationary test stimulus (in the direction opposite to the adaptor), as a function of adaptor speed. Dashed lines indicate the speed of a test stimulus (moving in the same direction as the adaptor) that “nulls” the aftereffect (i.e., makes the test stimulus appear to have no net motion). Inset shows measured nulling speeds for two subjects as reported by Wright and Johnston (1985; data replotted from their Figure 5). (b) Predictions of relative bias at the adaptor, as a function of adaptor speed. (c) Predictions of bias for a 6 deg s⁻¹ test stimulus after exposure to a nondirectional adaptation stimulus. Note that all of the above bias predictions are plotted relative to the control adaptation condition.

Next, we can predict biases for test stimuli at the adaptor, as a function of adaptor velocity ( Figure 7b). Because the directional signature is zero at the adaptor, biases for test stimuli moving at the same velocity as the adaptor are determined solely by the nondirectional signature. Thus, the model predicts that perceived speed at the adaptor is always reduced (Thompson, 1981). However, as predicted by the shape of the nondirectional signature the magnitude of this decrease is largest for intermediate adaptor speeds and fades for very low or high adaptor speeds. 

The fact that perceived bias at the adaptor reflects only the nondirectional mechanism leads to a related set of interesting predictions. In particular, the aftereffects observed for test stimuli with the same velocity as the adaptor should be observable for a variety of other adapting stimuli, including some that are nondirectional. For example, when using sine-wave gratings of a fixed spatial frequency, a counter-phase grating adaptor should produce the same aftereffect at the adaptor as the grating adaptor. This has been verified by Clifford and Wenderoth (1999). 

Last, and more generally, we can predict the bias for any fixed directional test stimulus velocity (here, 6 deg/s) after adaptation to a nondirectional adaptor, as a function of equivalent adaptor speed (see Figure 7c). We do not rule out the possibility that the biases in this case can be positive at very low adaptor speeds, when compared against a different control condition, i.e., when not constraining the amplitude of the nondirectional signature to be zero for a zero velocity adaptor. 

Discussion

We have shown that the perceptual effects of adaptation to stimuli moving at a wide range of velocities (including zero motion) can be explained as a superposition of effects arising from two isomorphic adaptation mechanisms, one of which is directional and the other nondirectional. We posit that each mechanism leads to a characteristic change in perceived velocity that we refer to as the signature of adaptation. We have demonstrated that the superimposed signatures of the two mechanisms can account for perceptual biases measured over a broad range of speeds and across two opposite directions of motion. This two-mechanism model is significantly better in explaining the measured data than alternative models that assume only a single adaptation mechanism. We have validated our model by making an accurate quantitative prediction of individual subjects' velocity biases measured in a separate experiment using an adaptor designed to isolate the nondirectional mechanism. 

We have also used the model to generate a set of predictions for aftereffects beyond those tested here. We find that predictions of subjects' static and nulling motion aftereffects for a continuous range of adaptor speeds are in agreement with previously reported measurements of the static MAE (Nishida & Johnston, 1999; Wright & Johnston, 1985). In addition, we find that predicted biases of a test stimulus moving at the speed of the adaptor are always toward slower speeds, as is generally assumed (see, e.g., Thompson, 1981), but that they vanish for both low and high adaptor speeds. Last, because the model distinguishes directional and nondirectional effects, we can predict the form of aftereffects arising from nondirectional adaptors. In total, this two-mechanism model serves to unify a broad and disjoint set of results from the literature on motion adaptation. 

The nondirectional portion of our model is consistent with a previously developed mechanistic model for motion perception known as the “ratio model” (Hammett et al., 2005; Harris, 1980; Perrone & Thiele, 2002; Tolhurst, Sharpe, & Hart, 1973). In its simplest form, this model consists of a single mechanism with two temporal frequency channels, one low-pass and one high-pass, whose relative responses may be used to estimate stimulus speed but not direction. Previous studies have proposed that differential gain changes in the two channels would be sufficient to account for motion aftereffects. Smith (1985), for example, shows that biases due to flicker adaptation can be qualitatively explained this way. This explanation is consistent with our model (see Figure 7c), since a flicker adaptor would affect the nondirectional mechanism only. In addition, Hammett et al. (2005) collected bias data for directional adaptors but used only a single test direction. They showed that their data could be approximately fit with the ratio model. However, as we have demonstrated (see Figure 5), this model cannot account for the asymmetry in the observed estimation biases across directions, and thus provides a relatively poor fit to data over the whole velocity range. 

Our analysis does not specify the physiological details nor loci of the two mechanisms, but the basic response properties of neurons in the dorsal stream of the primate visual pathway suggest a plausible instantiation in terms of a sequential processing cascade. Figure 8a illustrates an implementation of such a cascade. The first stage consists of a population of broadly tuned nondirectional speed-tuned channels, ³ similar to those posited for the ratio model. These might correspond to early visual neurons, e.g., those in the lateral geniculate nucleus (LGN). Recent studies show that, in contrast to earlier claims, magnocellular cells in the LGN adapt to stimuli of rather low temporal frequency (Solomon, Peirce, Dhruv, & Lennie, 2004). The output of this first stage provides the input to a second stage of directional velocity-tuned channels. This second stage encodes velocity using more narrowly tuned direction-selective neurons (e.g., direction-selective neurons in cortical areas V1 or MT). The perceived velocity is decoded from this second stage. During adaptation, the neurons in each stage undergo a reduction in response gain according to their own response level. After adaptation, the net gain change seen in the second stage is the superposition of (nondirectional) changes inherited from the first stage, along with the (directional) gain changes that occur within the second stage. 

Figure 8

 

Simulation of adaptation effects arising from gain reduction at two stages of a velocity processing cascade. (a) The first stage consists of two (classes of) nondirectional channels, tuned for low and high speeds. In the illustrated example, adaptation changes the relative response gain of the high speed channel (blue region). Channels of the first stage then provide input to a second processing stage containing more narrowly tuned direction-selective velocity channels, which inherit the adaptation induced gain changes from the first stage but also undergo additional (directional) gain changes themselves (red region). The percept is decoded from the second stage. The middle column shows one-dimensional slices along the horizontal velocity axis. (b) Simulation results for the case of inherited gain reduction from the first stage only (blue), as well as the full model with gain changes in both stages (red). The population consisted of 21 equally spaced channels with rectified cosine tuning curves and equal tuning width when plotted in the chosen perceptual space. The mean response of each channel is determined by its tuning curve (shown in (a)) and response variability was assumed to follow a Poisson process. The perceptual estimate was assumed to be the population average, computed as v^(v_s) = ∑_iv₀ⁱrⁱv_s/∑_irⁱv_s, where v₀ⁱ is the preferred velocity of channel i and rⁱ is its response to a stimulus with velocity v_s. The model qualitatively predicts the subjects' perceptual bias as shown in Figure 3. Results represent the average bias measured over 30,000 samples of the population response for each test velocity (see Supplementary material for simulation results for estimation variability). The gain reduction profile was Gaussian, with width twice and amplitude half as large for the nondirectional than for the directional mechanism, roughly matched to the fitted parameters from our subjects' data (see Figure 4b).

To illustrate more precisely how this can lead to perceptual aftereffects, we simulated a simple implementation of this cascade model using a population average read-out rule (i.e., the sum of the peak tuning speed of each channel, weighted by the response of that channel, normalized by the total population response) to link channel responses to percept. The simulated perceptual biases arising from the nondirectional mechanism alone are in close agreement with the proposed signature of the nondirectional mechanism (blue curves in Figure 8b). Incorporating adaptation-induced gain changes in the second (directionally tuned) stage leads to a superposition of two signatures, as exhibited by our subjects. The simulated biases (red curve in Figure 8b) are in good qualitative agreement with the data shown in Figure 3. The cascade model also serves to explain some of the characteristics of the fitted parameters (see Figure 4). For example, the increasingly negative weights for the nondirectional signature with increasing adaptor speed directly follow from the gain change in the first stage of the processing cascade: as adaptor speed increases, the gain of the high-speed channel is increasingly reduced relative to that of the low-speed channel. ⁴ 

In addition to explaining adaptation-induced biases, we can also make a prediction of the adaptation-induced changes in the variability of the perceived stimulus velocity by assuming an appropriate noise model for the channels. If we assume that each channel's response corresponds to a sample from a Poisson distribution with rate determined by the associated tuning curve, the underlying adaptation mechanism will not only generate a signature in estimation bias (mean of the estimate) but an additional signature in the changes of estimation variability. We show simulations of this in the Supplementary material and demonstrate that this extended signature is consistent with the measured discrimination thresholds of our subjects. 

Although the main experiment and analysis is based on data gathered for horizontal motion only, the results of the validation experiment suggest a generalization to the full two-dimensional motion (velocity) space and provide a natural explanation for the seemingly different motion aftereffects observed with respect to direction and speed. We assume that the adaptation signature will only reveal itself perceptually if the underlying adapted neural population is directly involved in solving the perceptual task on the test stimuli. Thus, in experiments for which subjects report only on the direction of a test stimulus, we would expect that one should see only the signature of the directional mechanism (i.e., repulsive biases in perceived direction away from that of the adaptor; Levinson & Sekuler, 1976) and not that of the nondirectional mechanism. If subjects are asked to report on both speed and direction of a test stimulus, we expect the nondirectional mechanism to generate additional effects, and previously reported joint aftereffects in direction and speed are qualitatively consistent with this (Schrater & Simoncelli, 1998). Thus, the model provides further evidence that direction and speed are not separately encoded in the brain (as has been suggested; Curran & Benton, 2006; Matthews & Qian, 1999) but are jointly encoded as a vector entity (Gardner, Tokiyama, & Lisberger, 2004; Heeger, 1987; Schrater & Simoncelli, 1998; Simoncelli & Heeger, 1998). 

Although our model simulations are based on the parsimonious assumption that adaptation results from response-driven gain changes, our notion of a perceptual signature is agnostic about the detailed nature of the underlying mechanism. We note that a variety of physiological effects other than gain changes have been measured and modeled (Jin et al., 2005; Kohn, 2007; Schwartz et al., 2007; Seriès et al., in press). It has been suggested that some of these observed effects may arise from changes that occur at stages of processing other than the one being measured (Kohn & Movshon, 2004), analogous to the changes in the second stage of our model that are partly inherited from the first stage. Taking this concept to its logical extreme, we can speculate that all physiologically observed adaptation effects (including changes in tuning curve shape or position and changes in noise properties) might be explained as originating from adaptation-induced gain changes that have occurred at different stages of the system. 

Finally, if adaptation is governed by a single, isomorphic mechanism that is repeated at different levels of a processing cascade, it is worth considering how that single mechanism can be understood from a normative perspective. Several functional roles for adaptation (which are not mutually exclusive) have been suggested: the minimization of correlations in sensory encoding (Barlow & Földiak, 1989; Wainwright, Schwartz, & Simoncelli, 2002), optimization of information transmission (Brenner, Bialek, & de Ruyter van Steveninck, 2000; Fairhall, Lewen, Bialek, & de Ruyter van Steveninck, 2001; Wainwright, 1999), or optimization of perceptual discriminability and constancy (Abrams, Hillis, & Brainard, 2007; see also Clifford et al., 2007; Grzywacz & Balboa, 2002; Stocker & Simoncelli, 2006b). While it remains unclear which of these roles best explains both the physiological and psychophysical data, we believe that such an overarching principle is essential for understanding how the brain can perform reliable and consistent computation in the face of the constantly fluctuating adaptation state of its processing elements. 

Supplementary Materials

Extended signature of adaptation 

The concept of a signature of adaptation can be extended to include changes in variability of the perceptual estimate. More specifically, if we assume a gainchange model as the underlying mechanism for adaptation and assume an appropriate noise model for the channel responses, we can predict what the characteristic change in estimation variability should be. For example, assuming the channels’ responses for a given stimulus to follow a Poisson distribution, we can demonstrate with simulations that the relative change in variability of the estimate is symmetric about the adaptor, typically increased at the adaptor and decreased at adjacent values (see also Schwartz, Hsu, & Dayan, 2007). Again, since the details are not important in the context of this paper, we use a simple, single-parameter function (second derivative of a Gaussian) that captures the main characteristics of the relative change in variability. Figure 1 shows the full signature of adaptation, for the combined changes in the mean and variability of perceptual estimates, here expressed as the difference in means and the relative change in standard deviation of the estimation distributions . 

Figure 1. Extended signature of adaptation.

 

(a) The gain-change model mechanism can be extended by considering response variability (Stocker & Simoncelli, 2006). (b) Decoding channel activity assuming Poisson noise statistics leads to a perceptual signature in bias and a change in estimation variability. The signature for variability (here expressed as the relative change of standard deviations is symmetric about but increased at the adaptor and decreased for stimuli that are further away in perceptual space.

Linking estimation variability to perceptual 2AFC data 

While the estimation bias can be directly measured as the shift of the psychometric function under adapted versus control conditions, the changes in estimation variability are only indirectly reflected in changes in perceptual discriminability as measured by the slope of the psychometric function. This relationship is further complicated by our single-sided 2AFC adaptation protocol (see Methods section in the main text). Alternatively, discriminability could have been extracted directly from a separate two-sided adaptation experiment in which both reference and test locations were adapted. However, this would have resulted in doubling the length of the already extensive experiment because the single-sided adaptation condition is still needed to compute the bias. 

In the following, we use signal detection theory (Green & Swets, 1966; Wickens, 2001) to extract the changes in variability from the subjects’ psychometric functions. We assume that when performing a 2AFC motion discrimination task between reference and test stimuli with velocities vr and vt, subjects perform independent estimates and of each stimulus velocity, and then take a decision accordingly (Stocker & Simoncelli, 2006). Due to noise, these estimates will fluctuate over repeated trials, leading to distributions of estimates. By approximating the distributions of estimates with Gaussians of mean and standard deviation , we can define the perceptual discriminability between a reference and a test stimulus as 

 

 

(1)

 

Figure 2. Signal detection theory and the relationship between perception and estimation.

 

The two grayscale plots illustrate the distributions of a subject's estimate under adaptation and reference conditions over a broad range of stimulus velocities v. Presumably, adaptation changes the mean μ*( v) as well as the standard deviation σ*( v) of these distributions as a function of the adaptor v_a and the stimulus velocity v. These changes in the subjects' estimation of velocity are then expressed as changes in the point-of-subjective equality v_pse and discrimination threshold v_thres. Signal detection theory links these perceptual measures to estimator bias ( μ( v)* − μ( v)) and relative change in variability (log σ*( v)/ σ( v)).

In general, the estimators for the test and reference stimuli are different (see Figure 2). In our single-sided adaptation experiment, the estimator for the reference stimulus was always in the unadapted state, whereas the estimator for the test was in one of the four adaptation conditions or the control condition. For each under each adaptation condition we fit the subjects’ psychometric function with a cumulative Gussian, and then extracted two values of from the fit that correspond to two particular values of d the point-of-subjective equality for which dV= 0, and the discrimination threshold velocity that is associated with a certain (arbitrarily chosen) cumulative probability on the psychometric curve (here, we use = 0.875, with an associated value of = 1.627). This is illustrated graphically in Figure 2. For each adaptation condition (fully adapted, control adapted, unadapted), Equation (1) may be used to define these two special values of through implicit equations. For example, the values and for the fully adapted condition are determined by 

 

 

(2)

 

 

 

(3)

 

for each test velocity vt under each condition, where the superscripts {*, 0} indicate the adaptation state (fully adapted and unadapted, respectively). In order to solve for and , we make the following assumptions: (i) Without loss of generality, we assume that the estimator for the reference stimulus is veridical, i.e., . Furthermore, we also assume that the variability of the estimator changes smoothly, i.e., it remains roughly constant over the velocity range of the discrimination threshold. 

Under assumption (i), Equation 2 implies that . Thus, we can extract the estimation bias directly as the difference in the measured poin-of-subjective equality under full adaptation and control adaptation conditions:  

 

 

(4)

 

We can solve for the estimation variability, , by substituting for in Equation (3) and rewriting as 

 

 

(5)

 

An analogous expression may be derived for the control condition (superscript {c}), and the two may be combined to compute the relative change in estimation variability . 

A second control experiment measured discrimination thresholds under unadapted conditions, and provided the estimates of that are required in Equation (4). We assume the unadapted variability at the test location to be approximately equal to that at the reference location when evaluated at the threshold velocity: 

 

 

(6)

 

for all . Substituting this into Equation 1, and again assuming that = v gives 

 

 

(7)

 

The values of σ⁰(v) required in Equation 5 are obtained by linear interpolation. 

Figure 3. Changes in estimation variability due to adaptation.

 

Dots indicate the adaptation-induced changes in estimation variability under all adaptation conditions and all three subjects. Note that they are not equivalent to but rather extracted from the measured perceptual discrimination thresholds using the analysis as discussed in the text. Changes are shown as the log ratio of the standard deviations in adaptation and control conditions. As in the case of bias, the change in variability typically increases in amplitude with increasing adaptor speeds. Bold red lines represent fits according to our model. Gray shaded area indicates the 5–95% quantiles over 1000 bootstrap samples of the data.

Figure 3 shows the adaptation-induced changes in estimation variability under all adaptation conditions and for all three subjects, extracted with the above analysis. Signal detection theory allowed us to extract both estimation bias and changes in variability from a single one-sided adaptation experiment (see also Figure 2), where the bias directly follows from the measured point-of-subjective equality , but the extraction of the estimation variability requires both measurements of and . 

Model prediction and fit for changes in variability 

In order to test whether the superposition of signatures also extends to estimation variability, we predicted the expected relative changes in estimation variability based on the model parameters fit to the bias data alone (Figure 2 in main text), and then compared this prediction to the measured variability data shown in Figure 3. More specifically, we applied the same linear superposition of signatures as for the bias (Figure 4), allowing only two additional parameters that account for potential scale differences in amplitude and size between the signatures for bias and relative change in variability. The fit in Figure 3 (red curves) shows that although the data are rather noisy, they are in agreement with the prediction from our model. 

Figure 4. Superposition of directional and nondirectional mechanisms.

 

(a) For each subject, we used the parameter values for signature amplitudes and widths over all adaptation conditions that were fit to the estimation bias (values shown in Figure 3 in the main text). Again, the nondirectional signature was centered at zero motion, while the directional signature was centered at the adaptor velocity. (b) To account for potential overall scale differences in signature width and amplitude, we allowed two extra parameters R_vb and A_vb.

We have shown in previous work (Stocker & Simoncelli, 2005, 2006), that it is important to account for both, estimation bias and variance in order to validate and constrain perceptual models with data. In this regard, it is reassuring that our hypothesis of a superposition of two isomorphic adaptation mechanisms is supported not only by the measured bias but also the changes in subjects' variabilities. 

Simulation of the cascade model including estimation variability 

We consider the same two stage cascade model as described in the main text, expanding it to include response variability in the channels. We assume that each channel response follows a Poisson distribution with rate determined by the associated tuning curve. As shown in Figure 5, not only bias but also estimation variability of the simulated model are in close agreement with the proposed extended signature (Figure 1). 

Figure 5. Simulated estimation variability of the cascade model.

 

(a) The first stage consists of two nondirectional channels, tuned for low and high speeds. Adaptation changes the relative response gain of the channels based on which channel is responding more strongly to the adapting stimulus. These channels provide input to a second processing stage containing more narrowly tuned direction-selective velocity channels, which inherit the adaptation-induced gain changes of the first stage and also undergo additional (directional) gain changes in response to the adapting stimulus. (b) Simulation results for biases and changes in variability for the case of inherited gain reduction from the first stage only (blue), as well as the full model with gain changes in both stages (red). The mean response of each channel is determined by its tuning curve (shown in (a)) and response variability was assumed to follow a Poisson process. The population response was decoded by a simple population average read-out (i.e., the sum of responses, weighted by the preferred speed of each channel).

References

Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York: Wiley. 

Schwartz, O., Hsu, A., & Dayan, P. (2007). Space and time in visual context. Nature Reviews Neuroscience, 8, 522–535. 

Stocker, A., & Simoncelli, E. (2005). Constraining a Bayesian model of human visual speed perception. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems NIPS 17 (pp. 1361–1368). Cambridge, MA: MIT Press. 

Stocker, A., & Simoncelli, E. (2006). Noise characteristics and prior expectations in human visual speed perception. Nature Neuroscience, 9, 578–585. 

Wickens, T. D. (2001). Elementary signal detection theory. Oxford University Press. 

Acknowledgments

We thank Suzanne McKee, Josh McDermott, and Adam Kohn for helpful comments on the manuscript, and the subjects for their energy and persistence in completing these somewhat exhausting experiments. 

Commercial relationships: none. 

Corresponding author: Alan A. Stocker. 

Email: [e-mail]. 

Address: Department of Psychology, University of Pennsylvania, 3401 Walnut Street 313C, Philadelphia, PA 19104‐6228, USA. 

Footnotes

Footnotes

¹  Xu, Dayan, Lipkin, and Qian (2008) have recently presented an example of multi-level adaptation in face processing.

Footnotes

²  Note that for subjects 1 and 3, there was almost a two-year interval between the original experiment (from which the model parameters are estimated and the prediction computed) and the validation experiment.

Footnotes

³  Whether this first stage is tuned for speed or temporal frequency cannot be conclusively determined based on our experiments.

Footnotes

⁴  Note that the weights of the nondirectional signature are not positive at low adaptor speeds because the perceptual changes are measured relative to the control stimulus condition (see Figure 1a), which, presumably, also adapts the low-speed channel.