Free
Visual Psychophysics and Physiological Optics  |   February 2010
Spatiotemporal Interactions in Retinal Prosthesis Subjects
Author Affiliations & Notes
  • Alan Horsager
    From the Neuroscience Graduate Program and
    the Zilkha Neurogenetic Institute, University of Southern California, Los Angeles, California;
  • Robert J. Greenberg
    Second Sight Medical Products, Inc., Sylmar, California; and
  • Ione Fine
    the Department of Psychology, University of Washington, Seattle, Washington.
  • Corresponding author: Alan Horsager, Zilkha Neurogenetic Institute, University of Southern California, 1501 San Pablo Street, Room 413, Los Angeles, CA 90033; horsager@usc.edu
Investigative Ophthalmology & Visual Science February 2010, Vol.51, 1223-1233. doi:10.1167/iovs.09-3746
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Alan Horsager, Robert J. Greenberg, Ione Fine; Spatiotemporal Interactions in Retinal Prosthesis Subjects. Invest. Ophthalmol. Vis. Sci. 2010;51(2):1223-1233. doi: 10.1167/iovs.09-3746.

      Download citation file:


      © 2016 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements
Abstract

Purpose.: Vision loss due to retinitis pigmentosa affects an estimated 15 million people worldwide. Through collaboration between Second Sight Medical Products, Inc., and the Doheny Eye Institute, six blind human subjects underwent implantation with epiretinal 4 × 4 electrode arrays designed to directly stimulate the remaining cells of the retina, with the goal of restoring functional vision by applying spatiotemporal patterns of stimulation. To better understand spatiotemporal interactions between electrodes during synchronous and asynchronous stimulation, the authors investigated how percepts changed as a function of pulse timing across the electrodes.

Methods.: Pulse trains (20, 40, 80, and 160 Hz) were presented on groups of electrodes with 800, 1600, or 2400 μm center-to-center separation. Stimulation was either synchronous (pulses were presented simultaneously across electrodes) or asynchronous (pulses were phase shifted). Using a same-different discrimination task, the authors were able to evaluate how the perceptual quality of the stimuli changed as a function of phase shifts across multiple electrodes.

Results.: Even after controlling for electric field interactions, subjects could discriminate between spatiotemporal pulse train patterns based on differences of phase across electrodes as small as 3 ms. These findings suggest that the quality of the percept is affected not only by electric field interactions but also by spatiotemporal interactions at the neural level.

Conclusions.: During multielectrode stimulation, interactions between electrodes have a significant influence on the quality of the percept. Understanding how these spatiotemporal interactions at the neural level influence percepts during multielectrode stimulation is fundamental to the successful design of a retinal prosthesis.

Vision loss from photoreceptor diseases such as retinitis pigmentosa (RP) and age-related macular degeneration (AMD) affects an estimated 15 million people worldwide. 1 RP begins with photoreceptor degeneration in the periphery, and this degeneration gradually spreads from the periphery to the fovea. In later stages of the disease, the spatial organization of the inner nuclear and ganglion cell layers becomes disorganized, and bipolar, amacrine, and ganglion cells begin to die. 2,3 However, the inner nuclear and ganglion cell layers maintain relatively high cell density, 46 and some functional circuitry remains, 79 even in later stages of disease. As a result, several groups have developed microelectronic retinal prostheses with the ultimate goal of restoring vision in blind subjects by stimulating remaining retinal cells with spatiotemporal sequences of electrical pulses. Indeed, both semiacute and long-term implanted devices have been demonstrated to be safe and capable of generating visual percepts in human subjects (Humayun MS. IOVS 2009;50:ARVO E-Abstract 4744; Sachs HG, et al. IOVS 2009;50:ARVO E-Abstract 4742). 1016  
Although limited data have been reported about how electrodes interact during spatiotemporal stimulation in the retina, it is well known that for cochlear implants the precise timing of stimulation across electrodes has perceptual consequences as a result of both electrical field 17,18 and neuronal interactions. 19 Within cochlear implants, the most common approach to dealing with electrode interactions has been to reduce channel interactions (or cross-talk between electrodes) by phase-shifting stimuli across electrodes, a technique also referred to as continuous interleaved sampling. 20 In the case of cochlear implants, interleaving patterns of electrical pulses reduces electrical and neural nonlinearities generated by channel interactions and makes the resultant electrical fields and percepts easier to computationally model. Besides reducing interactions, phase-shifting stimulation across electrodes provides the technical advantage of allowing multiple electrodes to share the same driver. This is not of great importance for cochlear implants because these devices have a relatively small number of electrodes (<30). Although the retinal devices implanted in the subjects tested here also had a relatively small number of electrodes, the design used shared drivers across pairs of electrodes: there were eight drivers for the 16-electrode implant. The ability to share drivers across multiple electrodes may be more critical in future retinal implants, which are likely to have many hundreds or even many thousands of electrodes. 
However, it is also possible that spatiotemporal interactions between electrodes might be a powerful tool for increasing the spatial resolution of retinal implants. Within the cochlear implant literature it has been shown that spatiotemporal interactions between adjacent cochlear electrodes can be used to produce stimuli with pitches that are intermediate between the two stimulated electrodes. Simultaneous 21 or near-simultaneous 22 stimulation of adjacent electrodes produces pitch percepts intermediate to those produced by each electrode separately, thereby increasing the number of place-pitch steps available to cochlear implant listeners (virtual electrodes). Spatiotemporal interactions in cochlear implants have been shown to be capable of creating two to nine virtual electrodes, depending on the observer. Given that the human fovea contains approximately 160,000 cones per square millimeter while current retinal implant technology consists of arrays of 1000 electrodes or fewer, the ability to exploit spatiotemporal interactions may prove critical in improving resolution. 
Here we systematically examined a variety of spatiotemporal interactions in two subjects with retinal prostheses. The experiments described here focus on the ability of subjects to discriminate between pulse patterns across groups of electrodes in which the stimulation on any individual electrode was identical across the two pulse patterns, but the temporal relationship (phase-shifting) between electrodes varied. For example, in experiment 1, we tested the ability of subjects to discriminate between stimuli in which 4 electrodes were stimulated simultaneously or were stimulated using pulse trains that were temporally phase-shifted with respect to each other. 
It should be noted that in the experiments described here, most results were obtained using pulse patterns that were well above the critical flicker fusion (CFF) limit, the rate at which there is no conscious awareness of flicker. Measured CFF values in our two subjects (75% discrimination thresholds for stimulation on a single electrode) were 60 Hz and 40 Hz (see Supplementary Material). Consistent with these measured CFF values, our subjects did not report flicker for any stimuli of 40 Hz or greater. 
It is known for light stimuli that sensitivity to flicker depends on the intensity and size of stimuli and on their position in the receptive field. Our electrode arrays were positioned fairly centrally (including the fovea). The physical size of individual electrodes corresponded to approximately 1° to 2° of visual angle 23 ; however, the size of the percepts elicited by a given electrode seemed to be approximately twice that according to subject report, 16 possibly because of the spread of current activation. Our stimuli were presented at current levels of 119 to 470 μA (2–3 times threshold), an intensity that seemed bright, but not uncomfortably bright, to the subjects. Our finding of CFF limits of 40 to 60 Hz is, therefore, consistent with data on the light CFF for visually normal observers, which finds CFF limits of approximately 45 Hz for 2° to 4° stimuli presented centrally at asymptotic brightness levels. 24 For comparison, with large, bright peripheral stimuli, subjects have a maximum CFF of approximately 60 Hz. However, it is also worth noting that there is some evidence of cortical sensitivity to rates of flicker above the perceivable limit, as discussed below. 25  
Subjects, Materials, and Methods
Subjects
Results reported here are based on data from two subjects, S05 and S06, who underwent long-term implantation of 16-electrode retinal prostheses (Second Sight Medical Products, Inc.). The subjects were 59 and 55 years old, respectively, at implantation in 2004. Before surgery, subject S05 had bare light perception (BLP) in the implanted eye and had BLP for 8 years before implantation; subject S06 had no light perception (NLP) for 10 years before implantation. Without stimulation, subjects reported that their visual fields had a grayish background, and this perception of a gray background remained fairly consistent throughout the period of testing. 
Collection of data reported here began several months after subjects underwent implantation with the prosthetic devices. These tests were carried out during a period of approximately 90 to 1170 days after implantation for S05 and 30 to 1110 days after implantation for S06. 
These two subjects were a subset of six subjects who underwent implantation since February 2002. The other four subjects were excluded for a variety of reasons: one subject was excluded because of geographic location (this subject lived approximately 2600 miles from the testing site), and two subjects were excluded because of unrelated medical conditions. In one subject the array cable became exposed. The combination of a thin conjunctiva and an epithelialized cable meant that repositioning the cable would have required grafting of conjunctiva, sclera, or both over the cable site. Scarring from the previous surgery was likely to have reduced the effectiveness of local anesthetic, and the cardiac status of this subject precluded general anesthesia. As a consequence, the decision was made to cut the multiwire cable connecting the array to the external stimulator and to leave the intraocular portion of the array in place. 
In some experiments, data were only collected in subject S06 because a surgical procedure was carried out on S05 in 2008 to adjust the extraocular cable component. This adjustment caused a slight lifting of the array from the retina, which resulted in a substantial increase in perceptual thresholds (in many cases, single-electrode thresholds could not be measured), making it impossible to continue data collection using the suprathreshold paradigms discussed in this article. Some of the brightness-matching experiments (experiment 6) could not be carried out in S06 because his geographic distance (60 miles from the testing center) limited his general availability for testing. 
All tests were performed after obtaining informed consent under a protocol approved by the Institutional Review Board at the Keck School of Medicine at the University of Southern California and under the principles of the Declaration of Helsinki. 
Retinal Prosthesis
Subjects underwent epiretinal implantation with a 4 × 4 array of disc electrodes in the macular region (Fig. 1A). Electrodes were either 260 or 520 μm in diameter, arranged in an alternating checkerboard pattern with 800 μm center-to-center separation between each electrode. As described elsewhere, 15,26,27 pulse train signals were generated and sent to an external signal processor using custom software run on a personal laptop computer. Power and signal information were sent from this processor through a wire to an external transmitter coil that attached magnetically, and communicated inductively, to a secondary coil that was implanted subdermally in the subject's temporal skull (Fig. 1B). From this secondary coil, power and signal information were sent through a subdermally implanted wire that traversed the sclera to the array of electrodes. The timing and current of electrical pulses on each electrode could be controlled independently. 
Figure 1.
 
(A) Electrode array. The electrode array consists of 260- or 520-μm electrodes arranged in a checkerboard pattern, with center-to-center separation of 800 μm. The entire array covers approximately 2.9 × 2.9-mm of retinal space, subtending approximately 10° of visual angle. Electrodes are designated by a letter/number combination (A1-D4). (B) Prosthesis. Pulse sequences are programmed using custom software run on a personal laptop computer, which communicates stimulus parameters to an external visual processing unit (not shown). Signal and power information is then passed through an external inductive coupling device (not shown) that attaches magnetically to a subdermal coil implanted in the subject's temporal skull. This signal is then sent through a parallel system of wires to the epiretinally implanted electrode array. Note that the power and signal information can be independently controlled for each electrode. (A) and (B) Adapted from Horsager A, Greenwald SH, Weiland JD, et al. Predicting visual sensitivity in retinal prosthesis patients. Invest Ophthalmol Vis Sci. 2009;50:1483–1491. (C) Pulse train. All pulse train stimuli consisted of biphasic, cathodic-first, charge-balanced, square-wave pulses. The cathodic and anodic phases of each biphasic pulse were 0.075 ms in duration, with a 0.075 ms interphase delay. Stimulation on each electrode in the group of four electrodes consisted of a 500-ms pulse train of variable frequency. The example shown here is of a 20-Hz pulse train of 500 ms duration.
Figure 1.
 
(A) Electrode array. The electrode array consists of 260- or 520-μm electrodes arranged in a checkerboard pattern, with center-to-center separation of 800 μm. The entire array covers approximately 2.9 × 2.9-mm of retinal space, subtending approximately 10° of visual angle. Electrodes are designated by a letter/number combination (A1-D4). (B) Prosthesis. Pulse sequences are programmed using custom software run on a personal laptop computer, which communicates stimulus parameters to an external visual processing unit (not shown). Signal and power information is then passed through an external inductive coupling device (not shown) that attaches magnetically to a subdermal coil implanted in the subject's temporal skull. This signal is then sent through a parallel system of wires to the epiretinally implanted electrode array. Note that the power and signal information can be independently controlled for each electrode. (A) and (B) Adapted from Horsager A, Greenwald SH, Weiland JD, et al. Predicting visual sensitivity in retinal prosthesis patients. Invest Ophthalmol Vis Sci. 2009;50:1483–1491. (C) Pulse train. All pulse train stimuli consisted of biphasic, cathodic-first, charge-balanced, square-wave pulses. The cathodic and anodic phases of each biphasic pulse were 0.075 ms in duration, with a 0.075 ms interphase delay. Stimulation on each electrode in the group of four electrodes consisted of a 500-ms pulse train of variable frequency. The example shown here is of a 20-Hz pulse train of 500 ms duration.
Psychophysical Methods
Stimulation Paradigm.
All pulse waveforms on each electrode consisted of biphasic, cathodic-first, charge-balanced square wave pulses, presented as 500-ms pulse trains (Fig. 1C). For safety reasons, all individual pulses within a pulse train were charge-balanced. Here, we used cathodic and anodic pulses of equal width (0.075 ms). Each biphasic pulse within the pulse train contained a 0.075-ms interphase delay. All stimuli were presented in photopic conditions. All pulse train stimuli were set to be approximately 2× to 3× the measured threshold 15 of each electrode. 
Same-Different Discrimination.
Experiments 1, 2, 3, and 5 measured performance using a two temporal interval same-different discrimination paradigm. In each trial, subjects were presented with two temporal intervals of stimulation. Each interval contained 1 of 2 pulse trains (A or B). The stimuli in the two intervals could be A and A, B and B, A and B, or B and A. The order of the A and B stimuli across the two intervals was randomized across trials, and each possible combination was presented with equal frequency. Subjects were asked to judge whether the two temporal intervals contained stimuli that were the same or different through a button-press response. 
A and B stimuli consisted of suprathreshold pulse train stimulation across groups of four electrodes in a square configuration (Fig. 2). The temporal properties of the pulse train presented on each electrode were identical in every way (pulse train frequency and pulse width) except for the phase-shift between pulses across electrodes. On any individual electrode, the pulse train presented was identical across synchronous (zero-phase shift across electrodes), pseudosynchronous (0.225-ms phase shifts across electrodes), or asynchronous (1.5- to 12-ms phase shifts across electrodes) stimuli. Stimuli for each experiment are described in further detail below. Subjects were instructed to use any visual cue to discriminate between the two stimulation patterns. Phosphenes on single electrodes were generally reported as round or oval and white or yellow. Shapes were reported as approximately 0.5 to 2 inches in diameter at arm's length, corresponding to roughly 1° to 3° of visual angle. When the percept was reported as oval, the longer axis was generally 2 to 3 times the length of the shorter axis. 
Figure 2.
 
(A) Experiment 1: Synchronous versus asynchronous stimulation. Subjects discriminated between synchronous and asynchronous pulse train stimuli across groups of four electrodes using a same-different task. (B) Subject performance experiment 1. Percentage correct is shown for frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). 50% is chance performance. SEMs were calculated by taking each run as a separate measure, with a single run carried out on each electrode group.
Figure 2.
 
(A) Experiment 1: Synchronous versus asynchronous stimulation. Subjects discriminated between synchronous and asynchronous pulse train stimuli across groups of four electrodes using a same-different task. (B) Subject performance experiment 1. Percentage correct is shown for frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). 50% is chance performance. SEMs were calculated by taking each run as a separate measure, with a single run carried out on each electrode group.
When synchronous, pseudosynchronous, or asynchronous stimulation was presented on the 2 × 2 sets of electrodes, the percept was generally of a larger spot of relatively uniform brightness, which was reported to appear to be approximately 2 to 4 inches in diameter at arm's length, corresponding to roughly 3° to 6° of visual angle. The complexity of the stimulus was much greater than with single electrodes: the percept generally consisted of multiple phosphenes but did not generally contain phosphene patterns that aligned with the map of activated electrodes; in other words, the percept elicited by a 2 × 2 array of activated electrodes did not generally map neatly onto a 2 × 2 array of visual percepts in the expected location in space. Synchronous, pseudosynchronous, and asynchronous stimuli were generally perceived as spatially identical and differed only in perceived temporal properties (i.e., flicker) for pulse frequencies of 20 Hz. 
Brightness Matching.
Experiments 4 and 6 used a brightness discrimination procedure to find the point of equibrightness for two different stimuli. We used a two-interval, forced-choice procedure. One interval contained the “standard” stimulus, and the other contained the “test” stimulus. The order of the two intervals was randomized across trials. On each trial, subjects were asked to report which interval contained the brighter stimulus (note that subjects performed a discrimination task rather than a brightness-matching task). A one-up, one-down staircase method was used to adjust the amplitude of the test stimulus based on the observer's response. A cumulative normal was then used to find the point of subjective equibrightness (the stimulus intensity for which subjects were equally likely to report either stimulus as brighter), and error bars were estimated using an adaptive sampling Monte-Carlo simulation. 28 Each brightness match was based on a minimum of 100 trials. Each individual psychometric function was inspected to ensure that an adequate fit was obtained, and data were recollected if fits were inadequate (based either on the estimated error or visual inspection). 
Results
Experiment 1: Synchronous versus Asynchronous Stimulation
The goal of this experiment was to measure the perceptual impact of electric field and neuronal spatiotemporal interactions. The synchronous stimulus (A) consisted of pulse train stimuli that were presented simultaneously across all four electrodes. In the asynchronous stimulus (B) the pulse train was phase-shifted across each electrode by 12, 6, 3, or 1.5 ms (Fig. 2), depending of the frequency being used (20, 40, 80, and 160 Hz, respectively). These phase-shifts were chosen to maximize the temporal separation between stimulation on each electrode. 
The question presented to the subject was, “Are the stimuli in the two temporally separated intervals the same or different?” In half the trials, stimuli were physically identical; in the other half, they were different (synchronous versus asynchronous). A correct response included either successfully identifying two physically identical stimuli (e.g., both synchronous) as the same or successfully identifying synchronous versus asynchronous stimuli as different. An incorrect response included either identifying two physically identical stimuli as different or identifying synchronous versus asynchronous stimuli as the same. For each of the two subjects, five 4-electrode groups were evaluated for each of the four different frequencies. Electrodes were always spatially separated by 800 μm center to center. One hundred trials were run for each frequency, and data were averaged across all five electrode groups. Each bar in the graph represents 500 trials collected across five runs of 100 trials (Fig. 2B). 
As seen in Figure 2B, both subjects easily discriminated between synchronous and asynchronous stimuli, with performance consistently greater than 80%. Performance was significantly greater than chance for every frequency (one-tailed t-test; P < 0.01). We found no effect of subject, condition, or frequency (three-factor ANOVA, subject × condition × frequency; P > 0.05). Performance (both percentage correct and d-prime) is reported for all experiments in Table 1. Discrimination tasks such as these are often modeled in terms of signal detection theory, by which the internal responses to both stimuli are described by Gaussian probability distributions that vary in their means along an internal response axis. According to such models, the relationship between stimulus discriminability and percentage correct is nonlinear. A change in percentage correct from 65% to 70% cannot be considered comparable to a change from 90% to 95%. d-Prime describes the separation between means of inferred signal and noise distributions in units of the SD of the noise distribution. As such, within the context of signal detection theory, a d-prime improvement of 1 to 1.5 can be considered equivalent to the improvement between 3 and 3.5. 29,30  
Table 1.
 
Performance Values for Each Subject in Each of the Same/Different Tasks for Experiments 1, 2, and 5
Table 1.
 
Performance Values for Each Subject in Each of the Same/Different Tasks for Experiments 1, 2, and 5
Experiment Subject Frequency*
20 Hz (%/d′) 40 Hz (%/d′) 80 Hz (%/d′) 160 Hz (%/d′)
1 S05 83/2.65 94/3.74 95/3.90 93/3.59
S06 92/3.46 95/3.90 94/3.74 89/3.14
2 S05 58/1.04 64/1.45 62/1.31 55/0.80
S06 61/1.24 53/0.60 58/1.04 53/0.60
5 S05 NA 66/1.57 68/1.67 NA
S06 NA 63/1.38 64/1.45 NA
Generally, subjects described the synchronous stimulus as brighter and having a different shape than the asynchronous stimulus. In experiment 6, we measured the charge needed to brightness match synchronous versus asynchronous stimulation patterns and found that almost twice as much current was needed in the asynchronous patterns to match the brightness of a synchronous standard, consistent with subjective verbal reports in this experiment. 
Experiment 2: Pseudosynchronous versus Asynchronous Stimulation
Here we measured performance after having eliminated electric field interactions by introducing very small phase shifts. We modified the synchronous stimulus used in experiment 1 by adding a 0.225-ms phase shift across each of the four electrodes, creating a pseudosynchronous stimulation pattern (Fig. 3A). The entire set of four pulses across all four electrodes occurred within a 1-ms time window. This phase shift was adequate to eliminate electric field interactions (see Fig. S1 in the Supplementary Material) but is still relatively short compared with the integration period of bipolar (approximately 100 ms), 31 amacrine (49–69 ms), 32 or ganglion (4.5–81.6 ms) cells. 33  
Figure 3.
 
(A) Experiment 2: Discriminating between pseudosynchronous and asynchronous stimuli. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli on groups of four electrodes using a same-different task. All electrodes were neighboring (i.e., 800 μm center-to-center separation). (B) Subject performance in task. Subjects performed a same-different discrimination task in which they were asked to differentiate between pseudosynchronous and asynchronous stimuli presented at frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). The x-axis represents frequency and the y-axis represents the percentage correct. SEMs were calculated by taking each run as a separate measure, and either one or two runs were carried out on each electrode group. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 3.
 
(A) Experiment 2: Discriminating between pseudosynchronous and asynchronous stimuli. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli on groups of four electrodes using a same-different task. All electrodes were neighboring (i.e., 800 μm center-to-center separation). (B) Subject performance in task. Subjects performed a same-different discrimination task in which they were asked to differentiate between pseudosynchronous and asynchronous stimuli presented at frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). The x-axis represents frequency and the y-axis represents the percentage correct. SEMs were calculated by taking each run as a separate measure, and either one or two runs were carried out on each electrode group. P values represent the probability that a subject was performing above chance (dotted line at 50%).
For both subjects, the five 4-electrode groups that were evaluated for each of the four different frequencies were the same as in experiment 1. One or two runs (100 or 200 trials) were run for each condition, and once again data were averaged across electrodes, and standard errors of the mean were calculated across each run. For S05, most bars (Fig. 3B) represent five runs (500 trials). For S06, most bars represent nine runs (900 trials). Other methodological details were identical with those of experiment 1. 
Performance was significantly worse than in experiment 1 (Fig. 3B, Table 1). The difference in performance at each frequency between experiment 1 and experiment 2 was significant for every frequency (three-factor ANOVA, subject × condition × frequency; P < 0.001). ANOVA did not find a significant difference between different electrode groups or frequencies but did find a significant difference in performance between the two subjects (P < 0.05). However, performance was still significantly greater than chance for intermediate frequencies (single-tailed t-test; P < 0.05 for 40 and 80 Hz for both subjects; S05, P < 0.05 for all frequencies), indicating that pulse timing across electrodes affected percepts even after electric field interactions were eliminated. 
The drop in performance compared with experiment 1 suggests that, in the prosthetic device tested here, stimulation did result in overlapping electrical fields and that these interactions between electric fields had a significant effect on subjects' percepts. However, despite a drop in performance, pseudosynchronous and asynchronous stimulation generally remained perceptually distinct, even at frequencies well above the CFF limit. 
One parsimonious explanation of these results is that differentiation between these two stimuli is mediated by neurons lying in between, and receiving stimulation from, more than one electrode. Any individual neuron sitting intermediate between two electrodes will receive a different pattern of stimulation from pseudosynchronous and asynchronous stimulation patterns. If individual neurons are sensitive to these small differences in timing, as is suggested by earlier work measuring threshold sensitivity to different temporal patterns of stimulation 16 and as tested explicitly in experiment 4, then these neurons could potentially mediate the ability of subjects to differentiate the two stimulus conditions. If this were the case, we would expect differences between pseudosynchronous and asynchronous stimulation patterns to decrease as a function of electrode separation. 
Experiment 3: The Effects of Interelectrode Distance—Pseudosynchronous versus Asynchronous Stimulation
Here we evaluated subjects' ability to discriminate between pseudosynchronous and asynchronous stimuli as a function of electrode spacing. A and B stimuli were identical with those described in experiment 2, but we measured discrimination performance for electrodes groups that were separated by 1131, 1600, and 2400 μm (Fig. 4). Only stimuli at 80 Hz were evaluated. For the 800-, 1131-, 1600-, and 2400-μm distances, five, four, four, and one (there is only one possible configuration for the 2400-μm separation) electrode groups were evaluated, respectively, for each subject. At least 300 trials were run for each electrode separation. Standard errors of the mean were calculated by taking each run as a separate measure. For 800-, 1131-, and 1600-μm configurations, either one or two runs were carried on each separate electrode group. For the 2400-μm configuration three runs were carried out on the single possible configuration. 
Figure 4.
 
Experiment 3: Subject performance as a function of electrode separation. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli (80 Hz) on groups of four electrodes using a same-different task. Electrodes were separated by center-to-center distances of 800, 1131, 1600, and 2400 μm. Example schematics of these electrode distances are presented on the y-axis. SEMs were calculated by taking each run as a separate measure. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 4.
 
Experiment 3: Subject performance as a function of electrode separation. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli (80 Hz) on groups of four electrodes using a same-different task. Electrodes were separated by center-to-center distances of 800, 1131, 1600, and 2400 μm. Example schematics of these electrode distances are presented on the y-axis. SEMs were calculated by taking each run as a separate measure. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Subjects' ability to discriminate pseudosynchronous from asynchronous stimulation decreased as a function of increasing electrode separation, to the point at which they were performing at chance (one-tailed t-test; P > 0.05) for electrodes separated by 2400 μm (Fig. 4). For both subjects, performance was significantly above chance for electrode separations of 800 and 1131 μm (single-tailed t-test; P < 0.05), and for subject S05 performance was also above chance for electrode separations of 1600 μm. Although there seems to be a decrease in performance as a function of increasing electrode distance, we did not see a significant drop in performance as a function of electrode spacing when evaluated with a two-factor ANOVA (subject × electrode distance). Post hoc pairwise analyses (t-test, uncorrected for multiple comparisons) comparing discrimination performance between 800 and 2400 separations and the 1131 and 2400 separations were also below significance for both subjects. 
Experiment 4: Pulse Timing Effects for a Single Electrode—Brightness Matching
As described, one possibility is that discrimination between pseudosynchronous and asynchronous stimulation might be mediated by differences in the response to the two stimulation patterns within individual neurons lying between two electrodes that receive direct stimulation from two (or more) electrodes. It is known that the intensity of electric fields decrease as a function of distance from the electrode. 34,35 For example, suppose electrode 1 was stimulated before electrode 2. A neuron lying closer to electrode 1 would be stimulated by a high-amplitude pulse, followed by a low-amplitude pulse, whereas a neuron lying closer to electrode 2 than to electrode 1 would be stimulated by a low-amplitude pulse followed by a high-amplitude pulse. If low-high versus high-low pulse pairs have a differential effect on driving cell activity, then these different pulse patterns might result in perceptually distinguishable responses even at rates well above the CFF. According to this model, it is only those neurons that happen to receive equal amplitude stimulation from a pair of electrodes (the location of these neurons will depend on the relative amplitudes of current on each electrode and factors such as the height of the electrodes from the retinal surface) that will be insensitive to the timing of stimulation across that electrode pair. 
According to this model, differences between low-high versus high-low pulse pairs should also be distinguishable when presented on a single electrode. To test whether this was, in fact, the case, we carried out subjective brightness matching (on a single electrode) between a standard that consisted of pulse pairs of equal amplitude and test stimuli consisting of either low-high versus high-low pulse pairs. The standard pulse pair consisted of a pair of biphasic pulses of equal amplitude (304.8 μA for every electrode). The brightness of this standard was compared to the brightness of two test stimuli, a low- followed by high-amplitude biphasic pulse pair or a high- followed by a low-amplitude biphasic pulse pair. All pulses had 0.075-ms pulse width and a 0.075-delay or interpulse interval. At the start of the experiment, these test pulse pairs were set to have relatively the same total charge as the standard stimulus. The low-amplitude pulse had half the amplitude of the standard stimulus (151.2 μA), and the high-amplitude test pulse was set to have 1.5 times the charge of the standard stimulus (455.1 μA). We used a two-interval, forced-choice procedure, as described, to adjust the charge of the test stimuli based on subject responses. Increases or decreases in test stimuli amplitude were carried out on a logarithmic scale such that a step increase across the pulse pair would lead to an increase of 167.1 μA on the low-amplitude pulse and 503.1 μA on the high-amplitude pulse (an increase of 16 and 48 μA, respectively). Data were collected on three individual electrodes in S06. 
As shown in Figure 5, we found a difference in the charge needed to obtain a brightness match between high-low versus low-high pulse pairs. There was no significant difference in the amount of charge needed to match low-high pulse pairs to the standard containing pulse pairs of equal amplitude (two-tailed t-test; P > 0.05). In contrast, high-amplitude followed by low-amplitude pulses required significantly less charge (∼10% less) to appear as bright as the standard (two-tailed t-test; P < 0.05). 
Figure 5.
 
Experiment 4: Subject performance discriminating different timing patterns on a single electrode. Subjects compared the brightness of a standard (two pulses of equal amplitude) to two test stimuli (a low- followed by high-amplitude pulse pair or a high- followed by low-amplitude pulse pair). Plotted is the cathodic charge required to reach the point of equibrightness between standard and test stimuli. Standard errors were calculated across runs, and one run was carried out on each electrode pair.
Figure 5.
 
Experiment 4: Subject performance discriminating different timing patterns on a single electrode. Subjects compared the brightness of a standard (two pulses of equal amplitude) to two test stimuli (a low- followed by high-amplitude pulse pair or a high- followed by low-amplitude pulse pair). Plotted is the cathodic charge required to reach the point of equibrightness between standard and test stimuli. Standard errors were calculated across runs, and one run was carried out on each electrode pair.
In a series of previous experiments we evaluated thresholds as a function of the temporal properties of stimulation for a wide variety of pulse trains. 16 Consistent with these previous findings, these results are consistent with the notion of rapid adaptation across pulses that seems to be proportional to charge accumulation. Such a mechanism is consistent with earlier experiments showing rapid adaptation effects in rabbit retina. 36  
Experiment 5: Clockwise versus Counterclockwise Stimulation
If performance in experiments 2 and 3 were mediated by differences in responses within neurons lying between two electrodes that received direct stimulation from two (or more) electrodes, then we might predict that subjects would be able to differentiate between clockwise and counterclockwise stimulation patterns even at rates well above the critical fusion frequency. 
In the clockwise pattern shown in Figure 6, electrode B2 is stimulated before electrode A2, whereas in the counterclockwise pattern, electrode A2 is stimulated before electrode B2. Neurons lying between B2 and A2 but closer to electrode B2, are, therefore, stimulated by high- followed by low-amplitude pulses in the case of the clockwise pattern and low- followed by high-amplitude pulses in the counterclockwise pattern. Given the results of experiment 4 suggesting that neurons are less sensitive to low-high than to high-low patterns of stimulation, we might then expect local discriminable differences in brightness across the two patterns. 
Figure 6.
 
An illustration of how the expected percept might differ for clockwise (A) versus counterclockwise (B) stimulation if the response of a neuron at a given point in time is suppressed by previous stimulation. Here we used the simple model: B t1t2 = B{C t1 + [C t1/(C + C t1)]C t2}, where B is apparent brightness across the retina, C t1 and C t2 are the current fields at each point in time, and C is a constant. We assumed equal electrode sizes (unlike the checkerboard array implanted in the subjects tested here). This model is simply used as an intuitive illustration because actual interactions over time are demonstrably more complex. 16
Figure 6.
 
An illustration of how the expected percept might differ for clockwise (A) versus counterclockwise (B) stimulation if the response of a neuron at a given point in time is suppressed by previous stimulation. Here we used the simple model: B t1t2 = B{C t1 + [C t1/(C + C t1)]C t2}, where B is apparent brightness across the retina, C t1 and C t2 are the current fields at each point in time, and C is a constant. We assumed equal electrode sizes (unlike the checkerboard array implanted in the subjects tested here). This model is simply used as an intuitive illustration because actual interactions over time are demonstrably more complex. 16
We asked subjects to discriminate between clockwise and counterclockwise stimulation patterns, presented on groups of four electrodes. In the clockwise stimulus, phase-shifts across pulses were presented across electrodes in a sequentially clockwise order, whereas in the counterclockwise stimulus, identical pulses were presented in the reverse order. Pulse train frequencies were 40 or 80 Hz, corresponding to phase-shifts of 6 or 3 ms between electrodes (Fig. 7A). Our stimulation patterns were designed so that the first electrode was always the same, regardless of whether stimulation was clockwise or counterclockwise. It should be noted that these interactions are likely to occur even at subthreshold levels of stimulation (i.e., electrodes that produce nonoverlapping phosphenes may nonetheless demonstrate spatiotemporal interactions). 
Figure 7.
 
(A) Clockwise and counterclockwise stimulation. Pulse train stimuli were presented on groups of four electrodes in either clockwise or counterclockwise rotating order, and subjects were asked to differentiate between the two patterns using a same-different task. All electrodes had 800-μm center-to-center separation. (B) Subject performance experiment 4. Subjects successfully discriminated between clockwise and counterclockwise stimuli. SEMs were calculated across runs, and three runs were carried out for each electrode group (total, 15 runs).
Figure 7.
 
(A) Clockwise and counterclockwise stimulation. Pulse train stimuli were presented on groups of four electrodes in either clockwise or counterclockwise rotating order, and subjects were asked to differentiate between the two patterns using a same-different task. All electrodes had 800-μm center-to-center separation. (B) Subject performance experiment 4. Subjects successfully discriminated between clockwise and counterclockwise stimuli. SEMs were calculated across runs, and three runs were carried out for each electrode group (total, 15 runs).
As previously described for experiments 1 to 3, stimulation on each individual electrode was suprathreshold, with current levels adjusted so that the percept across each individual electrode of the group of four was roughly brightness-matched. Performance was measured using a two-interval, same-different paradigm by which subjects were asked to say whether the two stimulation patterns presented in the two intervals were the same or different. For the two subjects, both 40- and 80-Hz stimuli were evaluated on five 4-electrode groups. Three hundred trials were run for each of these conditions, for each group of electrodes. Thus, each bar in the graph of Figure 7 consists of 1500 trials. 
Subjects were able to reliably discriminate between clockwise and counterclockwise stimulation (Fig. 7B; 1-tailed t-test, P < 0.05 for all frequencies and subjects). A three-factor ANOVA (subject × electrode group × frequency) found no difference between subjects or frequencies. However, there was a significant difference in performance across electrode groups (P < 0.01). Percentage correct and d-prime values are reported in Table 1
Experiment 6: Electrode Order and Brightness
If performance in experiments 2 to 5 is mediated by differences in response within individual neurons lying between two electrodes that receive direct stimulation from two (or more) electrodes, then performance differentiating clockwise and counterclockwise patterns should not be based on any change in mean brightness. Therefore, we measured the amount of current required to brightness match stimulation patterns that differed in the order in which stimulation was presented on electrode pairs. 
We used the standard two-interval, forced-choice brightness-matching procedure described. All pulse train stimuli were 500-s pulse trains at 50 Hz using biphasic pulses of 0.45 ms per phase. Each trial contained two intervals with either time-synched pulse trains on each electrode or phase-shifted pulse trains on each electrode (Fig. 8A). The phase-shift was 0.075, 0.375, 1.8, or 9 ms (note that for the 0.075-ms and 0.375-ms phase-shifted stimuli, there were likely to be electric field interactions across electrodes). Subjects were asked to report which interval contained the brighter stimulus. A one-up, one-down staircase method was used to adjust the amplitude of the phase-shifted pulse train based on the observer's response. Increases or decreases in amplitude as a function of the staircase were applied to both pulse trains on the electrode pair. 
Figure 8.
 
(A) Brightness matching as a function of electrode order. Subjects compared the brightness of a standard (500-ms pulse trains, 50 Hz, 0.45-ms pulse width) where pulse trains were synchronous across the electrode pair to test stimuli (identical with the standard except for a phase-shift across the electrode pair). Speaker symbols represent a prestimulus auditory cue to prime the subject that a stimulus is about to be presented. Either E1 or E2 was stimulated first. (B, C) Normalized charge required to match brightness between synchronous and asynchronous stimuli in S05 and S06. The x-axis represents the phase-shift when either E1 or E2 was stimulated first, and the y-axis represents normalized (based on the synchronous stimulus) charge necessary to maintain equibrightness. S06 was unavailable for testing in the E2-E1 condition because of limited experimental time.
Figure 8.
 
(A) Brightness matching as a function of electrode order. Subjects compared the brightness of a standard (500-ms pulse trains, 50 Hz, 0.45-ms pulse width) where pulse trains were synchronous across the electrode pair to test stimuli (identical with the standard except for a phase-shift across the electrode pair). Speaker symbols represent a prestimulus auditory cue to prime the subject that a stimulus is about to be presented. Either E1 or E2 was stimulated first. (B, C) Normalized charge required to match brightness between synchronous and asynchronous stimuli in S05 and S06. The x-axis represents the phase-shift when either E1 or E2 was stimulated first, and the y-axis represents normalized (based on the synchronous stimulus) charge necessary to maintain equibrightness. S06 was unavailable for testing in the E2-E1 condition because of limited experimental time.
We measured perceived brightness for E1-first and E2-first stimulation (in separate blocks of trials) to evaluate whether electrode stimulation order had an effect on perceived brightness (Fig. 8B). Each brightness match was based on a minimum of 100 trials. A cumulative normal was used to find the point of subjective equibrightness, and error bars were again estimated using an adaptive sampling Monte-Carlo simulation. 28 Each psychometric function was inspected to ensure that an adequate fit was obtained, and data were recollected if fits were inadequate (based either on the estimated error or visual inspection). Data were collected on three electrode pairs in S05 and two electrode pair in subject S06. 
As expected given subjects' subjective reports in experiment 1, we found that the brightness of electrode pair stimuli was substantially brighter when pulses were presented synchronously than when pulses were phase shifted across electrodes (Fig. 8B). The amount of charge necessary to maintain brightness equal to that of the synchronous stimulus increased as a function of the phase-shift across electrodes; for a 9-ms phase shift, 20% more charge was required to match the synchronous stimulus. 
Consistent with our notion that performance discriminating between clockwise and counterclockwise stimulation patterns is not based on any mean change in brightness, we found that there was no significant difference in the amount of current required to match the standard between E1-first and E2-first stimuli (Fig. 8C; P > 0.05; one-tailed t-test). 
Discussion
We show here that changes in spatiotemporal stimulation patterns well above the critical flicker fusion limit do affect perception: subjects can discriminate stimuli that are differentiated by phase shifts of 12 ms or less (corresponding to frequencies of 80 Hz or higher), even when electric field interactions are removed. 
Experiment 1 showed that subjects could differentiate between synchronous and asynchronous stimulation patterns with high accuracy. In experiment 2, we found that subjects could still differentiate between pseudosynchronous and asynchronous stimulation, but there was a significant drop in performance. This drop in performance between experiments 1 and 2 suggests that there were significant electrical field interactions under the condition of simultaneous stimulation used in experiment 1. Experiment 6 further confirmed this result by demonstrating that synchronous stimulation results in brighter percepts than asynchronous stimulation. In our experiment, the asynchronous pulse pattern required nearly 20% more charge than the synchronous pattern to appear matched in brightness. 
In experiments 2 to 6, we examined spatiotemporal interactions across electrodes once electric field interactions had been eliminated. Experiment 2 demonstrated that subjects could differentiate between pseudosynchronous and asynchronous stimulation; removing electrical field interactions was not sufficient to cause the percept elicited by a given electrode to be independent of stimulation by other electrodes. 
One explanation is that these pulse patterns create local differences in brightness mediated by individual neurons that lie intermediate between electrodes. We carried out four experiments to further test this hypothesis. Spatiotemporal interactions decreased with electrode separation (experiment 3). Timing differences analogous to those tested in experiments 1 to 3 were perceptually distinguishable on a single electrode: a high-low pattern of stimulation resulted in a brighter percept than a low-high pattern of stimulation (experiment 4). In experiment 5 we demonstrated that subjects were able to distinguish clockwise from counterclockwise stimulation, and experiment 6 demonstrated that these judgments were unlikely to be based on overall (rather than local) differences in brightness between the two stimuli. 
However, it is possible that neuronal lateral connections or cortical sensitivity to precise timing patterns across space may also play a role. Recent evidence suggests very fine temporal sensitivity within lateral connections mediated by wide-field amacrine cells. These connections can span up to many millimeters within the retina. 37,38 These connections, therefore, have many of the qualities required to mediate our subjects' ability to discriminate between patterns differentiated by extremely fine temporal information across relatively wide regions of space. The sensitivity to clockwise versus counterclockwise stimulation demonstrated in experiment 5 is harder to explain in terms of retinal lateral connections. However, it should be noted that though current levels were chosen to roughly brightness-match the percepts across each of the four electrodes in the group, this brightness matching was not perfect. Moreover, electrodes differ in their height from the retinal surface, which presumably means that the extent of current spread on the retinal surface is different across electrodes. Finally, it is likely that there are inhomogeneities in retinal wiring across the 3 mm covered by the electrode array. These inhomogeneities across electrodes and the retinal surface might conceivably produce perceptually distinguishable patterns for clockwise versus counterclockwise stimulation. 
It is also possible that the ability to differentiate these patterns is mediated by cortical sensitivity to precise timing information. It is likely that our stimulation patterns created very precise spatiotemporal patterns of spiking activity in the retina. Stimulation using extremely short pulses (∼0.1 ms) results in precise single spikes within ganglion cells that are phase-locked to the pulses with a precision of <0.7 ms, 39,40 and presynaptic-driven spiking is abolished with stimulation frequencies above 10 Hz. 41,42 If precise timing information resulting from direct stimulation of ganglion cells is passed from retina to cortex, it is possible that the sensitivity to pulse timing across electrodes is the result of a cortical mechanism sensitive to spatiotemporal firing patterns originating in the retina. 
There is some evidence that the cortex may be sensitive to very high temporal frequencies. For example, it has been suggested that the representation of objects and contours across the visual field is at least partially mediated by synchronous neuronal activity within those neurons representing the contour. 4351 Synchronous firing at high temporal frequencies has been recorded for contour stimuli within both the retina 37,5255 and the visual cortex. 50,5660 However, it is still not known whether these synchronous firing patterns have functional importance. 
Psychophysically, there is evidence that grouping of visual stimuli occurs based on temporal structure at frequencies up to approximately 35 Hz, 61,62 but it is possible that such stimuli still contain visible flicker, 63 motion information, or both. 6470 Although it has not yet been clearly shown that frequencies beyond the CFF mediate grouping performance, there is some evidence of orientation specific (implying a cortical substrate) adaptation to temporal frequencies above the CFF. 25  
Modeling percepts in visual prosthetic devices would be computationally simpler if it were possible to create spatiotemporally independent electrodes. However, the interactions described here (both between synchronous and nonsynchronous stimulation and between different patterns of nonsimultaneous stimulation) do offer the potential for significant perceptual flexibility. Simply by altering the order of stimulation, it is possible to create distinct percepts on a given set of electrodes. 
Supplementary Materials
Footnotes
 Supported by Second Sight Medical Products, Inc., the Fletcher Jones Foundation, National Institutes of Health Grant NEI EY012893, and Research to Prevent Blindness.
Footnotes
 Disclosure: A. Horsager, Second Sight Medical Products Inc. (F), P; R.J. Greenberg, Second Sight Medical Products Inc. (E, I), P; I. Fine, P
The authors thank Matthew McMahon and Mark Humayun for their valuable feedback and contributions. 
References
Chader GJ . Animal models in research on retinal degenerations: past progress and future hope. Vis Res. 2002; 42: 393–399. [CrossRef] [PubMed]
Marc RE Jones BW . Retinal remodeling in inherited photoreceptor degenerations. Mol Neurobiol. 2003; 28: 139–147. [CrossRef] [PubMed]
Jones BW Watt CB Marc RE . Retinal remodelling. Clin Exp Optom. 2005; 88: 282–291. [CrossRef] [PubMed]
Santos A Humayun MS de Juan EJr . Preservation of the inner retina in retinitis pigmentosa: a morphometric analysis. Arch Ophthalmol. 1997; 115: 511–515. [CrossRef] [PubMed]
Humayun MS Prince M de Juan EJr . Morphometric analysis of the extramacular retina from postmortem eyes with retinitis pigmentosa. Invest Ophthalmol Vis Sci. 1999; 40: 143–148. [PubMed]
Mazzoni F Novelli E Strettoi E . Retinal ganglion cells survive and maintain normal dendritic morphology in a mouse model of inherited photoreceptor degeneration. J Neurosci. 2008; 28: 14282–14292. [CrossRef] [PubMed]
Lagali PS Balya D Awatramani GB . Light-activated channels targeted to ON bipolar cells restore visual function in retinal degeneration. Nat Neurosci. 2008; 11: 667–675. [CrossRef] [PubMed]
Margolis DJ Newkirk G Euler T Detwiler PB . Functional stability of retinal ganglion cells after degeneration-induced changes in synaptic input. J Neurosci. 2008; 28: 6526–6536. [CrossRef] [PubMed]
Stasheff SF . Emergence of sustained spontaneous hyperactivity and temporary preservation of OFF responses in ganglion cells of the retinal degeneration (rd1) mouse. J Neurophysiol. 2008; 99: 1408–1421. [CrossRef] [PubMed]
Humayun MS de Juan EJr Weiland JD . Pattern electrical stimulation of the human retina. Vis Res. 1999; 39: 2569–2576. [CrossRef] [PubMed]
Rizzo JFIII Wyatt J Loewenstein J Kelly S Shire D . Perceptual efficacy of electrical stimulation of human retina with a microelectrode array during short-term surgical trials. Invest Ophthalmol Vis Sci. 2003; 44: 5362–5369. [CrossRef] [PubMed]
Weiland JD Yanai D Mahadevappa M . Visual task performance in blind humans with retinal prosthetic implants. Proceedings of the 26th Annual International Conference of the IEEE EMBS. Conf Proc IEEE Eng Med Biol Sci. 2004; 6: 4172–4173.
Yanai D Weiland JD Mahadevappa M Greenberg RJ Fine I Humayun MS . Visual performance using a retinal prosthesis in three subjects with retinitis pigmentosa. J Ophthalmol. 2007; 143: 820–827.
Zrenner E . Subretinal implants for the restitution of vision in blind patients. Presented at the annual meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, FL. http://www.abstractsonline.com . Accessed November 12, 2009.
de Balthasar C Patel S Roy A . Factors affecting perceptual thresholds in epiretinal prostheses. Invest Ophthalmol Vis Sci. 2008; 49: 2303–2314. [CrossRef] [PubMed]
Horsager A Greenwald SH Weiland JD . Predicting visual sensitivity in retinal prosthesis patients. Invest Ophthalmol Vis Sci. 2009; 50: 1483–1491. [CrossRef] [PubMed]
Boex C de Balthasar C Kos MI Pelizzone M . Electrical field interactions in different cochlear implant systems. J Acoust Soc Am. 2003; 114: 2049–2057. [CrossRef] [PubMed]
Stickney GS Loizou PC Mishra LN Assmann PF Shannon RV Opie JM . Effects of electrode design and configuration on channel interactions. Hear Res. 2006; 211: 33–45. [CrossRef] [PubMed]
de Balthasar C Boex C Cosendai G Valentini G Sigrist A Pelizzone M . Channel interactions with high-rate biphasic electrical stimulation in cochlear implant subjects. Hear Res. 2003; 182: 77–87. [CrossRef] [PubMed]
Wilson BS Finley CC Lawson DT Wolford RD Zerbi M . Design and evaluation of a continuous interleaved sampling (CIS) processing strategy for multichannel cochlear implants. J Rehabil Res Dev. 1993; 30: 110. [PubMed]
Donaldson GS Kreft HA Litvak L . Place-pitch discrimination of single- versus dual-electrode stimuli by cochlear implant users (L). J Acoust Soc Am. 2005; 118: 623–626. [CrossRef] [PubMed]
McDermott HJ McKay CM . Pitch ranking with nonsimultaneous dual-electrode electrical stimulation of the cochlea. J Acoust Soc Am. 1994; 96: 155–162. [CrossRef] [PubMed]
Oyster CW . The Human Eye: Structure and Function. 1st ed. Sunderland, MA: Sinauer Associates, Inc.; 1999: 766.
Hecht S Verrijp CD . The influence of intensity, color and retinal location on the fusion frequency of intermittent illumination. Proc Natl Acad Sci U S A. 1933; 19: 522–535. [CrossRef] [PubMed]
Shady S MacLeod DI Fisher HS . Adaptation from invisible flicker. Proc Natl Acad Sci U S A. 2004; 101: 5170–5173. [CrossRef] [PubMed]
Humayun MS Weiland JD Fujii GY . Visual perception in a blind subject with a chronic microelectronic retinal prosthesis. Vis Res. 2003; 43: 2573–2581. [CrossRef] [PubMed]
Mahadevappa M Weiland JD Yanai D Fine I Greenberg RJ Humayun MS . Perceptual thresholds and electrode impedance in three retinal prosthesis subjects. IEEE Trans Neural Syst Rehabil Eng. 2005; 13: 201–206. [CrossRef] [PubMed]
Wichmann FA Hill NJ . The psychometric function, II: bootstrap-based confidence intervals and sampling. Percept Psychophys. 2001; 63: 1314–1329. [CrossRef] [PubMed]
Green DG Swets JA . Signal Detection Theory and Psychophysics. New York: John Wiley & Sons; 1988.
Fine I Jacobs RA . Comparing perceptual learning across tasks: a review. J Vis. 2002; 2: 190–203. [PubMed]
Mao BQ MacLeish PR Victor JD . The intrinsic dynamics of retinal bipolar cells isolated from tiger salamander. Vis Neurosci. 1998; 15: 425–438. [CrossRef] [PubMed]
Coleman PA Miller RF . Measurement of passive membrane parameters with whole-cell recording from neurons in the intact amphibian retina. J Neurophysiol. 1989; 61: 218–230. [PubMed]
O'Brien BJ Isayama T Richardson R Berson DM . Intrinsic physiological properties of cat retinal ganglion cells. J Physiol. 2002; 538: 787–802. [CrossRef] [PubMed]
Wiley JD Webster JG . Distributed equivalent-circuit model for circular dispersive electrodes. IEEE Trans Biomed Eng. 1982; 29: 385–389. [CrossRef] [PubMed]
Cottaris NP Elfar SD . How the retinal network reacts to epiretinal stimulation to form the prosthetic visual input to the cortex. J Neural Eng. 2005; 2: S74–S90. [CrossRef] [PubMed]
Jensen RJ Rizzo JF3rd . Responses of ganglion cells to repetitive electrical stimulation of the retina. J Neural Eng, 2007; 4: S1–S6. [CrossRef]
Amthor FR Tootle JS Grzywacz NM . Stimulus-dependent correlated firing in directionally selective retinal ganglion cells. Vis Neurosci. 2005; 22: 769–787. [PubMed]
Baccus SA . Timing and computation in inner retinal circuitry. Annu Rev Physiol. 2007; 69: 271–290. [CrossRef] [PubMed]
Fried SI Hsueh HA Werblin FS . A method for generating precise temporal patterns of retinal spiking using prosthetic stimulation. J Neurophysiol. 2006; 95: 970–978. [CrossRef] [PubMed]
Sekirnjak C Hottowy P Sher A Dabrowski W Litke AM Chichilnisky EJ . High-resolution electrical stimulation of primate retina for epiretinal implant design. J Neurosci. 2008; 28: 4446–4456. [CrossRef] [PubMed]
Sekirnjak C Hottowy P Sher A Dabrowski W Litke AM Chichilnisky EJ . Electrical stimulation of mammalian retinal ganglion cells with multielectrode arrays. J Neurophysiol. 2006; 95: 3311–3327. [CrossRef] [PubMed]
Ahuja AK Behrend MR Kuroda M Humayun MS Weiland JD . An in vitro model of a retinal prosthesis. IEEE Trans Biomed Eng. 2008; 55: 1744–1753. [CrossRef] [PubMed]
Milner PM . A model for visual shape recognition. Psychol Rev. 1974; 81: 521–535. [CrossRef] [PubMed]
Reitboeck HJ . Fiber microelectrodes for electrophysiological recordings. J Neurosci Methods. 1983; 8: 249–262. [CrossRef] [PubMed]
von der Malsburg C Schneider W . A neural cocktail-party processor. Biol Cybern. 1986; 54: 29–40. [CrossRef] [PubMed]
Gray CM Konig P Engel AK Singer W . Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature. 1989; 338: 334–337. [CrossRef] [PubMed]
Konig P Engel AK Singer W . Relation between oscillatory activity and long-range synchronization in cat visual cortex. Proc Natl Acad Sci U S A. 1995; 92: 290–294. [CrossRef] [PubMed]
Fries P Roelfsema PR Engel AK Konig P Singer W . Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry. Proc Natl Acad Sci U S A. 1997; 94: 12699–12704. [CrossRef] [PubMed]
Roelfsema PR Engel AK Konig P Singer W . Visuomotor integration is associated with zero time-lag synchronization among cortical areas. Nature. 1997; 385: 157–161. [CrossRef] [PubMed]
Brosch M Bauer R Eckhorn R . Stimulus-dependent modulations of correlated high-frequency oscillations in cat visual cortex. Cereb Cortex. 1997; 7: 70–76. [CrossRef] [PubMed]
Singer W . Neuronal synchrony: a versatile code for the definition of relations? Neuron. 1999; 24: 49–65, 111–125. [CrossRef] [PubMed]
Mastronarde DN . Interactions between ganglion cells in cat retina. J Neurophysiol. 1983; 49: 350–365. [PubMed]
Mastronarde DN . Correlated firing of cat retinal ganglion cells, II: responses of X- and Y-cells to single quantal events. J Neurophysiol. 1983; 49: 325–349. [PubMed]
Mastronarde DN . Correlated firing of cat retinal ganglion cells, I: spontaneously active inputs to X- and Y-cells. J Neurophysiol. 1983; 49: 303–324. [PubMed]
Meister M Lagnado L Baylor DA . Concerted signaling by retinal ganglion cells. Science. 1995; 270: 1207–1210. [CrossRef] [PubMed]
Kreiter AK Singer W . Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. J Neurosci. 1996; 16: 2381–2396. [PubMed]
Singer W Kreiter AK Engel AK Fries P Roelfsema PR Volgushev M . Precise timing of neuronal discharges within and across cortical areas: implications for synaptic transmission. J Physiol Paris. 1996; 90: 221–222. [CrossRef] [PubMed]
Fries P Neuenschwander S Engel AK Goebel R Singer W . Rapid feature selective neuronal synchronization through correlated latency shifting. Nat Neurosci. 2001; 4: 194–200. [CrossRef] [PubMed]
Fries P Reynolds JH Rorie AE Desimone R . Modulation of oscillatory neuronal synchronization by selective visual attention. Science. 2001; 291: 1560–1563. [CrossRef] [PubMed]
Gabriel A Eckhorn R . A multi-channel correlation method detects traveling gamma-waves in monkey visual cortex. J Neurosci Methods. 2003; 131: 171–184. [CrossRef] [PubMed]
Usher M Donnelly N . Visual synchrony affects binding and segmentation in perception. Nature. 1998; 394: 179–182. [CrossRef] [PubMed]
Lee SH Blake R . Visual form created solely from temporal structure (see comment). Science. 1999; 284: 1165–1168. [CrossRef] [PubMed]
Graham CH . Vision and Visual Perception. 1st ed. New York: John Wiley & Sons, Inc.; 1965: 637.
Fahle M Poggio T . Visual hyperacuity:spatiotemporal interpolation in human vision. Proc R Soc Lond B Biol Sci. 1981; 213: 451–477. [CrossRef] [PubMed]
Heeger DJ . Model for the extraction of image flow. J Opt Soc Am A. 1987; 4: 1455–1471. [CrossRef] [PubMed]
Grzywacz NM Amthor FR Mistler LA . Applicability of quadratic and threshold models to motion discrimination in the rabbit retina. Biol Cybern. 1990; 64: 41–49. [CrossRef] [PubMed]
Grzywacz NM Yuille AL . A model for the estimate of local image velocity by cells in the visual cortex. Proc R Soc Lond B Biol Sci. 1990; 239: 129–161. [CrossRef] [PubMed]
Nowlan SJ Sejnowski TJ . A selection model for motion processing in area MT of primates. J Neurosci. 1995; 15: 1195–1214. [PubMed]
Simoncelli EP Heeger DJ . A model of neuronal responses in visual area MT. Vis Res. 1998; 38: 743–761. [CrossRef] [PubMed]
Bair W Movshon JA . Adaptive temporal integration of motion in direction-selective neurons in macaque visual cortex. J Neurosci. 2004; 24: 7305–7323. [CrossRef] [PubMed]
Figure 1.
 
(A) Electrode array. The electrode array consists of 260- or 520-μm electrodes arranged in a checkerboard pattern, with center-to-center separation of 800 μm. The entire array covers approximately 2.9 × 2.9-mm of retinal space, subtending approximately 10° of visual angle. Electrodes are designated by a letter/number combination (A1-D4). (B) Prosthesis. Pulse sequences are programmed using custom software run on a personal laptop computer, which communicates stimulus parameters to an external visual processing unit (not shown). Signal and power information is then passed through an external inductive coupling device (not shown) that attaches magnetically to a subdermal coil implanted in the subject's temporal skull. This signal is then sent through a parallel system of wires to the epiretinally implanted electrode array. Note that the power and signal information can be independently controlled for each electrode. (A) and (B) Adapted from Horsager A, Greenwald SH, Weiland JD, et al. Predicting visual sensitivity in retinal prosthesis patients. Invest Ophthalmol Vis Sci. 2009;50:1483–1491. (C) Pulse train. All pulse train stimuli consisted of biphasic, cathodic-first, charge-balanced, square-wave pulses. The cathodic and anodic phases of each biphasic pulse were 0.075 ms in duration, with a 0.075 ms interphase delay. Stimulation on each electrode in the group of four electrodes consisted of a 500-ms pulse train of variable frequency. The example shown here is of a 20-Hz pulse train of 500 ms duration.
Figure 1.
 
(A) Electrode array. The electrode array consists of 260- or 520-μm electrodes arranged in a checkerboard pattern, with center-to-center separation of 800 μm. The entire array covers approximately 2.9 × 2.9-mm of retinal space, subtending approximately 10° of visual angle. Electrodes are designated by a letter/number combination (A1-D4). (B) Prosthesis. Pulse sequences are programmed using custom software run on a personal laptop computer, which communicates stimulus parameters to an external visual processing unit (not shown). Signal and power information is then passed through an external inductive coupling device (not shown) that attaches magnetically to a subdermal coil implanted in the subject's temporal skull. This signal is then sent through a parallel system of wires to the epiretinally implanted electrode array. Note that the power and signal information can be independently controlled for each electrode. (A) and (B) Adapted from Horsager A, Greenwald SH, Weiland JD, et al. Predicting visual sensitivity in retinal prosthesis patients. Invest Ophthalmol Vis Sci. 2009;50:1483–1491. (C) Pulse train. All pulse train stimuli consisted of biphasic, cathodic-first, charge-balanced, square-wave pulses. The cathodic and anodic phases of each biphasic pulse were 0.075 ms in duration, with a 0.075 ms interphase delay. Stimulation on each electrode in the group of four electrodes consisted of a 500-ms pulse train of variable frequency. The example shown here is of a 20-Hz pulse train of 500 ms duration.
Figure 2.
 
(A) Experiment 1: Synchronous versus asynchronous stimulation. Subjects discriminated between synchronous and asynchronous pulse train stimuli across groups of four electrodes using a same-different task. (B) Subject performance experiment 1. Percentage correct is shown for frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). 50% is chance performance. SEMs were calculated by taking each run as a separate measure, with a single run carried out on each electrode group.
Figure 2.
 
(A) Experiment 1: Synchronous versus asynchronous stimulation. Subjects discriminated between synchronous and asynchronous pulse train stimuli across groups of four electrodes using a same-different task. (B) Subject performance experiment 1. Percentage correct is shown for frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). 50% is chance performance. SEMs were calculated by taking each run as a separate measure, with a single run carried out on each electrode group.
Figure 3.
 
(A) Experiment 2: Discriminating between pseudosynchronous and asynchronous stimuli. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli on groups of four electrodes using a same-different task. All electrodes were neighboring (i.e., 800 μm center-to-center separation). (B) Subject performance in task. Subjects performed a same-different discrimination task in which they were asked to differentiate between pseudosynchronous and asynchronous stimuli presented at frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). The x-axis represents frequency and the y-axis represents the percentage correct. SEMs were calculated by taking each run as a separate measure, and either one or two runs were carried out on each electrode group. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 3.
 
(A) Experiment 2: Discriminating between pseudosynchronous and asynchronous stimuli. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli on groups of four electrodes using a same-different task. All electrodes were neighboring (i.e., 800 μm center-to-center separation). (B) Subject performance in task. Subjects performed a same-different discrimination task in which they were asked to differentiate between pseudosynchronous and asynchronous stimuli presented at frequencies of 20, 40, 80, and 160 Hz (corresponding to phase-shifts of 12, 6, 3, and 1.5 ms). The x-axis represents frequency and the y-axis represents the percentage correct. SEMs were calculated by taking each run as a separate measure, and either one or two runs were carried out on each electrode group. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 4.
 
Experiment 3: Subject performance as a function of electrode separation. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli (80 Hz) on groups of four electrodes using a same-different task. Electrodes were separated by center-to-center distances of 800, 1131, 1600, and 2400 μm. Example schematics of these electrode distances are presented on the y-axis. SEMs were calculated by taking each run as a separate measure. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 4.
 
Experiment 3: Subject performance as a function of electrode separation. Subjects discriminated between pseudosynchronous and asynchronous pulse train stimuli (80 Hz) on groups of four electrodes using a same-different task. Electrodes were separated by center-to-center distances of 800, 1131, 1600, and 2400 μm. Example schematics of these electrode distances are presented on the y-axis. SEMs were calculated by taking each run as a separate measure. P values represent the probability that a subject was performing above chance (dotted line at 50%).
Figure 5.
 
Experiment 4: Subject performance discriminating different timing patterns on a single electrode. Subjects compared the brightness of a standard (two pulses of equal amplitude) to two test stimuli (a low- followed by high-amplitude pulse pair or a high- followed by low-amplitude pulse pair). Plotted is the cathodic charge required to reach the point of equibrightness between standard and test stimuli. Standard errors were calculated across runs, and one run was carried out on each electrode pair.
Figure 5.
 
Experiment 4: Subject performance discriminating different timing patterns on a single electrode. Subjects compared the brightness of a standard (two pulses of equal amplitude) to two test stimuli (a low- followed by high-amplitude pulse pair or a high- followed by low-amplitude pulse pair). Plotted is the cathodic charge required to reach the point of equibrightness between standard and test stimuli. Standard errors were calculated across runs, and one run was carried out on each electrode pair.
Figure 6.
 
An illustration of how the expected percept might differ for clockwise (A) versus counterclockwise (B) stimulation if the response of a neuron at a given point in time is suppressed by previous stimulation. Here we used the simple model: B t1t2 = B{C t1 + [C t1/(C + C t1)]C t2}, where B is apparent brightness across the retina, C t1 and C t2 are the current fields at each point in time, and C is a constant. We assumed equal electrode sizes (unlike the checkerboard array implanted in the subjects tested here). This model is simply used as an intuitive illustration because actual interactions over time are demonstrably more complex. 16
Figure 6.
 
An illustration of how the expected percept might differ for clockwise (A) versus counterclockwise (B) stimulation if the response of a neuron at a given point in time is suppressed by previous stimulation. Here we used the simple model: B t1t2 = B{C t1 + [C t1/(C + C t1)]C t2}, where B is apparent brightness across the retina, C t1 and C t2 are the current fields at each point in time, and C is a constant. We assumed equal electrode sizes (unlike the checkerboard array implanted in the subjects tested here). This model is simply used as an intuitive illustration because actual interactions over time are demonstrably more complex. 16
Figure 7.
 
(A) Clockwise and counterclockwise stimulation. Pulse train stimuli were presented on groups of four electrodes in either clockwise or counterclockwise rotating order, and subjects were asked to differentiate between the two patterns using a same-different task. All electrodes had 800-μm center-to-center separation. (B) Subject performance experiment 4. Subjects successfully discriminated between clockwise and counterclockwise stimuli. SEMs were calculated across runs, and three runs were carried out for each electrode group (total, 15 runs).
Figure 7.
 
(A) Clockwise and counterclockwise stimulation. Pulse train stimuli were presented on groups of four electrodes in either clockwise or counterclockwise rotating order, and subjects were asked to differentiate between the two patterns using a same-different task. All electrodes had 800-μm center-to-center separation. (B) Subject performance experiment 4. Subjects successfully discriminated between clockwise and counterclockwise stimuli. SEMs were calculated across runs, and three runs were carried out for each electrode group (total, 15 runs).
Figure 8.
 
(A) Brightness matching as a function of electrode order. Subjects compared the brightness of a standard (500-ms pulse trains, 50 Hz, 0.45-ms pulse width) where pulse trains were synchronous across the electrode pair to test stimuli (identical with the standard except for a phase-shift across the electrode pair). Speaker symbols represent a prestimulus auditory cue to prime the subject that a stimulus is about to be presented. Either E1 or E2 was stimulated first. (B, C) Normalized charge required to match brightness between synchronous and asynchronous stimuli in S05 and S06. The x-axis represents the phase-shift when either E1 or E2 was stimulated first, and the y-axis represents normalized (based on the synchronous stimulus) charge necessary to maintain equibrightness. S06 was unavailable for testing in the E2-E1 condition because of limited experimental time.
Figure 8.
 
(A) Brightness matching as a function of electrode order. Subjects compared the brightness of a standard (500-ms pulse trains, 50 Hz, 0.45-ms pulse width) where pulse trains were synchronous across the electrode pair to test stimuli (identical with the standard except for a phase-shift across the electrode pair). Speaker symbols represent a prestimulus auditory cue to prime the subject that a stimulus is about to be presented. Either E1 or E2 was stimulated first. (B, C) Normalized charge required to match brightness between synchronous and asynchronous stimuli in S05 and S06. The x-axis represents the phase-shift when either E1 or E2 was stimulated first, and the y-axis represents normalized (based on the synchronous stimulus) charge necessary to maintain equibrightness. S06 was unavailable for testing in the E2-E1 condition because of limited experimental time.
Table 1.
 
Performance Values for Each Subject in Each of the Same/Different Tasks for Experiments 1, 2, and 5
Table 1.
 
Performance Values for Each Subject in Each of the Same/Different Tasks for Experiments 1, 2, and 5
Experiment Subject Frequency*
20 Hz (%/d′) 40 Hz (%/d′) 80 Hz (%/d′) 160 Hz (%/d′)
1 S05 83/2.65 94/3.74 95/3.90 93/3.59
S06 92/3.46 95/3.90 94/3.74 89/3.14
2 S05 58/1.04 64/1.45 62/1.31 55/0.80
S06 61/1.24 53/0.60 58/1.04 53/0.60
5 S05 NA 66/1.57 68/1.67 NA
S06 NA 63/1.38 64/1.45 NA
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×