Retinotopic to Spatiotopic Mapping in Blind Patients Implanted With the Argus II Retinal Prosthesis

Abstract

Purpose: To quantify the precision of mapping from retinotopic (retina-centered) to spatiotopic (world-centered) coordinates in blind humans implanted with a retinal prosthesis device. Additionally, to demonstrate that an eye tracker can be calibrated on sightless patients based on the percept from a visual implant.

Methods: We directly activated epiretinal electrodes to create retinotopic stimuli and recorded the location of the percept at world-based coordinates. In contrast to normal Argus II use where stimulation is a function of the captured scene's image, in this research we directly controlled the waveform in each electrode and measured the percept's location using a trackable handheld marker. For eye tracking, pupil images were recorded with a timestamp synchronized to the stimulation and marker positions.

Results: Remapping of the measured world locations to the position of the electrodes on the retina is feasible by accounting for eye orientation at the onset of stimulation. Transformation of pupil images to the eye's orientation (i.e., eye tracker calibration) can be done by solving for the variables that minimize the spread of the remapped retinal electrode locations. After mapping to retinal coordinates based on eye positions, the measured precision of pointing was 2° to 3°, which is comparable to open-loop pointing in sighted individuals.

Conclusions: The brain accurately maps the artificial vision induced by a retinal prosthesis based on instantaneous gaze position. Remapping based on eye position is feasible and will increase visual stability in prosthetic vision.

The observation that electrical stimulation of a retina that is no longer sensitive to light can create a visual percept,¹ led after several decades of research to a chronically implanted retinal prosthesis device.² For the percept to be useful, the electrical stimulation should convey information to the brain that is associated with the correct spatial location. It is desirable to stimulate various retinal locations that will create a percept in world coordinates corresponding to the receptive field of the retinal cell covered by the implanted electrodes. In the Argus II visual prosthesis, the imager (i.e., the camera) captures a real-time image of the scene and stimulates the retinal electrodes based on the visual information of the scene. 

Preferably, the image sensor should be implanted on the retina at the location of the photoreceptors. However, this requires both sensor miniaturization and assuring the sensor's survival in the eye's hostile environment. Coating the sensors to withstand the vitreous environment showed promising results in the lab,³ but has not yet been tested in humans. One straightforward solution is to use an external sensor and to only implant the stimulation unit in the eye. This is the solution adopted by the Argus II retinal prosthesis, which is currently the only Food and Drug administration–approved retinal prosthesis for the blind.⁴ The Argus II stimulates the retina based on an image acquired by a head-mounted camera installed on the nose-bridge of the glasses. In this configuration, there is an inherent disassociation between the image acquisition device and eye movements. Indeed, it was shown that eye positions affect the stimulation's perceptual location of the Argus II.⁵ Nevertheless, Argus II implantation is done as a routine medical procedure and the device has been shown to help in accomplishing activities of daily living (ADL) for blind people.⁶ To overcome the eye movements' disassociation, Argus II users are instructed to keep their eyes straight when using the system, thus aligning the pupillary and imager axes. 

Other types of retinal implants intend to steer the captured image in parallel with the gaze by placing an array of photodiodes in the subretinal space. In the Alpha-IMS implant,⁷ conversion of light to electrical waveform takes place within the eye, mimicking some aspects of the oculomotor behavior of sighted individuals.⁸ The PRIMA implant, which is currently in the animal testing phase, uses a pulsed high-intensity infrared image in front of the eyes to activate the subretinal electrodes.⁹ All these and future retinal implants, can only treat blindness due to retinal degeneration. Treating blindness due to diseases that affect the inner retina or the optic nerve will require stimulation at a higher location in the visual pathway, for example at the lateral geniculate nucleus (LGN),¹⁰ or at the visual cortex.¹¹ The topographic map within the brain is retina based¹² or retinotopic modulated by eye positions.¹³ Thus, cortical visual implants will stimulate areas in the brain that have a retinotopic map based on an image from an external sensor and will have the same disassociation between eye movements and the imager as the Argus II. 

Overcoming the disassociation of the eye movements and the world image can be achieved by steering the head-mounted camera's line of sight based on gaze position. Electrical stimulation at retina-centered coordinates should match the visual information at the visual field as captured by a wide field-of-view sensor and have it shifted based on the eyes' position. Current Argus II users scan the scene using head motion as opposed to the natural eye scanning of sighted individuals. Due to the relatively narrow field-of-view of the implanted array (18° × 11°) scanning is critical. Integration of an eye tracker will allow a more natural scanning by using eye movements. In addition, due to the narrow field-of-view of the implant, the saccade destination is not visible prior to the initiation of the saccade. This mechanism is similar to patients with peripheral visual loss that saccade to the blind area.¹⁴ We therefore needed to verify that the brain can accurately spatiotopically map the retinotopic artificial vision from the electrical stimulation and that we have a method to calibrate eye trackers for the blind. 

Eye tracker calibration is an essential step in measuring the position of the eyes relative to the orbit. In a mobile eye tracker, a small camera or cameras take images of the pupils and its locations are transformed to the eyes' orientations based on calibration data. In the calibration stage, pupil location within the tracker's camera is registered while the subject looks on predefined points on a two-dimensional plane, such as a computer monitor. During calibration, the tracker extracts the parameters that are needed to transform the pupil's location within the image to orient the eye in the orbit. Obviously, this cannot be done with blind patients. 

Herein, we designed an experiment that measures the reverse paradigm. We directly stimulated electrodes on the retina and recorded the location of the percept in world coordinates. By integrating an eye tracker in the experimental system, we are able to measure the accuracy of the mapping. Accurate mapping by the brain between stimulation locations on the retina to location in world coordinates is critical for retinal prosthesis devices that intend to place the sensor within the oculus. Our research has focused on whether the brain of totally blind patients implanted with a retinal prosthesis can map transient stimuli from retina- to world-based coordinates and to measure the accuracy of this mapping. The ability to transform the locations of the percept to retinal location based eye positions will validate the eye tracker for the blind. 

Materials and Methods

Experimental Procedure

Each session consisted of approximately 30 trials. In each trial, the array was stimulated once for a duration of 600 ms and the patients were instructed to place a marker, a red racquetball (57-mm diameter), at the location of the “light.” The patients placed the marker in space at an arm's length to indicate the location of the percept in world-centered coordinates (Fig. 1). 

Figure 1

 

Left panel: illustration of the head-mounted setup used to record pupil location with the position of the handheld marker. (1) Pupil's camera; (2) face-forwarding camera to record the location of the handheld marker; (3) the transmission coil of the Argus II that transmits power and data to the implant. Right panel: captured frame from the face-forwarding camera. (4) Red racquetball used as a hand-held trackable marker. The marker is placed in space by the patient to indicate the position of the percept relative to the patient's head.

In each session, we preselected three groups of electrodes to be measured. Each group of electrodes consisted of either four or eight neighboring electrodes and created a pattern of stimulation on the retina. For each trial, one of the session's three groups of electrodes was randomly selected. At the beginning of the trial, the patient was instructed either to gaze straight ahead or to move the pupils either to the right or the left. After the experimenter observed on the real-time pupil image that the pupils were at the requested position, the experimenter triggered the stimulation of the selected group of electrodes. A single group of electrodes was turned on in each trial. At the end of each trial, after the duration of the stimulation, the patient responded by placing a handheld marker in the space indicating the location of the percept. 

The Argus II System and Patients

The Argus II retinal prosthesis consists of an implanted array of 60 electrodes arranged in 10 × 6 rectangular layout, a video processing unit, and an external scene camera mounted on the glasses' nose-bridge. Camera images are acquired by the video processing unit that subsample the image to a low-resolution pattern. Stimulation waveforms at each of the 10 × 6 electrodes are calculated based on the visual content at the receptive field of the electrodes and are sent wirelessly to the implant. 

Two patients, one male and one female, whose blindness was caused by retinitis pigmentosa, were implanted with the Argus II retinal prosthesis. The implantation and rehabilitation of the Argus II were done as a routine medical procedure at the local medical center near the patients' residence. The patients did not have any observed abnormal oculomotor behavior such as nystagmus or strabismus. 

The patients were invited to participate in the psychophysics research study that took place at Second Sight Medical Products, Inc., in Sylmar, California. The clinical research study, including the eye tracking protocol, was approved by the Western Institutional Review Board (in the public domain, www.wirb.com). The informed consent was read to the patients and the patients signed the consent prior to the beginning of the experiment. 

Stimulation amplitudes on each electrode were set according to the programming used by the patient in daily activities with the Argus II. In order to achieve approximate uniform brightness for each electrode, the stimulation current for white was the threshold current of the electrode multiplied by a factor greater than 1. The factor was set during programming based on patient's preference. 

The surgical placement of the array on the retina is different between subjects. As part of the post-surgery programming of the Argus II, the rotation of the array is measured using a fundus image of the array on the retina.¹⁵ Due to the structure of the implant, the nominal value of the rotation is 45°. In programming for daily use, the rotation value is used to rotate the image of the camera to match the relative position between electrodes in the array. Herein, we used the rotation angle to present the layout of the array that matched the position for the specific patient in retina-centered coordinates. 

Experimental Setup

The experimental setup consisted of two computers that were powered by internal batteries. The first computer controlled the stimulation of the electrodes and the second recorded video images of the pupil and the locations of the handheld marker. Stimulation, pupil recording, and marker recording had synchronized timestamps. To generate a controlled real-time stimulation, the current experimental setup bypassed the system's camera and sent a computer generated image to the video processor of the Argus II. A binary large object (BLOB) image was created on the computer using MATLAB (MathWorks, Natick, MA, USA) and delivered to the Argus II system via the camera port using a VGA to NTSC adapter (VGA2VID; Startech.com, in the public domain, www.startech.com). The image was white (video level of 255) at the areas that correspond to the electrodes assigned in the specific trial and black (video level of 0) for the rest of the image. The image was fed to the video processor of the Argus II. The video processor subsampled the image and sent stimulation waveform to each electrode based on the video level in the corresponding area in the image generated by the computer. This setup allowed a real-time stimulation of the designated electrodes in each trial. 

Pupil images were acquired at 30 frames per second using a USB camera (HD-6000; Microsoft, Redmond, WA, USA) with an infrared (IR) pass filter to block the room lighting in the visible spectrum. The pupil was illuminated by an IR LED (SFH 4050-Z; Osram, Munich, Germany). A head-mounted front-facing camera (C930; Logitech, Newark, CA, USA) acquired images of the handheld marker at 30 frames per second. The front-facing camera used a wide lens having a focal length of 2.7 mm (DSL315B-650-F2.3; Sunex, Inc., Carlsbad, CA, USA). The pupil and the front facing cameras were mounted on a modified Pupil Lab frame (in the public domain, www.pupil-labs.com). The transmission coil of the Argus II was taped to the Pupil Lab frame. Figure 1, illustrates the cameras and transmission coil setup. To synchronize the stimulation onset with the pupil and marker recording, a copy of the image that was sent to the Argus II was sent to the recording computer using a VGA to USB adapter (DVI2USB 3.0; Epiphan, Palo Alto, CA, USA). 

Data Analysis

The center of the marker (i.e., the red ball) was found in each frame and the indicated location was the location in the frame that the ball was at rest. The position of the ball in the image was transformed to angular coordinates and was corrected for radial (fisheye) distortion of the lens of the front-facing camera. After the radial correction, each pixel in the front facing camera spanned an angular size of 0.0836°. 

To convert the pupil location in pixel coordinates to angular orientation of the eye relative to the orbit, we used the model in Equation 3. The six free variables were found by minimizing the spread (i.e. SD) of the retinal location of each group of electrodes. Minimization was carried out using MATLAB's (in the public domain, www.mathworks.com) fminsearch function that uses the Nelder-Mead simplex algorithm.¹⁶ The function was programmed to find the minimum of the sum of the variances of the locations of the two-dimensional retinal position of the three tested groups of electrodes as given by Equation 4. 

To verify that the conversion is a function of the pupil location at each trial, we ran the same procedure with random matchings of the trials between the pupil and marker locations. Table 1 shows that when “shuffling” the trial indices, we did not get a separation between the clusters that matched the stimulating retinal locations, indicating that convergence of the proposed algorithm relates to the pupil location in each trial. 

Table 1

 

Separation Between Clusters Per Davies Bouldin Index

 

Figure 2

 

The epiretinal array of 60 electrodes implanted in a patient's eye. Each electrode is wired to an implanted extraocular driver and can be individually controlled. In each trial, four neighboring electrodes were activated to create a percept. The group of electrodes that was stimulated in the different trials is marked by letters and is color-coded. The microtack that attaches the array to the retina is visible in the upper left corner of the array. The plastic tab in the lower right side of the array is used to place the array on the retina using the insertion tool during surgery.

Figure 3

 

Uncorrected open loop spatial responses. In each trial, the patient was instructed to shift the gaze either straight or to look to the right. Each symbol indicates the head-centered marked percept location as recorded by the head-mounted camera in a single trial. In the trial, one group of electrodes was turned on. Because the percept location is a combination of retinal location of the stimulation and gaze orientation, the percept locations are spread all over the visual field. The color codes of each symbol indicate the electrode group that was activated in the trial. Right arrow symbols mark trials in which the patient looked to the right, whereas left arrow symbols mark trials that the patient looked to the left. There are 33 independent measurements in the chart. The gray circles indicate the centroid of each group.

Figure 4

 

Retinal-angular position after mapping the recorded world-centered locations (Fig. 3) based on the pupil's location in each trial. The color codes and labels of each symbol indicate the electrode group that was activated in the trial and match the color of the electrodes in Figure 2. The direction of the arrows indicate the viewing direction at each trial as in Figure 3. There are 33 independent measurements in the chart. The gray circles indicate the centroid of each group. For comparison, the axis scale is identical to Figure 3. It can be seen that there is a clear separation between the electrodes and that the centroid locations match the layout of the electrodes in the implanted array.

Figure 5

 

Retinal-angular position after mapping the recorded world-centered locations (Fig. 3) based on the pupil's location in each trial at the time that the patient marked the perceived location. The color codes and labels of each symbol indicate the electrode group that was activated in the trial and is matched to the color code of the electrodes in Figure 2. There are 33 independent measurements in the chart. The gray circles indicate the centroid of each group. For comparison, the axis scale is identical to Figure 3. The Inset shows a zoom-in of the chart. It can be seen that there is no separation between the electrodes.

Figure 6

 

Data of an additional session from patient A. (a) Head-centered marked percept locations as recorded by the head-mounted camera. (b) Retinal angular position after mapping the recorded head-centered locations based on pupil location at the onset of the stimulation. The inset shows a zoom-in of the chart. (c) Layout of the array rotated to account for array placement on the retina with the group of electrodes that were stimulated in the different trials.

Figure 7

 

Data from a session recorded with patient B. (a) Head-centered marked percept locations as recorded by the head-mounted camera. (b) Retinal angular position after mapping the recorded head-centered locations based on pupil location at the onset of the stimulation. (c) Layout of the array rotated to account for array placement on the retina with the group of electrodes that were stimulated in the different trials.

Separation between clusters was quantified using the Davies Bouldin index¹⁷ that is based on the ratio of within-cluster and between-cluster distances. The Davies Bouldin index is defined by:  

 

where n is the number of clusters,   is the average distance between each point in cluster k and the centroid of cluster k, and d_k,q is the distance between the centroids of cluster k and cluster q. The values of the Davies Bouldin index are positive and a smaller value indicates that the separation between clusters is better.  

Results

Figure 2 shows the Argus II epiretinal implanted array consisting of 60 electrodes each, wired to a current driver that controls the electrical current waveform of each electrode. During daily use, stimulation waveforms in the electrodes are continuous and are set by the image produced by a head-mounted camera. Here, in each trial, we directly activated a group of electrodes for short time durations and the patients indicated the location of the percept using a handheld marker. To measure the mapping accuracy from the retinal stimulation to the location of the marker, we recorded the pupil location with a timestamp synchronized to the electrical stimulation. 

The location of the percept in a world-centered coordinate system is the sum of the angular positions of the stimulations versus the eye and the eye versus the head, assuming that the head and body are stationary. The spherical representation is:  

 

where:  

•  
  ϕ_eye(i), θ_eye(i) are the angular positions of the eye within the orbit (i.e., the direction of the pupillary axis with respect to the head) at trial i in the horizontal and vertical dimension, respectively.
•  
  ϕ_electrode(p), θ_electrode(p) are the angular positions of the stimuli due to electrical stimulation of the group of electrodes marked by p in the horizontal and vertical dimensions, respectively. These variables are in a retina-centered coordinate system and are relative to the pupillary axis.
•  
  ϕ_world(p,i), θ_world(p,i) are the angular positions of the percept due to electrical stimulation of the group of electrodes marked by p at trial i along the horizontal and vertical dimensions. These variables are at a world-centered coordinate system. In the current experiment, both the head and body were steady. Therefore, world-centered is identical to head-centered and body-centered coordinate systems.

The percept location measured in each trial at a world-centered coordinate system is given in Figure 3. The data in Figure 3 is a result of stimulating the groups of electrodes as specified in Figure 2. Because the patient “looked” (i.e., fixated the eyes) in a different direction at each trial, the location of the percepts are spread over the visual field. It can be seen that in all trials in which the patient looked rightward (marked with a right arrow symbol) the percepts were at the right of the visual field. Accordingly, in all trials that the patient looked leftward (marked with a left arrow symbol) the percepts were at the left of the visual field. This confirms previous results that eye position affects the percept's location in Argus II patients.⁵ Next, we wanted to remap the percept's locations to retinal locations that are independent of the eye position. 

To map the patient's response from world-centered to retina-centered coordinate systems using Equation 2, we needed to know the gaze orientation for each trial. A pupil tracker can give a measure of the gaze direction after a calibration process. With sighted subjects, the eye tracker is calibrated by asking the subject to look at a known target and then matching the pupil locations to the angular position for the calibration targets. With blind subjects, calibration of eye tracking is not a trivial matter. The orientation of the pupillary axis at a world-centered coordinate system as a function of the pupil's location at the coordinate of the eye camera is given by:  

 

where X_pupil(i) and Y_pupil(i) are the coordinates, in pixels, of the center of the pupil and a₀, a₁, a₂, b₀, b₁, b₂ are the free coefficients in the model. We used a first order polynomial model between pupil location and eyeball orientation.¹⁸ By substituting the angular gaze from Equation 3 for Equation 2 and solving for the angular retinal position of the electrode, we get the following:  

 

The six free variables can be found by minimizing the variance of ϕ_electrode(p,i) and θ_electrode(p,i) for each group of electrodes. Because each group of electrodes creates a spatially distinct precept, the angular position on the retina converges to the same value in all trials with a zero standard error. Values of the six variables represent the transformation between pupil position and gaze orientation. Minimizing the variance yields a set of values for the specific patient and the specific position of the pupil's camera for all the electrodes that were recorded in the run. A different position of the glasses with the pupil's camera will yield a different set of parameters to convert the pupil location to angular gaze.  

A map of retinal stimulation locations derived from the measured percept at world-centered coordinates using the pupil's location and Equation 4, is given in Figure 4. The pupils' location was sampled at the onset of the stimulation and the six free variables were found by minimizing the total spread in the retinal location of each group of electrodes. The correct mapping between retinal stimulation and spatial location was achievable when taking the pupil location at the onset of the stimulation. Figure 5 shows the results of an attempt to minimize the spread of each point relative to the centroid when using pupil location at the time of the pointing. Similar results were obtained by stimulating different electrodes (Figure 6) and from patient B (Figure 7). Comparison of the separation between the clusters of the retinal locations of the patterns for all cases is given in Table 1. We observed that the best separation was achieved when mapping is based on eye position at the beginning of the stimulation and not at the time of pointing, indicating that the brain registers and memorizes the spatial location at stimulation onset. 

Based on the measured retinal locations, we calculated the angular distances between the centroid of the tested groups. The distance between the centroids of the groups in Figure 5 compared with the angular distance based on the physical separation between the electrodes are given in Table 2. The calculated angular distance assumed a linear model in which every 0.293 mm of arc length on the retina equates to 1° in the field of view.¹⁹ It can be seen that the subjects amplified the separation between the stimulation locations. This observation is in agreement with open-loop pointing with sighted subjects that shows that peripheral retinal displacement is exaggerated.^20–22 Furthermore, we expect to see a deformation of the relative position and orientation between the electrodes because each electrode activates axon fiber tracts in addition to cell bodies. The activation of the fiber tracts might create an elongated percept along the axon.²³ Thus, the percept can be moved along the direction of the axons underneath the electrode causing relocation of the location of the percept. Herein, we asked the patients to indicate only the location of the percept and not the shape. From Table 2 we can see that the largest amplification of the separation occurred between groups A and C. The separation between group A and C is significantly more peripheral relative to the other separations, as can be seen from Figure 2. In addition to the activation of axonal fiber tracts, this can be also explained by the nonlinear relation of the projection on the retina with spatial perception. The retinal arc that corresponds to 1° of visual field decreases as a function of the peripheral angle from the optic axis.²⁴ 

Table 2

 

Measured Versus Calculated Angular Distance Between the Centroids of the Electrodes

 

The precision of the retinal locations (i.e., mean distance from the centroid of each pattern of a specific group of electrodes) for all sessions is given in Table 1. The measured imprecision in the retinal locations is a combination of an error in pointing in an open loop plus the error in the transformation between spatiotopic and retinotopic coordinates. In human subjects, we cannot directly measure the accuracy of the transformation between spatiotopic and retinotopic mapping. Here, we evaluated the accuracy using an open loop pointing task. In a similar task in which sighted subjects were required to point in an open loop setting without a visual feedback, the mean distance from the centroid was approximately 2° to 3°.²¹ The precision of open loop pointing in a response to a visual stimuli measured in artificial vision is comparable to findings in sighted individuals. 

Discussion

Our results show that blind individuals can map the percept of a retinal-centered electrical stimulation to the correct location in a head-centered coordinate system. This is done based on the instantaneous position of the eye within the orbit at the onset of the stimulation. This shows that the oculomotor system can steer the line of sight of a visual implant. Steering the line of sight is desirable for extending the field-of-view of the artificial visual prosthesis. Verifying that the brain can accurately shift the percept from electrical stimulation based on the position of the eye is of relevance for all types of retinal prostheses. Practically, eye movements can be used to shift the region of interest of an external sensor, like the Argus II, based on an eye tracker. Eye movements can directly steer the line of sight of retinal prostheses that use the optics of the eye such as the Alpha-IMS or the proposed PRIMA implant. 

The artificial visual information created by the electrical stimulation of the retina is mapped in the brain to build a correct representation of the world. Nonetheless, not all eye position signals that are available in natural vision are available in prosthetic vision. In patients with a visual prosthesis, some of the signals are inherently absent or affected by retinal and cortical reorganization following vision deprivation. These results are of interest with respect to the brain circuitries of visual mapping. In sighted individuals, establishing a correct and stable representation of the world is based on integrative information from inflow and outflow signals, which include eye position and continuous visual information from the retina. Experimentally, we need to deactivate one or more of the brain components to study the relative contribution of the other components that encode eye positions in the brain, surgically²⁵ or pharmacologically.²⁶ Alternatively, we can design a gaze contingent paradigm that will understate the visual field motion.²⁷ In artificial vision, some of the signals are inherently absent. Thus, it is worthwhile to compare the mapping circuitries in sighted individuals and prosthetic vision. 

For nonsaccadic eye movements there is evidence that visual stability is based on visual field motion. It was shown that during involuntary eye vibration, the brain can stabilize a stroboscopic illumination only when the entire retinal image is oscillating. However, the brain cannot stabilize an oscillatory stimulus in the presence of an additional continuously lit object in the visual field.²⁸ Eye position signals are identical regardless of the content of the retinal image. Hence, the content of the retinal-image is critical for visual stabilization. In prosthetic vision, it is unreasonable to think that the relative low resolution and fast fading perception of the currently available retinal prostheses can use visual field motion to produce eye position signals. Moreover, in this study we used a transient short stimulation and therefore, there was no visual field motion signal. 

The common model for saccadic eye movements is that visual stability is achieved using an internal signal within the brain, referred as a corollary discharge or an efferent copy of the motor command.²⁹ In this model, a copy of the signal that is sent to the motor area in the brain is sent to visual mapping areas of the brain. Corollary discharge signals are well documented in monkey brains,^26,30 which are the best available experimental model for the human visual system. The corollary discharge signals in monkeys are interpreted as a shift in the receptive field to the future foveal location prior to the onset of the saccade. The shift in the receptive field occurs at the planning stage of the saccade and is not based on proprioception signals from the ocular muscles. Nonetheless, it was argued that the brain uses the proprioception signal to improve the precision after a sequence of many saccades.³¹ In artificial vision there is no foveal vision and it is not clear if the brain uses the efferent copy of the motor command as in sighted subjects. In our study, the subjects performed a nonfoveal driven eye movement prior to the saccade that included a substantial drift. In this case, the blind subject performed a voluntary eye movement to a general direction without any visual feedback from the target's location. It is not very probable that the brain integrates the eye position from the nonvisual eye movements prior to the onset of the stimulation. 

Proprioception, the inflow signal to the brain of the eye position in the orbit, is probably the key signal that was available for mapping in our experimental setup. The physiology and anatomy of the eye muscle proprioception is well documented in humans.³² In addition, it was shown that the proprioception signal can be swayed by mechanically moving one eyeball while the target is observed by the other eye. Passive mechanical movements of the eyeball can be achieved by either a controlled force introduced by suction³³ or by simply pressing the eyeball with a finger.³⁴ Disruption of the proprioceptive representation in the somatosensory cortex can also be carried out by using repetitive transcranial magnetic stimulation that shifts the perceived location of the target.³⁵ Thus, in sighted individuals, manipulation of the proprioception signal affects the spatially perceived location of the targets. In monkeys, the deafferentation of the extraocular muscles does not affect ocular motor control.²⁵ This hints that the efferent copy is sufficient for eye movement control without the need of proprioception outflow signals. Proprioception affects the perceived spatial location but is not necessary for eye movement control. The experiment presented here demonstrates that proprioception signals of the eye position with artificial sight can yield an open-loop hand pointing accuracy of 2° to 3°. 

The measured error is a combination of the error in mapping the visual input and the error of placing the marker in an open loop task. The latter is probably associated with the proprioception signals from the arm's muscle and not from the visual information. We can therefore assume that the errors are similar for native and artificial sight. Our observation that the open loop pointing error in artificial sight is similar to natural sight, suggests that the mapping accuracies of both natural and artificial sight are the same. The accuracy of the hand pointing might not be critical because the hand-camera coordination varies over time in patients implanted with a retinal prosthesis.³⁶ However, here we only used the hand point as a probe to indicate the location of the percept. An accurate mapping between retina-centered stimulation and world-centered visual information will improve the visual stability of the artificial sight and will enable better integration of visual information. 

Calibration of an eye tracker for a blind patient implanted with a retinal prosthesis can be done using the procedure described herein. In practice, the patient will indicate the location of percepts using a handheld marker. The location of the marker can be measured using the integral scene camera of the prosthesis. By minimizing the spread, we can find the calibration parameters (i.e., the value of the coefficients in Equation 3). 

A calibrated eye tracker integrated with a visual prosthesis will enable further research of the oculomotor functionalities of blind users with a retinal prosthesis. This will include the temporal properties of eye movements in blind people (i.e., the time window around the saccade that the brain acquires information). Because one cannot initiate smooth pursuit eye movements without moving visual stimuli, it would be interesting to ascertain whether a blind patient with a retinal prosthesis can perform smooth pursuit eye movements using a retinal prosthesis with an integrated eye tracker. Furthermore, blind patients that will use a visual prosthesis with a closed-loop oculomotor steering might adapt a preferred retinal location (PRL) as seen in patients with central scotoma³⁷ and suggested for blind patients with retinal prostheses.³⁸ 

By quantitating the patient's eye position, measured using an eye tracker, we were able to get a spatial map of the percept of electrodes in retina-centered coordinates. Previous work has shown the qualitative relationship of the location perceived by the patient and the direction of the eyes.⁵ The general dependency of the perceived location on eye movements was used to explain discrepancies in the perceived location in an experiment used to describe the spatial map of electrodes in the retinal prosthesis.³⁹ The methodology presented here can be used to assess the spatial map of the electrodes in retina-centered coordinates that is not affected by eye position. This can be used to isolated factors that affect the integrity of the spatial map of electrodes. It was observed that there is intrapatient variation in the sensitivity among the electrodes in the implanted array. This is apparently due to the distance of the electrode from the retina and the retinal health.⁴⁰ Future multicentered research can use this method to assess the spatial map in retinal coordinates that is not affected by eye position to find the factors that influence the spatial map of the electrodes. 

In conclusion, we have shown that the brain of a blind individual has the necessary signals to accurately map artificial sight generated by electrical stimulation at retina-based coordinates and process that signal to world-based coordinates. Because mapping might be based only on the outflow proprioception signal from the eye muscle, the relatively long latency of the proprioception signal^41,42 must be taken into consideration. Further work is needed to study if the brain selectively masks the percept from electrical stimulation during saccadic eye movements to prevent blurring.⁴³ Better understanding of how the brain modulates the artificial sight based on eye position will provide an efficient use of eye movements in retinal and cortical visual implants. 

Acknowledgments

A. Caspi thanks Eli Peli, OD, from the Schepens Eye Research Institute for his keen insights regarding the results. 

Supported by grants from the Alfred Mann Foundation (Valencia, CA, USA). 

Disclosure: A. Caspi, Second Sight Medical Products, Inc. (C), P; A. Roy, Second Sight Medical Products, Inc. (E, I), P; J.D. Dorn, Second Sight Medical Products, Inc. (E, I), P; R.J. Greenberg, Second Sight Medical Products, Inc. (E, I), P