Reading Pixelized Paragraphs of Chinese Characters Using Simulated Prosthetic Vision

Abstract

Purpose.: Visual prostheses offer a possibility of restoring useful reading ability to the blind. The psychophysics of simulating reading with a prosthesis using pixelized text has attracted attention recently. This study was an examination of the reading accuracy and efficiency of pixelized Chinese paragraphs after different parameters were altered.

Methods.: Forty native Chinese speakers with normal or corrected visual acuity (20/20) participated in four experiments. Reading accuracy and efficiency were measured after changing the character resolution, character size, pixel dropout percentage, number of gray levels, and luminance.

Results.: A 5° × 5° character appeared to be the optimal size necessary for accurate pixelized reading. Reading accuracy close to 100% could be achieved with 10 × 10 pixels/character and ∼60% with a 6 × 6 pixel resolution. Pixel dropout adversely affected accuracy, and paragraphs with a 50% dropout were unreadable. Luminance had little effect; however, the number of gray levels significantly affected reading performance. Paragraph reading was at least 5% more accurate at each resolution than was the accuracy of Chinese character recognition.

Conclusions.: Character size and resolution, pixel dropout, and the number of gray levels clearly affected the reading performance of pixelized Chinese paragraphs. Compared with pixelized character recognition, pixelized Chinese paragraph reading achieved higher accuracy; thus, optimal Chinese reading performance may require prostheses with more electrodes (1000) than are required to read paragraphs in the Latin alphabet (500).

In recent decades, the restoration of vision using a visual prosthesis has attracted the attention of many researchers around the world. Several research groups have successfully implanted different kinds of visual prosthetic devices (in the visual cortex, retina or optic nerve) in blind patients, to elicit phosphenes and to evaluate prosthetic vision. ^{1 –15} Dobelle and Mladejovsky ¹ and Brindley and Lewin ² explored the feasibility of a visual prosthesis by implanting a prototype prosthesis in the patient's visual cortex. Veraart et al., ³ on the other hand, used an optic nerve prosthetic approach in a blind patient and reported that the patient was able to interact with the environment and that pattern recognition and orientation discrimination progressively increased with more training. Yanai et al. ⁴ implanted a 16-electrode epiretinal prosthesis in a completely blind patient who, as a result of the epiretinal stimulation, could detect motion and locate objects. Based on these promising results, devices with 60 to 1500 electrodes are either currently in clinical trials or are being developed. ⁵  

Prosthetic vision is elicited through arrays of discretely spaced electrodes that, in an ideal situation, should produce a phosphene map similar to a pixelized image. ⁶ Consultations with patients who may benefit from a visual prostheses have indicated that the most important visual functions include mobility without a cane, face recognition, and reading. ⁶ The possibility that large electrode arrays may be used to restore a more complex visual percept, such as edge and movement detection and text reading, has encouraged some investigators to explore the limitations of phosphene-generated percepts through the use of simulated phosphenes. ^{7 –9}  

Pixelized reading has been systematically studied by Legge et al. ^{10 –15} They reported that maximum reading rates for normal subjects were achieved with characters subtending 0.3° to 2° of visual angle and that reading rate increased with field size. Reading ability was also very tolerant to luminance or color contrast reductions. Other research groups have made great progress in isolated word recognition (Latin alphabet) as reported in psychophysical studies. ^{16 –22} Cha et al. ¹⁶ hypothesized that a 25 × 25-pixel array representing four letters of text, when projected onto a 1.7° foveal visual field, would be sufficient to allow 100 to 170 words/min to be read from fixed or scrolled text, respectively, Sommerhalder et al. ^17,18 and Fornos et al. ¹⁹ assessed the minimum requirements for useful artificial vision and concluded that real-time stimulus pixelization favors reading performance. Under these conditions, 400 to 500 contacts covering a retinal area of 2 × 3 mm² would be necessary to transmit sufficient visual information for full-page text reading. Dagnelie et al. ²⁰ reported on changes in reading ability using a pixelized grid defined by five parameters (dot size, grid size, dot spacing, random dropout percentage, and gray-scale resolution) and concluded that a 3 × 3-mm² prosthesis with 16 × 16 electrodes would allow paragraph reading. Fu et al. ²¹ developed a closed-circuit television reading platform with digital image-processing capacities and suggested that reading at 15 words/min is possible with as few as 6 × 6 binary pixels. 

Although there are several simulation studies using the Latin alphabet, little attention has been paid to Chinese character (CC) reading. Chinese is an ideographic system based on hieroglyphs and pictophonetic characters that are substantially different from words based on the Latin alphabet and, in most cases, are visually much more complex. It is assumed that the parameters that influence the ability to read the Latin alphabet (such as pixel size, pixel spacing, grid size, and gray-scale levels) will also have an impact on the ability to read CCs. Given the potentially large number of Chinese patients who may benefit from a prosthetic device, it is of importance to compare the effect of these parameters on Chinese reading ability with those noted in previous studies. 

Chai et al. ²² studied the effect of character resolution, stroke number, and character typeface on isolated CC recognition using simulated prosthetic vision. The results indicated that perfect performance could be achieved with 12 × 12 pixels (equivalent to 144 electrodes), as opposed to nearly no recognition with 6 × 6 pixels. In addition, Song and Hei fonts, which occupy much more space than other fonts, achieved higher recognition accuracy. 

We decided to test the ability of subjects to read pixelized Chinese paragraphs (CPs) as a way of simulating paragraph reading using prosthetic vision. We investigated the following: (1) the influence of CC size on the ability to read pixelized CPs, to determine which CC size would optimize reading performance; (2) reading performance using different pixelization resolutions; (3) the influence of dropout percentage, gray scale, and luminance on pixelized CP reading ability, to mimic similar scenarios in patients with an implanted prosthetic device; and (4) the recognition accuracy of single pixelized CCs compared with pixelized CP reading accuracy (RA; i.e., the effect of contextual information). 

Methods

Subjects

Forty native Chinese speakers aged between 20 and 30 years were chosen from Shanghai Jiao Tong University and participated in four experiments (n = 10/experiment). They all had normal or corrected visual acuity (20/20) and passed a Standard Chinese Proficiency Test (PSC Test). Before the formal experiments, the purpose and procedures of the experiment were explained, and written consent was obtained from all subjects. The research adhered to the tenets of the Declaration of Helsinki. 

Experimental Setup

The experiment was completed in a quiet, dark testing room. The setup included a head-mounted display (HMD; 800-26, 2 LCOS displays, 800 × 600 resolution per display, 26° diagonal viewing field; 5DT, Inc., Irvine, CA), a CCD microcamera with a ¼-in. CCD (Sharp; PAL 752 × 582, XingZhiLin Electronic Company, Ltd., ShenZhen, China), a personal computer (2.4 GHz CPU; Dell. Inc., Round Rock, TX), and a separate 19-in. LCD monitor (1280 × 1024 at 60 Hz, 300 cd/m², 5 ms; Dell. Inc.) connected to the computer by a VGA distributor. Experimental software, developed in-house with Visual C++ or Delphi language, was run in Windows XP OS (Microsoft, Redmond, WA) and displayed the CPs on the LCD monitor and recorded the time between two mouse clicks as part of the subject's response. 

The camera was mounted onto the HMD and was used to capture the images of the pixelized CPs presented on the 19-in. LCD monitor and transfer them to the HMD. A head-rest was used to stabilize the distance between the subject's head and the LCD monitor. A microphone was used to record the spoken responses of each subject throughout the experiments. 

Reading Materials

To ensure that the paragraphs used in the experiments had a similar degree of difficulty for all subjects, we chose 200 paragraphs from primary school Chinese language textbooks (grades 4–6; People's Education Press 5th edition, 2008, PR China). The number of CCs in each paragraph were counted and >95% of the CCs were among the top 1000 most commonly used characters according to government statistics. ²³ This method ensured that unfamiliar or infrequently used CCs were avoided, and the CCs could provide nearly 92% of daily reading and written information. All CPs had a similar CC frequency distribution, and the CCs had a similar number of strokes, such that, for each CP, (1) the number of CCs was between 40 and 44, (2) the average number of strokes per CC in each CP was between 6 and 8, and (3) the average CC frequency in the CPs was between 0.24 and 0.36. 

Experimental Library

Each of the original CCs in a paragraph (Song typeface) was pixelized into 6 × 6-, 8 × 8-, or 10 × 10-resolution Gaussian phosphene maps, according to the GB 2312-80 Code (Guo Biao/Chinese National Standard). Adopting the method proposed by Sommerhalder et al.,¹⁹ a two-dimensional circularly symmetric Gaussian distribution was applied to each pixel of each CC. The function is as follows²⁴:  

 

 

 

where I(x, y) denotes the gray-level intensity at position (x, y) of the simulated phosphene map; A(μ_x, μ_y) represents the mean gray level of the original n × n pixel matrix with center coordinates (μ_x, μ_y); G(x, y) denotes the two-dimensional Gaussian function viewing distance defined in equation 2; and σ represents the SD of the particular Gaussian function around its horizontal and vertical center coordinates μ_x and μ_y, is equal to the Gaussian width, and thus determined the amount of overlap between pixels in our experiments. 

Reading material for each experiment was randomly chosen from the 200 CPs. Each data set contained each CP at each resolution (i.e., 6 × 6, 8 × 8, and 10 × 10), each CC was pixelized by matching the CC GB code with their corresponding grapheme, and then each character in the paragraph was adjusted to form a new test paragraph that varied according to character size (S-CP), dropout percentage of pixels (D-CP), luminance (L-CP), and number of gray levels (G-CP). Note that although the same paragraph was used for each of the three resolutions, the locations of the altered pixels differed between resolution libraries and between the different levels (Fig. 1). 

Figure 1.

 

The structure of experimental libraries. Chinese paragraphs (CPs; n = 200) were chosen, and each of the original Chinese characters in each paragraph was pixelized (Gaussian distribution) to different resolutions (6 × 6, 8 × 8, and 10 × 10). The new character set was used to reform the 200 CPs. The characters in randomly selected pixelized CPs were then adjusted to form parameter test libraries according to character size (S-CP), pixel dropout percentage (D-CP), luminance (L-CP), and number of gray levels (G-CP).

During text reading, the asymmetric perceptual span extended one character to the left of the fixated character and three characters to its right. ²⁵ The size of right-directed saccades extended across 2 to 2½ character spaces, indicating that the perceptual spans of successive fixations overlapped slightly and that some linguistic information was integrated across fixations. Therefore, one CP contained two pages with 4 × 6 CCs per page (using a character space to represent punctuation). Subjects in our experiments may have relied on a small amount of scanning when viewing the pixelized CCs and CPs, even though they were asked to fixate throughout the experiment. 

Experiment 1: Effect of Character Size

Chai et al. ²² studying CC reading showed that 100% accuracy could be achieved with 12 × 12 pixels, but this decreased to almost 0% when using 6 × 6 pixels. Therefore, we used three resolutions (6 × 6, 8 × 8, and 10 × 10) and five character sizes (1° × 1°, 3° × 3°, 5° × 5°, 7° × 7°, and 9° × 9°). The size was changed by adjusting the viewing distance (from the camera on HMD to the external monitor screen). Seventy-five S-CPs were randomly chosen and presented in three blocks corresponding to the five sizes (five CPs for each of the sizes [n = 25] × three resolutions = 75 CPs); each trial consisted of five CPs (for a sample S-CP; Fig. 2). The presented block sequence was 6 × 6, 8 × 8, and 10 × 10 resolution, and the trial sequence was 1° × 1°, 3° × 3°, 5° × 5°, 7° × 7°, and 9° × 9° within each block. Note that the order of block presentation was kept the same for all experiments. 

Figure 2.

 

Sample images for the size test in experiment 1. Each column corresponds to the number of pixels in the array that form the individual characters, and each row corresponds to a particular character size. The viewing area spanned a 20° × 15° region of the subject's visual field and may include more than one CC. Note that the best viewing angle of one CC for the subject is ∼10° to 11°.

Experiment 2: Effect of Dropout Percentage

Sixty CPs were randomly chosen, and phosphenes across the whole visual field were randomly dropped out to create four dropout levels (0%, 10%, 30%, or 50%) within the library (D-CP). CC size was set to 5° × 5° based on the results of experiment 1. D-CPs were presented in three blocks containing four trials, and each trial was made up of five CPs (Fig. 3). 

Figure 3.

 

Sample images used for testing the effect of pixel dropout. The same CP is shown at four dropout levels (rows) and for each of the arrays used to form the individual 5° × 5° characters (columns).

Experiment 3: Effects Caused by the Number of Gray Values and Luminance

Four luminance values (100%, 50%, 25%, and 12.5%) and three gray levels (level 1 = 255; level 4 = 63, 127, 191, and 255; or level 8 = 31, 63, 95, 127, 159, 191, 223 and 255; for example, using option 4, meant that the gray-scale value of each simulated phosphene in the CP would be randomly assigned one of four values: 63, 127, 191, or 255) were applied to 90 randomly chosen CPs. The CC size was set to 5° × 5° and the CPs presented in three blocks containing six trials, with each trial made up of five CPs (5 CPs/trial × 6 trials/block × 3 blocks = 90 CPs). Three of the six trials in each block altered luminance while maintaining a gray level 1, and three trials altered the gray levels while maintaining 100% luminance. Within each block, subjects read each trial CP in the following sequence: 100% luminance with an increasing number of gray levels followed by gray level 1 with decreasing luminance (Fig. 4). 

Figure 4.

 

Sample images used for testing the effect of altering the number of gray values within the text and changes to the overall luminance (5° × 5°/character). Each column corresponds to the pixel resolution and each row corresponds to a given gray and luminance level. The column–row order follows the order of presentation used in the tests.

Experiment 4: Single Pixelized CC Recognition versus Pixelized CP Reading

Two hundred pixelized CCs were randomly chosen from the CPs at each resolution (6 × 6, 8 × 8, and 10 × 10 data set). CC size was set to 5° × 5°, and the CCs were presented in three blocks, each containing 200 characters. 

Procedure

Subjects were given a standard training program before each formal experiment to familiarize them with the experimental setup and what was expected during CP reading and CC recognition. Reading materials were chosen randomly and differed from those used in the formal tests. 

The subjects viewed the screen using the HMD and camera and read aloud each page of the CP; the subject could move his or her head to view different portions of the screen and, after finishing a page, used a mouse click to bring up the next page or next trial. The viewing window of the HMD was fixed to 20° × 15°. The experimenter was able to simultaneously monitor the CPs, listen to the subject, and record the whole reading process and the time between mouse clicks. The reading time for each subject was recorded by a special time-count program. Subjects were allowed a 5-minute rest between blocks. 

The RA of each subject was analyzed according to the audio recording after each experiment: RA = CC_correct/CC_total, for each CP. Reading efficiency (RE) = CC_correct/T_total where T_total denotes the total time used to read one paragraph. The mean for the five trials for each subject was calculated, and the mean for all subjects was calculated from the result. Data were analyzed with ANOVA and t-tests (two-tailed; a Bonferroni correction was applied to multiple comparisons; SPSS 16.0 for Windows; SPSS Inc., Chicago, IL). Comparisons with P ≤ 0.05 were considered statistically significant. 

Results

Experiment 1: Effect of Character Size

The 10 subjects were able to achieve RA scores >40%, 60%, and 90% respectively, with the 6 × 6, 8 × 8, and 10 × 10 pixels per character resolutions (see Fig. 5A). One-way ANOVA indicated that, for a 6 × 6 or 8 × 8 resolution, character size significantly affected CP reading performance (P < 0.001). RA was low for small character sizes but increased as character size increased, to reach a peak with 5° × 5° characters. Individual comparisons for 6 × 6 and 8 × 8 resolutions showed no significant differences between character sizes 5° × 5° versus 3° × 3° or 7° × 7°. Two-way ANOVA showed no significant interaction between 6 × 6 and 8 × 8 resolutions and 3° × 3°, 5° × 5°, and 7° × 7° character sizes. No significant effect of character size was seen when using the 10 × 10 resolution and RAs were almost 100%. Because the 5° × 5° characters provided the highest RAs, this size was used for all subsequent experiments. 

Figure 5.

 

Mean RA (±SD) for a range of pixel resolutions is shown as a function of (A) character size, (B) dropout percentage, (D) the number of gray levels, and (F) luminance levels. Mean RE (±SD) for the same parameters are shown in (C), (E), and (G). In (A) statistical comparisons were made with respect to the values obtained for 5° × 5° characters. In (B–G) comparisons made within a resolution level refer to the data point to the right; comparisons between resolution levels are shown in brackets and refer to the lowest resolution level. *P < 0.05, **P < 0.01, ***P < 0.001.

Experiment 2: Effect of Dropout Percentage

Figures 5B and 5C show the mean (±SD) RA and RE values as a function of dropout percentage for the three different resolutions. Increasing the dropout percentage significantly (P < 0.001; n = 10) reduced both RA and RE. RA decreased to <20% and RE to <10 characters/min when using a 50% pixel dropout, effectively making all the CPs unreadable. 

When the dropout percentage increased from 0% to 10%, RA decreased by ∼30% for the 6 × 6 and 8 × 8 resolution, but only ∼5% for the 10 × 10 arrays (57.5% ± 5.9% vs. 39.8% ± 7.9%, 90.5% ± 7.1% vs. 64.9% ± 10.6%, and 99.5% ± 0.8% vs. 94.9% ± 3.7% respectively; P < 0.001). A 30% dropout caused a decrease in RA of ∼75% to 80% for the 6 × 6 and 8 × 8 resolutions and ∼27% for 10 × 10 arrays (6 × 6, 13.8% ± 1.7%; 8 × 8, 28.6% ± 7.8%; and 10 × 10, 72.4% ± 9.6%; P < 0.001). A 50% dropout caused RA to decrease by 95% to 80% for the three resolutions (4.8% ± 2.2%, 11.2% ± 3.2%, 18.7% ± 4.4%, respectively; P < 0.001). RA exceeded 90% only when a high resolution or a low dropout level was used (10% dropout for 10 × 10 resolution, 0% for 8 × 8 resolution). 

RE decreased with an increase in pixel dropout, especially for the 10 × 10 resolution (152.7 ± 29.8 characters/min to 8.5 ± 2.4 characters/min for 0% vs. 50% dropout; P < 0.001). Multiple comparisons showed a significant difference between RE values at two dropout percentages (0% vs. 10%, 30% or 50%; 10% vs. 30% or 50%) for the 10 × 10 resolution (P < 0.001), whereas there was no significant effect between 30% vs. 50% dropout for the 6 × 6, 8 × 8, and 10 × 10 resolution. 

Experiment 3: Effects Caused by Altering the Number of Gray Values and Luminance

Gray Levels.

RA decreased with an increase in the number of gray values in the characters (Fig. 5D). This effect was more pronounced for the 6 × 6 and 8 × 8 resolution CCs (59.1% ± 4.4% to 25.9% ± 6.9% and 92.5% ± 3.4% to 59.1% ± 13.1%, respectively; P < 0.05, n = 10). In contrast, RA was not dramatically affected by gray levels when the resolution was 10 × 10 (all, >90%; NS); only gray level 1 and an 8 × 8 resolution allowed a RA > 90%. 

The RE significantly decreased with an increase in the number of gray values (Fig. 5E) at all three resolutions (6 × 6, P < 0.01; 8 × 8, P < 0.01; and 10 × 10, P < 0.05; n = 10). Although the RE was significantly lower for the 10 × 10 resolution with eight gray values, RE still exceeded 70 characters/min, a value that was only matched using an 8 × 8 resolution with one gray level. 

Luminance.

The range of luminance changes used here had very little effect on RA within each of the character array sizes (Figs. 5F, 5G). When using a 10 × 10 resolution, RA remained at ∼100% for all luminance levels, whereas the RA when using the 8 × 8 or 6 × 6 resolution, remained at ∼92% and ∼58%, respectively. In contrast to RA, the RE was significantly affected by an increase in luminance (6 × 6, P < 0.01; 8 × 8, P < 0.01; 10 × 10, P < 0.001), although multiple comparisons, showed that this difference was mainly seen between lowest and highest luminances (12.5% vs. 50%, P < 0.01; 12.5% vs. 100%; P < 0.001). 

Experiment 4: Single Pixelized CC Recognition versus Pixelized CP Reading

When the RA of single CCs was compared with that of CPs, there was a clear and significant increase in the RA for CP reading (6 × 6, 15.2%, t = 7.457, P < 0.001; 8 × 8, 19%, t = 7.941, P < 0.001; 10 × 10, 4.6%, t = 4.713, P < 0.01; n = 10). RAs for single CCs shows the same trend as CP reading, with respect to character resolution, although there appeared to be a somewhat larger difference between CC recognition with 8 × 8 vs. 10 × 10 pixels compared with that seen with CPs (Fig. 6). 

Figure 6.

 

RA of isolated single Chinese characters versus RA of Chinese paragraphs (**P < 0.01; ***P < 0.001) as a function of character resolution (n = 10).

Discussion

Optimum Character Size and Resolution

Our results show that CC size is an important parameter to consider if a visual prosthesis is to be used for reading. The results of Feng et al. ²⁶ and governmental reports ²⁷ have shown that a 5° visual angle is the most suitable visual dimension. This finding may be related to the fact that normally sighted subjects focus the image on the macula/fovea when reading and watching television or movies and in any activity in which visual detail is of primary importance. Experiment 1 indicates that a size between 3° × 3° and 5° × 5° is optimal for pixelized CP reading and that CCs smaller or larger than this will impair reading performance. Character size has also been shown to have a noticeable effect on reading pixelized English text. ¹⁶  

Chai et al. ²² studied the effect of pixel resolution on CC recognition and reported that RAs varied between ∼0% and 10% when using a 6 × 6 to 8 × 8 resolution and was considerably lower than the accuracy of CC recognition in our study (45.5%–72.6%). The reason for these differences may be due to the larger number of CCs used by Chai et al., some of which could not be recognized, even under normal conditions, compared with our study (3755 vs. 600 CCs from 1000 commonly used CCs). A more recent study by the same group ²⁸ examining the effect of complexity and minimum resolution needed for recognition of pixelized CCs used a database similar that used here and obtained results similar to those in the present study. 

Effect of Pixel Dropout

A RA higher than 90% was observed only if the dropout level was 10% or lower for a 10 × 10 resolution, and any increase in the dropout percentage caused a dramatic drop in RA to levels that would be unacceptable for reading. Dagnelie et al. ²⁰ have shown that English RA with an optimal character size, defined as the character size with the highest RE, fell from ∼80% to below 60% when the dropout rate went from 30% to 70%. Although both our study and the previous study used a paragraph-reading task and therefore provided word context, it appears that recognition accuracy of CCs is more sensitive to pixel dropout than English RA. These differences may be partly attributable to a lack of context, because CC recognition is dependent on clearly establishing context to determine character meaning. Nevertheless, this difference is more likely to be due to differences between the complexity and variety of CCs compared with English text. In the Latin alphabet, characters contain roughly one to three strokes, whereas CCs contain from 1 to 36 strokes, ²² forming a complex system of pictograms, ideograms, phonograms, and phono-ideograms. ²⁹ Small changes to the apparent strokes can have a more dramatic effect on interpretation compared with small changes to letters. A more complete explanation and examples of why small differences to stroke shape affect RA is given in our previous report. ³⁰ Initial estimates suggest that a visual prosthesis with 400 to 500 electrodes would be required to read text using the Latin alphabet. ¹⁹ CP reading could require at least 1000 electrodes if RA is to exceed 90% and RE to approach 70 characters/min (i.e., a 10 × 10 resolution per CC and 10 to 12 CCs per view window). 

Effect of Gray Level and Luminance

Dagnelie et al. ³¹ have shown previously that adequate English RA could be accomplished with two gray levels. In contrast to English paragraph reading, ²⁰ increasing the number gray values in the CCs significantly reduced reading performance and increased reading time. 

Luminance changes had little effect on RA, but did affect RE, albeit mostly when luminance fell below 25%. A similar conclusion was reached when comparing the ability to read English text under different contrast conditions, ranging from ∼1.0 to 0.1. ¹³ Legge et.al. ¹² demonstrated that for observers with normal vision the luminance (or the text luminance when the background was dark) determined reading rate regardless of the color of the test. It suggests that luminance is not a primary factor for visual tasks using prosthetic vision. It is assumed that luminance and contrast may not be a significant problem for prosthesis users, given that the cameras used to initially capture the image can automatically alter the overall luminance and contrast of the visual scene so that they remain within defined limits. However, variations in the gray levels or brightness of individual phosphenes ³² that are unrelated to the captured image may be more problematic. Previous research indicates that phosphene brightness can be altered in response to changes in stimulation parameters. ³³ Therefore, in the future it should be possible to adjust stimulus parameters so that the resulting phosphene brightness is more closely related to differences within the captured camera image or is set to a level that does not affect reading ability. 

Single Character Recognition versus Paragraph RA

Experiment 4 demonstrated that RA was significantly higher when reading a CP compared with single CC recognition and showed that linguistic context is often necessary for understanding text. These results have important implications for prosthetic vision, given that the number of electrodes will ultimately determine the amount of text available to the user. Prosthetic devices with more electrodes, in theory, should increase the number of available phosphenes and readable characters. This increase in phosphenes will allow the user to interpret words or characters in a contextual setting and thus will provide a more effective and efficient reading capacity. While larger arrays could make available more characters simultaneously, this does not negate the usefulness of smaller arrays. In this situation the ability of subjects to scan a paragraph of characters, through eye or head movements, could provide all the necessary contextual information to provide a sufficient level of comprehension. Nevertheless, character-by-character, or word-by-word recognition for paragraph reading will significantly lower the RA and/or RE, as shown in the present study. ^17,18 In addition, patients may find that more head movement is required for scanning, thus adding to training difficulties. Partly due to these factors and the desire to provide the largest amount of visual information possible, the trend in visual prosthesis development will be toward more and more electrodes, assuming this indeed results in higher effective resolution that can provide a richer visual percept. 

Limitations of the Study

Our experiments present simulation data based on a highly idealized map in which all the points in the array have a possibility of representing a phosphene resulting from the stimulus probe. We choose an ideal round spot with a Gaussian distribution to mimic the phosphene. ¹⁹ The appearance of the phosphenes elicited by electrical stimulation in human trials are most commonly described as small round spots of light and or as having a cloudy appearance, ^2,32 although other forms have also been reported. ⁷ Despite this diversity, a circular shape is the most common choice in simulation studies, based on assumptions that refinements to future prosthetic devices will provide such a phosphene map. ³⁴  

Previous work has shown that prosthetic devices are unlikely to produce such well-organized and complete phosphene maps. According to human trials ^32,35,36 the phosphene position, relative to the retinotopic position of the stimulating electrodes may be distorted and incomplete maps can result from electrode failure or less-than-perfect placement of the prosthesis. However, the phosphene maps and phosphene characteristics created by array stimulators, while apparently stable over time, can be altered by changing the stimulus parameters in conjunction with verbal responses by subject. ³³ This ability of the patient to interact or interface with the prosthesis may make it possible to modify and, we hope, to improve the ability of the user to get the maximum benefit from the prosthesis. Modifications to stimulus parameters must be monitored closely for adverse effects on phosphene appearance. For example, although the initial phosphene dropout may remain stable, lowering stimulus intensity at a particular electrode to alter the phosphene percept (e.g., brightness) could lead to additional phosphene dropout in the same area and thus lead to reduced CC recognition. 

Additional limitation is that subjects in our experiments may have relied on a small amount of scanning when viewing the pixelized CCs and CPs, even though they were asked to fixate throughout the experiment. Although patients using prosthetic stimulation also use a scanning strategy to determine basic shapes, ³³ the ability to scan the CP by our subjects differs in principle from prosthetic stimulation in which the pixelized image is stabilized on the retina after capture from an external source such as a head-mounted camera. A more natural scanning method for a prosthesis with an external camera can be accomplished through image translation based on eye tracking. However, future prosthetic devices with larger electrode arrays and appropriately adjusted stimulation parameters and refresh rates may provide a visual percept that more closely matches simulation studies than can be achieved with current prosthetic devices. 

Conclusion

Character size and resolution affect the ability to read pixelized CPs. A 5° × 5° character size would be appropriate for pixelized CP reading. Increasing the resolution to 10 × 10 pixels per character allows an RA of nearly 100%, although subjects using a 6 × 6-pixel resolution can still achieve nearly 60% accuracy. However, these results are under ideal conditions, and RA and speed decrease as the percentage of pixel dropout, or the number of gray levels, increases. Luminance apparently has little effect on CP RA, but could influence RE. Character recognition was more accurate when CCs were presented in the context of a CP compared with single CC presentation. These simulation studies, although idealized, give an indication of what may be possible for a prosthetic user with an optimal or suboptimal phosphene map. Researchers making future refinements to prosthetic devices to achieve larger phosphene maps can use the current data to determine realistic outcomes with respect to text reading ability. 

Footnotes

 Supported by the National High Technology Research and Development Program of China (863 Program, 2009AA04Z326); The National Basic Research Program of China 973 Program, 2011CB707502, 2011CB707503; The National Natural Science Foundation of China Grants 60871091, 31070895; The National Key Technology R&D Program 2007BAK27B04 and 2008BAI65B03; and the 111 Project from the Ministry of Education of China Grant B08020.

Footnotes

 Disclosure: Y. Zhao, None; Y. Lu, None; J. Zhao, None; K. Wang, None; Q. Ren, None; K. Wu, None; X. Chai, None

The authors thank the volunteers for their participation and Thomas FitzGibbon for comments and suggestions on earlier drafts of the manuscript.