Visual Search with Image Modification in Age-Related Macular Degeneration

Abstract

Purpose.: AMD results in loss of central vision and a dependence on low-resolution peripheral vision. While many image enhancement techniques have been proposed, there is a lack of quantitative comparison of the effectiveness of enhancement. We developed a natural visual search task that uses patients' eye movements as a quantitative and functional measure of the efficacy of image modification.

Methods.: Eye movements of 17 patients (mean age = 77 years) with AMD were recorded while they searched for target objects in natural images. Eight different image modification methods were implemented and included manipulations of local image or edge contrast, color, and crowding. In a subsequent task, patients ranked their preference of the image modifications.

Results.: Within individual participants, there was no significant difference in search duration or accuracy across eight different image manipulations. When data were collapsed across all image modifications, a multivariate model identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interactions among scotoma size, age, acuity, and contrast (P < 0.05). Additionally, an analysis of image statistics showed no correlation with search performance across all image modifications. Rank ordering of enhancement methods based on participants' preference revealed a trend that participants preferred the least modified images (P < 0.05).

Conclusions.: There was no quantitative effect of image modification on search performance. A better understanding of low- and high-level components of visual search in natural scenes is necessary to improve future attempts at image enhancement for low vision patients. Different search tasks may require alternative image modifications to improve patient functioning and performance.

Introduction

Foveal and peripheral vision serve complementary roles in visual search. To locate a particular target within a natural scene, the visual system is required to identify candidate targets with low-resolution peripheral vision and then move the fovea to the optimal target and scrutinize it with high acuity. ¹ This system can be compromised if information is lost following pathological insult in the either in the periphery ² (Wiecek E, et al. IOVS 2011;52:ARVO E-Abstract 5731) or in the fovea. ³  

AMD is the leading cause of blindness for adults over the ages of 55 years in the western world with an estimated prevalence in the United States population of 6.5% and 30% of the population over 74 years old. ^2,4 In addition to loss of central vision, quality of life is significantly reduced by AMD. ^5–7 People with AMD have difficulty with tasks that depend on high-resolution central vision, such as reading, driving, and face recognition. ^8–11 However, many of these deficits are also observed in older adults without measurable vision impairments, ^8,12–15 so that the extent to which observed functional deficits may be attributed to aging, rather that the visual field loss is currently unknown. ¹⁶  

Several researchers have closely examined how visual impairment affects vision related tasks. Jacko et al. ¹⁶ and Kuyk et al. ¹⁷ reported that people with visual impairment were significantly slower than normally-sighted age-matched controls in visual search tasks in artificial arrays. As the size of the central scotoma increased, there was a greater deficit in search performance. Alternatively, a more naturalistic task involving scene discrimination has shown only minor deficits in AMD patients. ¹⁸ Patients with AMD were better at categorizing natural versus urban rather than indoor versus outdoor scenes, but overall performance was comparable to controls and led authors to conclude that peripheral vision was sufficient to recognize the gist of the scene. 

In an effort to help improve quality of life in patients with AMD, many others have developed image enhancement methods that aim to overcome specific visual deficits. Jacko et al. ¹⁶ found that while background color, number of distracters, and icon size contributed to performance, magnification did not assist search performance for AMD patients. Other groups have tailored image enhancement to the contrast sensitivity of a patient. Loss of foveal vision is associated with a decrease in contrast sensitivity at high spatial frequencies ^19,20 and several engineering-based approaches selectively increase the contrast of these almost visible components. Additionally, in basic research, it has been demonstrated that, for foveal vision, some object frequencies in letter and face images are more important for identification than others. ^21,22 Thus, assuming that the same frequencies are optimal for peripheral vision, some groups have selectively increased the contrast of these critical frequencies. 

Some researchers have reported significant increases in reading speed, ²³ as well as elevated subjective preferences for custom enhanced images. ²⁴ Other groups, however, failed to show any benefit for reading speed, ²⁵ or face recognition ²⁶ with such methods. These custom enhanced images were not preferred and many patients actually read enhanced text at a significantly slower speed. ²⁵  

Image enhancement has been more successful in moving images. Al-Atabany et al. ²⁰ introduced three new image enhancement techniques and implemented an image processing model with a virtual scotoma to compare patient performance in a face detection task. Edge overlays and tinted reduced outlined nature (TRON) algorithms were the most useful in dynamic scenes, while image cartoonization was helpful for spatial feature detection. Fullerton and Peli ^27,28 examined the effects of a method that reweights Moving Picture Experts Group (MPEG) contrast in different image regions and frequency bands. Although they did not directly compare unenhanced and enhanced movies, they reported that in side-by-side comparisons, a moderate level of enhancement was preferred over low or high enhancement levels by low vision patients with a variety of impairments. 

In summary, many studies have examined a variety of image enhancement methods for tasks including reading, face recognition, and simple preference; however, few, if any have directly compared alternative enhancement methods on the same media. We therefore implemented a range of existing and novel image enhancement methods in order to evaluate their utility for people with central vision loss from AMD. We used an objective visual search task to determine the effectiveness of these alternative methods. 

Methods

Participants

Participants were recruited from Vision Rehabilitation Clinic at the Massachusetts Eye and Ear Infirmary (MEEI) in Boston, MA. Seventeen patients with AMD participated in the study. Patient information is given in Table 1. The mean age of the patients was 77 years old. We included patients with a wide range of acuities, contrast sensitivity, and visual field loss. Twelve of the 17 patients had foveal sparing in at least one eye. Visual acuity was measured with the Early Treatment Diabetic Retinopathy Study (ETDRS) letter chart and contrast sensitivity was measured with the Pelli-Robson chart. Values are reported only for the better eye on the assumption that visual search was dominated by this eye. All participants received a score of 25 or higher on the Mini Mental Exam for Visually Impaired Outpatients (adapted from the Mini-Mental State Examination). ²⁹ The study was approved by the institutional review board committees of MEEI and Schepens Eye Research Institute and adhered to the tenets of the Declaration of Helsinki. 

Table 1. 

 

Patient Information and Demographics

Subject	Diagnosis	Age	Sex	Acuity OD	Acuity OS	Contrast Sensitivity	Perimetry OD	Perimetry OS	Perimetry Test
1	Wet AMD	90	M	0.398	1.602	1.05	41/52	40/52	Polar 5–21 Goldman III
2	AMD	77	F	1.079	1.301	1.05	50/52	13/52	Polar 5–21 Goldman V
3	Dry AMD	76	F	0.176	0.176	1.05	45/52	35/52	Polar 5–21 Goldman III
4	Wet AMD	87	M	1.0	0.544	0.9	24/52	32/52	Polar 5–21 Goldman III
5	Dry AMD	68	M	1.301	0.301	1.35	44/52	44/52	Polar 5–21 Goldman III
6	Dry AMD	55	F	0.544	0.602	1.05	43/52	43/52	Polar 5–21 Goldman III
7	Dry AMD	86	F	0.875	0.875	1.05	30/52	26/52	Polar 5–21 Goldman III
8	Dry AMD OD Wet AMD OS	85	F	1.0	0.602	0.6	23/25	20/25	5×5–9 Goldman IV
9	Wet AMD	81	F	1.0	2.114	0.75	32/52	N/A	Polar 5–21 Goldman IV
10	Dry AMD OD Wet AMD OS	84	F	0.602	0.544	0.9	39/52	49/52	Polar 5–21 Goldman III
11	Dry AMD OS Wet AMD OD	75	M	0.875	0.544	1.05	43/52	50/52	Polar 5–21 Goldman IV
12	Wet AMD	64	M	0.544	0.544	1.5	39/52	50/52	Polar 5–21 Goldman III
13	Wet AMD	65	M	0.176	0.097	1.05	46/52	47/52	Polar 5–21 Goldman III
14	AMD OU	86	F	0.602	1.301	1.2	47/52	12/52	Polar 5–21 Goldman III
15	Wet AMD	84	F	0.699	0.544	1.35	49/52	44/52	Polar 5–21 Goldman IV
16	Dry AMD	84	F	0.875	0.875	1.05	38/52	24/52	Polar 5–21 Goldman IV
17	Dry AMD	79	M	0.875	0.544	1.35	42/52	45/52	Polar 5–21 Goldman III

 Data were collected at the Vision Rehab Center at MEEI. Microperimetry data are displayed as a ratio of points seen over total points presented on a microperimetry exam using the Opko SLO/OCT microperimeter. Acuity was measured using ETDRS and contrast sensitivity was measured with Pelli-Robson charts. The highest contrast sensitivity between the two eyes was reported and used in analysis. The final column specifies the particular microperimetry task used for each individual.

Stimuli

Participants viewed a natural scene presented on a 27 inch iMac light-emitting diode (LED) display (Apple, Inc., Cupertino, CA) at a resolution of 2560 × 1440 pixels with a refresh rate of 60 Hz, which subtended 60° by 33.5° at the viewing distance of 57 cm. The stimulus image was presented from a collection of 90 images from the LabelMe Database. ³⁰ The 90 scenes were selected based on content, as well as the number and accuracy of objects labeled. This set comprised both indoor and outdoor scenes, as well as a variety of everyday objects, faces, persons, and buildings. The 90 images were then processed with a series of eight different enhancement methods, resulting in a database of 720 images (eight variations of each of the 90 scenes). Images were scaled up or down in size to fit the full height or width of the screen, without cropping or changing the aspect ratio of the image. 

Image Manipulations

The modification methods were performed in Matlab (Mathworks, Ltd., Natick, MA) prior to data collection and modified images were stored in a database accessed when running the experiment through Psychtoolbox (in the public domain, http://psychtoolbox.org/). ³¹ The eight variations were as follows: 

1.  
   The original image in red-green-blue (RGB) color format, scaled to cover the full 0 to 255 look-up table (LUT) range with the darkest and lightest image pixel;
2.  
   The original image converted to gray scale using the Matlab function red-green-blue to gray (rgb2gray()) and scaled to cover the full 0 to 255 LUT range;
3.  
   An edge enhanced image, performed with a variant of a previously published method. ³² Instead of filtering with a bank of band-pass filters and applying a threshold for like-signed pixels across spatial scales, the image was processed with a Laplacian of Gaussian filter (Mathworks, Ltd.) with a SD of 1 pixel, using Matlab's fspecial (‘log') function. This broadband filter inherently selects pixels that are correlated across scales and bypasses the need for filtering at multiple spatial scales. The filtered image was then thresholded at ±2 SDs of luminance to produce black-white signed contours at the location of edges in the image. These black-white edges were superimposed on the original RGB image;
4.  
   An image with segmented objects. Image areas labeled by users of the Labelme project ³⁰ were classed as objects; areas not labeled were classed as background. There was a range of 2 to 58 objects labeled in the set of 90 images with a mean of 16 labeled objects in each image. It is important to note that the images varied in scale and included photographs of outdoor and indoor scenes, thus, creating large differences in the size of the labeled objects. For target objects (those that were searched in the experiment), the mean labeled object occupied 0.85% (SD 0.93) of the total image area. The total area of the image that consisted of labeled objects was 57.91% (SD 49.57). Object areas were presented at their original contrast, and background areas were set at mean luminance (82 cd/m²). This process served to reduce crowding between background and objects that might impair the visibility of objects in the peripheral visual field;
5.  
   An image with a 50% contrast masked background and segmented objects. As in (4), except that background areas were presented at 50% contrast. This method attempted to reduce crowding of objects by the background, but to leave intact information about scene layout or gist;
6.  
   Local root mean square (RMS) contrast enhanced RGB image. Local RMS contrast was computed using the method described in our previous work. ³³ In brief, local RMS contrast is computed as the local, Gaussian weighted SD of luminance divided by the local mean luminance. In the present method, luminance and color were extracted with Matlab's function rgb2yuv( ). Contrast operations were performed on the luminance (y plane) image. Local mean luminance (σ _x,y = 1°) was subtracted from the image and local SD was computed for all pixels. Each pixel was then divided by the local SD, which relatively increased the absolute values of pixels in areas of low RMS contrast and relatively decreased the absolute values of pixels in areas of high RMS contrast. This process produces an image with a flat distribution of local contrast. Color was restored to the image with Matlab's function yuv2rgb( ) and the final image was scaled to the cover the full 0 to 255 LUT range. This method maximized image contrast at all locations in the image, avoiding global saturation by the lightest and darkest pixels;
7.  
   A local contrast enhanced gray scale image. The same as (6), except a gray scale image was created by omitting the stage of color conversion; and
8.  
   A within-band local contrast enhancement. The same as (6), except that local contrast normalization was performed separately on a set of narrow spatial scales within the image. Each image was filtered with log-cosine band-pass filters with a 1-octave bandwidth. 

    

   where A is the amplitude, ω is spatial or temporal frequency, and ${\omega }_{\mathrm{peak}}$ is the peak frequency. These filters have the desirable property of summing to unity, so that the sum of all band-pass images reproduces the original. Local RMS contrast normalization was applied to each band-pass filtered image, and then the summed image was converted to RGB and scaled cover the full 0 to 255 LUT range.

Figure 1 depicts examples of the eight different versions of image modification for a single experimental image. 

Figure 1. 

 

Example of eight different image modifications in one particular scene. Participant was asked to locate the salt shaker.

Procedure

Observers completed a total of 80 search trials, 10 trials for each modification method, in random order. The assigned object label from the LabelMe database was used as the identity of target objects. At the start of each trial, a computer generated voice (“Alex,” a voice option built in to the Mac OSX, the operating system native to Apple computers; Apple, Inc.) spoke the identity of the search target in the image. Participants were then required to freely view the search image with the goal of locating the announced target. The targets ranged in size and location across the 80 different trials. Figure 2 shows a heat map of spatial distribution of targets over the entire database of 90 images. Participants were instructed to use a mouse controlled pointer to indicate the position of the target within the image. The pointer was displayed as a large red dot (diameter of 1.5°, 64 pixels), easily visible to participant with limited acuity. The participant's response initiated the next trial. No time constraint was enforced during the task; instead participants were encouraged to continue to search the image until they were confident they had located the target. The time duration to locate the target and use the mouse to click on the location of the target was recorded. 

Figure 2. 

 

Spatial heat map of target object distribution across 90 different scenes used in the search task. Target pixel density was measured as the total number of occurrences of a target object at each pixel location in the display. Red, “hotter” areas depict regions of the display that more frequently contained a target object.

After each subject had completed 80 trials of the visual search task, they were asked to rank enhancements in order of preference. Eight versions of the same image were presented in each cell of a 3 × 3 grid that filled the screen with the center grid left empty. One scene was randomly selected from the database and each of the eight modification methods was applied to it. Observers were asked to consider which image they thought was “most aesthetically pleasing” as well as “of most functional value for performing an everyday task like watching television.” The subject was asked to click a mouse pointer on their preferred image. The selected image was then removed and the observer was required to click on the preferred image from the remaining images. This process was repeated until all images had been selected. Each subject repeated this ranking task for six different scenes. 

Results

Image Enhancements

Figure 3 shows the mean search time in seconds for each scene, averaged across all observers and all modification methods, error bars show 95% confidence intervals (CIs). Based on the observed variability in search duration across scenes, we further examined specific image features that may be correlated with this measure. We considered the total number of objects labeled in the scene, target area, and edge density. The number of objects labeled in the scene was extracted from the LabelMe database and varied across the 90 scenes (2–58 objects; mean of 16 objects). Target area was taken as the total number of pixels that made up the instructed search target. Finally, edge density was a measure of the total number of edges in the scene (found using the Canny edge detector in Matlab) over the total number of pixels in the scene. ³⁴ The mean normalized search duration across all observers for a particular image was used as a measure of performance (duration was first normalized to individual performance based on each participant's mean across all 80 trials). An average of 13 observers viewed each scene (including all modified versions of the scene). We found a significant correlation between target size and normalized search duration (r = −0.34; P = 0.001), but no significant relationship between number of objects in the scene and duration or edge density and duration. Subsequently, we attempted to fit the search duration data with a generalized linear model, including the above mentioned image features and interaction terms, but none of the predictors were significant. 

Figure 3. 

 

Mean search duration in seconds for each of the 90 scenes in the database. Duration was averaged across all observers and all modification methods for a particular scene. Error bars depict 95% CIs.

The observed interscene search duration variation could mask differences between modification methods. In order to correct for this variation, search duration was normalized to the average search duration for that scene. For each trial, the search duration was divided by the mean duration for that scene across all participants and trials. This correction minimized the possibility that a particularly difficult search scene could bias the results (recall that all participants did not search all scenes with all modification methods). We took an additional step to normalize search duration to the individual participant's own mean in order to isolate the effect of our image manipulations. There were differences in mean duration between participants, which may be attributed to differences in age, visual function, reaction times, and mouse control. Normalizing the duration to each individual's mean duration across all trials accounted for these differences. An ANOVA showed that there was no significant difference in the normalized search duration across the eight different image modification methods, for each individual participant as well as when collapsed across all participants (P = 0.96; n = 17). Additionally, subjects were divided into a younger (mean age = 73.12 years) and older cohort (mean age = 85.75 years) to examine the interaction of age on the eight different image enhancements. We excluded patient six in this analysis due to a significant difference in age (greater than two SDs). A two-way ANOVA showed no effect of age on search duration (P = 0.93, n = 16) over the eight different modifications. The mean search duration across all participants for each modification method is displayed in box and whisker plots in Figure 4A. The normalization correction did not change the results, there was no significant difference in the individually normalized search duration across modification methods, as shown in Figure 4B. 

Figure 4. 

 

Mean search duration and error trials across eight different image modification methods. Mean search duration for a particular modification from each individual participant was used to compare across different modification methods. The red line depicts the median of the distribution of the participant means. (A) compares normalized duration calculated from the mean duration for each image across all subjects. (B) shows the individually normalized data. Total number of error trials for each modification method from individual participants make up the distributions depicted in (C). The black lines depict the median error rate for each modification method across all participants.

We also evaluated search performance with error rates. Any trial in which the observer clicked the cursor on a background area or object other than the search target was classified as an error trial. An ANOVA showed that there was no significant difference in the total number error trials across the eight different image modifications within a single subject. Additionally, when data were collapsed across all subjects, there was no significant difference across image modifications, as shown in Figure 4C (Note: Participants 1 and 2 were not included in the analysis for number of error trials because the location of the cursor was not successfully recorded for these two participants, thus, n = 15 for this analysis). A two-way ANOVA showed no effect of age on number of error trials (P = 0.92; n = 14) over the eight different modifications. 

Intersubject Parameters

Next we used regression analysis to examine how intersubject differences, including patient age, visual acuity, and contrast sensitivity in the better eye, as well as the scotoma size in the better eye contributed to performance. Table 1 shows information for each patient. Although the patient completed the task binocularly, acuity, contrast sensitivity, and scotoma size of the better eye were used under the assumption that the better eye dominates visual function. ^35,36 Scotoma size was included as a ratio of points seen over total points presented on a microperimetry exam using the Opko Scanning Laser Ophthalmoloscope/Optical Coherence Tomography (SLO/OCT) microperimeter (Opko Health, Inc., Miami, FL). Sixteen out of 17 participants completed the 5° to 21° polar exam, which tested a total of 52 points in the central 21° visual field. The other patient completed a 5×5-9° degree square grid that tested 25 points within the central 9° visual field. The display of the dot stimuli ranged from Goldman III to Goldman V, with the majority of the patients tested on the Goldman III (Table 1). To address any discrepancy between differently sized targets in the microperimetry data, we completed all of the following analysis including only those participants tested with the Goldman III sized target (n = 11), but found no difference in any of the reported results, as compared with the entire sample of subjects (n = 17). 

Univariate analysis showed a significant positive correlation between acuity and search duration, and between age and search duration (R = 0.62, P = 0.007, and R = 0.55, P = 0.02, respectively); however, all other variables failed to show any significant correlation (Fig. 5). To further examine the contribution of these variables and their interactions on performance (i.e., duration and number of error trials) we used multivariate regression analysis. 

Figure 5. 

 

Significant univariate correlations considering age and acuity with search duration. Different colors represent the eight different image modification methods used. Each point represents the mean duration for a particular modification method within a single participant.

Multivariate regression produced a predictive model for search duration. Stepwise regression analysis identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interaction terms between scotoma size and age, age and acuity, contrast sensitivity and acuity, and contrast sensitivity and scotoma size (Table 2). The model fit the data with an adjusted R ² of 0.822 (P < 0.001). The model without the interaction terms resulted in an adjusted R ² of 0.47 (P = 0.01). The other performance variable, total number of error trials, could not be predicted by the linear model (R ² = 0.41, P = 0.03). 

Table 2. 

 

Intersubject Parameters

Considered Predictors	Univariate Analysis R Coefficient	Univariate Analysis P Value	Multiple Linear Regression Model Coefficient	Multiple Linear Regression P Value
Age	0.5472	0.0230*	0.000684	0.6996
Acuity	0.6239	0.007*	−10.83	0.0005*
Scotoma size	0.02	0.9119	28.10	0.0019*
Contrast	−0.1824	0.4835	−0.452	0.6206
Interaction terms
Age*acuity	0.6960	0.0019	0.1249	0.0001*
Age*scotoma size	0.0770	0.7688	−0.27896	0.0015*
Age*contrast	0.1792	0.4914	−0.002826	0.7819
Acuity*scotoma size	0.3105	0.2251	2.542	0.1685
Acuity*contrast	0.5363	0.0265	1.96554	0.0153*
Scotoma size*contrast	−0.0358	0.8914	−6.48805	0.0285*

 Age, acuity, scotoma size, and contrast were all considered to be predictors for search duration. The table displays values for both univariate correlation and multiple linear regression. In the multiple linear regression model, the significant predictors for search duration were scotoma size, acuity, and interaction terms between scotoma size and age, age and acuity, contrast sensitivity and acuity, and contrast sensitivity and scotoma size.

*   indicates P values < 0.05 and a significant predictor for search duration.

Modification Preference

Within an individual subject, image preference data showed no direct correlation with performance measurements; however, there was a trend for participants to choose the most modified images as least preferred. A nonparametric Freidman test showed that there was a significant difference in rankings across the eight different modifications (P < 0.001), and a follow up multiple comparison test showed that both image modifications 1 and 3 (the original image and the edge enhanced image) were significantly more preferred than the image modifications 4, 5, 6, and 7. Additionally, image modification 8 (within band local contrast enhancement) was significantly more preferred than image modifications 4 and 7. Image modification 2 was not significantly different from any other modification in post hoc analysis. The cumulative rankings are shown in Figure 6, with error bars depicting the variability (SD) across patients. The Freidman test accounted for within patient variance from repeated measures. Thus, images that subjectively minimally changed the original image were most preferred, even though visual search was unaffected by any image manipulation. 

Figure 6. 

 

Image preference data. Image rankings were given a score of 0 to 8, based on the order in which the image was selected. Data displayed are the mean ranking score across all participants. Higher scores indicate a higher preference for that particular manipulation. Error bars depict the SD of the ranking across all participants.

Discussion

Our data suggest that the image modification methods developed in this study may not be a useful method of improving visual search behavior in patients with central vision loss. We found no significant difference in search duration or total number of errors across eight different manipulations of natural image contrast. We cannot rule out the possibility that image modification may assist tasks other than visual search. In a subjective comparison of rankings there was a modest trend for patients to prefer the original, local band limited RMS and edge enhanced image over other image modifications. Across all methods, a multivariate model revealed six significant predictors for normalized search duration including scotoma size and acuity as well as interactions among scotoma size, age, acuity, and contrast. Our experimental setup employed a variety of modification techniques, some of which have previously been used in the literature. Why then was image modification not more helpful? 

Three of the eight image manipulations increased local contrast. The objective of these methods was to maximize the visibility of all areas of the image. However, instead of improving image appearance, this technique may have worsened the patient's subjective assessment of the image by increasing the incidence of crowding. Crowding is an impairment in identification of objects and letters when they are surrounded by other features, and it is particularly problematic in the periphery. The area over which crowding impairs identification increases with eccentricity ³⁷ and with contrast, ³⁸ for a recent review see Whitney and Levi (2011). ³⁹ Because patients with AMD rely on peripheral vision, crowding is a fundamental problem and any increase in contrast may have caused elevated levels of crowding. ⁴⁰ With our normalized images, the contrast was higher and more homogeneous than in the original images. While this potential elevation in crowding did not measurably affect search duration or error rates, it may have contributed to lower preference ratings. 

Two other image manipulations directly tested the effect of crowding on the patient's ability to search the visual scene. Previous research in normally-sighted observers has shown that search times in cluttered scenes are longer than in non cluttered scenes. ^41,42 Ho et al. ⁴³ that found search performance declined with increase in both age and clutter in a naturalistic search task. Additionally, other work has shown that background provides little information to low-vision patients in object recognition, and patients perform better with isolated objects. ⁴⁴ We, therefore, included masked images in which only objects that were identified and segmented in the scene by LabelMe users were fully visible. The rest of the image was classed as background and was reduced in contrast. The segmented images produced a surprising result since participants did not show a shortened search duration for these images even though they contained a small number of clearly segmented objects (between 2–10 objects per image) on a relatively uncluttered gray or 50% contrast background. We hypothesized that this segmentation would assist peripheral vision by lessening the effect of crowding, making the scene less cluttered and easier to search. However, the results did not support this hypothesis. In addition to being unhelpful for visual search, both segmented images were also ranked lowest for subjective preference. 

It is possible that the lack of improvement in visual search with segmented scenes may be attributed to a decrease in contextual information about the target object when it is removed from its background. Several groups have shown that context can be useful during visual search in artificial ^45,46 and natural scenes. ^47–51 Interestingly, the number of error trials was lower for the segmented condition with 50% contrast background, allowing for some contextual information about the scene, but search duration was longer than for the fully segmented images, with 0% contrast background. 

Although there was no significant difference in search duration between color versus gray scale images, color images were subjectively ranked higher. The naturalistic search task we employed allowed for semantic and contextual cues to contribute to performance (i.e., patients were instructed to locate a recognizable object within a recognizable scene). Consequently, we hypothesized that knowledge about the likely color of a given target object may have been a useful tool when completing the task (e.g., a red fire hydrant would be easier to locate in a colored image). Previous work has demonstrated color enhances visual memory and recognition in natural scenes. ^52,53 More specifically, other groups have examined the effect of color on object recognition in low-vision patients and found that colored images improved performance. ^44,54 However, we found no evidence of this benefit for patients with central vision loss, perhaps because chromatic sensitivity is reduced in the peripheral visual field. ⁵⁵  

Due to the small sample size of this study, one may question the power of the null result. However, even though there were only 17 patients, the results clearly demonstrate that a large benefit of image modification cannot be expected and may not be clinically useful. Nevertheless, it is interesting to note that the qualitative preference data mirrored the search data. There was a trend for participants to prefer the original images and dislike the most clearly modified images, especially ones in which the image statistics were most changed (i.e., manipulations 4, 5, 6, and 7). These results suggest that visual processing does not adapt to visual impairment. Some have argued that the visual system is optimized to the natural statistics of images through evolution and development; perhaps these entrenched preferences and selectivity for natural image statistics persist following visual impairment. ⁵⁶ It is possible that low-vision patients may only benefit from enhanced/modified images following a period of rehabilitation training while the patient learns to use the new sources of visual information. 

Although this study found no significant effect of image modification on performance, as quantified by search duration and error, none of the manipulations made performance significantly worse. After collapsing data across all eight image modifications, we found a significant correlation between age and duration, consistent with previous research. ^{15,43,57–59} We also found a significant correlation between acuity and duration, which has not been found to be correlated with search performance in normally-sighted observers. ⁶⁰ Surprisingly though, there was no significant correlation between total scotoma size and search duration. This may be attributed to the fact that we had patients complete the task binocularly, and 12 out of the 17 patients had foveal sparing in at least one eye. However, further analysis showed that there was no significant correlation between scotoma size and search duration with the group of patients that had no foveal sparing. A two-sample t-test showed that acuity, search duration, and error rate were not significantly different between the non-foveal and foveal sparing group. 

Our multivariate model identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interactions among scotoma size, age, acuity, and contrast. It is likely that this interaction may have been attributed to the univariate findings in acuity and duration. We note that neither our univariate nor multivariate analysis was predictive of total number of error trials. It is possible that error rates reflect additional non-visual factors such as perseverance that are not directly related to early visual parameters. 

In addition to examining patient factors that may have affected performance across the different parameters, we also examined image features that may have contributed to the observed variability in search duration across different image scenes (Fig. 3). The selected target size varied in size across the 90 difference scenes (Fig. 2). Previous work has provided mixed evidence for an effect of target size on search duration. ^61–63 We found a significant correlation between target area and search duration. 

As mentioned above, there was no significant effect for the modified version of segmented objects; however, there is a possibility that differences in clutter across the individual scenes may have confounded the results across modifications. Search duration has been known to increase proportionally with the number of objects in a search scene. ^61,64 We further examined this possibility by looking at the correlation between number of objects labeled in the scene and search duration across all image modifications, but found no significant relationship. This result may be due to our use of the LabelMe online database object count (i.e., some scenes were more accurately and completely labeled than others). In another attempt to assess the effect of clutter in scenes, we also considered edge density of the scene. We measured edge density (of the original image) and found only a slight positive correlation with the average search duration for a particular scene. 

In order to improve the ability for AMD patients to interact with their environments, a more complete understanding of visual search in natural environments is needed. We believe that the negligible impact of the image modification techniques on patients' search performance primarily stems from our poor understanding of search under such conditions. A clearer understanding of the multiple sources of information used by observers when performing naturalistic search, and how they are combined, must inform the development of enhancement. For example, knowing that observers rely heavily on contrast differences to identify potential targets, tells us both that contrast enhancement may be useful, but also that, for example, isolation of all elements (e.g., to minimize crowding) may actively hamper search by presenting a potentially large number of artificially isolated, salient targets. Evaluating these considerations is difficult, but considerably more tractable in the context of a particular task like visual search. It may be that a “one size fits all” approach will not work for enhancement, and different tasks may rely on different forms of visual manipulation to optimize patient performance. 

Footnotes

 Supported by National Institute of Health Grants R01EY018664 and R01EY019281, and by the NIHR Biomedical Research Centre at Moorfields Eye Hospital NHS Foundation Trust (SCD).

Footnotes

 Disclosure: E. Wiecek, None; M.L. Jackson, None; S.C. Dakin, None; P. Bex, None