Evaluation of a Portable Collision Warning Device for Patients With Peripheral Vision Loss in an Obstacle Course

Abstract

Purpose.: A pocket-sized collision warning device equipped with a video camera was developed to predict impending collisions based on time to collision rather than proximity. A study was conducted in a high-density obstacle course to evaluate the effect of the device on collision avoidance in people with peripheral field loss (PFL).

Methods.: The 41-meter-long loop-shaped obstacle course consisted of 46 stationary obstacles from floor to head level and oncoming pedestrians. Twenty-five patients with tunnel vision (n = 13) or hemianopia (n = 12) completed four consecutive loops with and without the device, while not using any other habitual mobility aid. Walking direction and device usage order were counterbalanced. Number of collisions and preferred percentage of walking speed (PPWS) were compared within subjects.

Results.: Collisions were reduced significantly by approximately 37% (P < 0.001) with the device (floor-level obstacles were excluded because the device was not designed for them). No patient had more collisions when using the device. Although the PPWS were also reduced with the device from 52% to 49% (P = 0.053), this did not account for the lower number of collisions, as the changes in collisions and PPWS were not correlated (P = 0.516).

Conclusions.: The device may help patients with a wide range of PFL avoid collisions with high-level obstacles while barely affecting their walking speed.

Severe peripheral visual field (VF) loss, such as the concentric loss caused by conditions like retinitis pigmentosa (RP) and glaucoma and the loss of the same half of the visual field in both eyes (homonymous hemianopia [HH]) have been correlated with poorer measures of mobility,^1–8 for example, by increasing the likelihood of falls and collisions.^9–13 As therapeutic vision restoration treatments are still in their infancy, rehabilitation approaches using assistive technologies are often viable alternatives for addressing vision loss-related mobility challenges. 

Over the past few decades, many electronic travel aids (ETAs) for collision avoidance have been proposed.¹⁴ The basic approach of these devices is to acquire scene information by ultrasound or image sensors, interpret the information with processing circuits, and then provide the visually impaired users information through audio or vibrotactile modes to aid obstacle avoidance and safe navigation. We developed a portable, video camera-based collision warning device¹⁵ that detects impending collisions by processing videos acquired from a single camera, using a novel computer vision algorithm¹⁶ and delivers collision warnings through simple, intuitively understandable auditory signals (Fig. 1a). The device estimates collision risk based on the relative motion between the camera and objects in its field of view (FOV). When the device and an obstacle approach each other, no matter which is in motion, the collision risk is resolved into two components: collision point (spatial) and time to collision (TTC), the temporal component. A collision warning, in the form of an audible “beep,” is issued for each processed frame only when the TTC is short and the collision point is close to the user (Fig 1b), based on preset thresholds. With a wide angle camera (≈90° and 55° FOV in horizontal and vertical directions, respectively) and customized hardware, our device can detect collision threats coming from multiple directions at 20 Hz, which is sufficient for collision avoidance when walking. 

Figure 1

 

(a) Portable, battery powered video camera-based collision warning device. (b) The concept of collision risk evaluation as seen by the user (located at the center of the horizontal axis). The collision warning device divides the collision risk with the approaching object into two components: TTC and collision point. The device issues warnings when TTC and collision point are below a preset threshold (high-risk area). (c) The device is ergonomic and can be carried easily in a front pocket.

Our approach to collision detection contrasts with most ETAs featuring ultrasound or infrared range sensors or stereo cameras that are designed primarily for the ultra-low vision or blind users. They generally tend to provide detailed spatial information about the surrounding environment through alternative sensory pathways (auditory or tactile). However, the surrounding objects are not necessarily obstacles unless they lie close to the path that the user is moving along. If, for collision avoidance purposes, warnings were given based on proximity, there would be too many intolerable false alarms for people who still had residual vision. Electronic travel aids such as vOICe¹⁷ require users to interpret information constantly presented through the alternative sensory mode, which likely cost the users a substantial amount of sensory and cognitive resources. For navigation, it is necessary to be aware of the surrounding environment, and correct path planning can help reduce the chances of collision. Residual vision is very valuable for the patients and always used by them to look around. However, unforeseen collisions still occur at a high rate and affect patients' mobility. In a study involving 109 low-vision patients, Lovie-Kitchin et al.¹⁸ showed that mobility started to be affected when visual field diameter was below 70°. Even patients with quite large residual VFs, as with HH, can face mobility challenges.^7,19,20 Avoiding obstacles that the patients often miss spotting is specifically addressed by our device. As simple audio warnings are provided only in the event of possible collisions, this method entails only minimal sensory and cognitive load as compared to those methods that convey complex visual information through alternative sensory pathways while requiring the users to interpret the collision risk. More importantly, the device can potentially complement the user's habitual mobility aids, such as the widely used and effective long cane, by detecting otherwise missed, high obstacles. Furthermore, compared to a large number of proposed ETA solutions, an advanced computer vision algorithm developed by us¹⁶ makes a highly portable device possible with current technology (Fig. 1c). 

Obstacle courses, in a variety of designs, have been commonly used for mobility assessment of visually impaired subjects,^2,18,21 as well as to evaluate mobility aids.^22–25 Among those ETAs proposed in the literature, only a few range sensor based devices have been evaluated with human subjects.^24–26 The other ETAs we are aware of include video camera-based ones, report no or limited human evaluation without clearly defined control conditions.^14,27,28 Without control conditions it is difficult to evaluate the added value of an assistive device for the users. Many seemingly workable devices may not help the visually impaired users to achieve better mobility than without the devices. In order to quantitatively determine the effect of our collision warning device on low-vision patients' mobility, a controlled study was conducted using a high-density obstacle course involving subjects with various degrees of peripheral vision loss, primarily due to RP or HH. 

Methods

Obstacle Course Design

Figure 2a shows the schematic layout of the obstacle course, set up in a large meeting room. The obstacle course was enclosed on three sides, and the fourth side was left open for people to enter and exit. A row of tables was placed in the middle of the room, creating a barrier, such that the walking path through the course became a loop that was approximately 41 meters long. Forty-six stationary objects at different heights, from floor to head level, served as the obstacles to avoid. These stationary obstacles included 32 inflatable trees, each approximately 1.8 m tall and with a radius of approximately 0.3 m at its base, 9 hanging obstacles (approximately 1.4–1.6 m from the ground) such as plastic flags and paper bags, and 5 floor level obstacles (empty cardboard boxes) that were less than 0.15 m in height. The device, when worn on the chest (at an average height of approximately 1.4 m), was able to cover the height range of 0.75 to 2 m from 1.2 m away, which included all the obstacles except the floor level obstacles. Each obstacle was assigned a specific location in the obstacle course according to the map in Figure 2a. A pedestrian walking in the opposite direction of the subject served as a moving obstacle that the subjects encountered at least once per loop at unpredictable locations in the obstacle course (Fig. 2b). The obstacles were arranged in such a manner that there was only one collision-free path through the course. 

Figure 2

 

(a) Obstacle course layout. A barrier of tables (blue rectangles) was set in a long meeting room with a variety of obstacles. The only safe walking path through the obstacle course is shown as a dotted line. All dimensions are in meters. (b) A snapshot of the experiment in progress. An experimenter walked behind the subjects for safety purposes and to record the mobility performance. A pedestrian walked in the opposite direction from the subject, acting as a moving obstacle that encountered the subject multiple times under each testing condition.

Experimental Protocol

Visually impaired subjects walked through the obstacle course under two conditions: with and without the collision warning device. The device was attached to the chest by using a flexible harness (Fig. 3), that was well suited for both male and female subjects with different body sizes. For each condition, the subjects walked four loops successively in one direction in the obstacle course. The direction of the walk was reversed for the alternate condition. The order of the conditions with or without the device and the walking direction was counterbalanced (four possibilities). The subjects did not use any other walking aid during the experiment. While walking through the course, the subjects also performed a secondary task that was intended to create distractions and increase the difficulty of obstacle avoidance, so that a relatively large number of collisions could occur during the short sessions, even for subjects with otherwise good obstacle avoidance skills. The secondary task was a 1-back recall task, a variation of the n-back task commonly used in human factor studies that measure performance in the presence of divided attention.^29–31 In our study, a series of random single digit numbers was played through speakers at a rate of one every three seconds, and the subjects were asked to report the previously played number after hearing the current one. The impact of such a secondary task on collision avoidance in our obstacle course had been confirmed with normally sighted pilot subjects wearing tunnel vision goggles. 

Figure 3

 

During the experiment, the collision warning device was mounted with a flexible harness at chest level. The use of such a harness ensured the mounting was consistent for different body sizes.

Time to complete each loop, and the number and type of collisions were recorded manually during the experiment by the experimenter walking behind the subject. The secondary task responses were recorded by a mini camcorder carried by the experimenter. Any contact with the obstacles was counted as a collision. The subjects were instructed to yield to the pedestrian. In case they did not see him, the pedestrian would gently brush against the subjects before stepping away. This was counted as a collision. 

Before the experiment, each subject was trained for the primary walking, and the secondary number recall tasks in a miniaturized version of the obstacle course. The subjects were instructed to walk at a comfortable pace, scan the surroundings using their residual vision. It was made clear that any contact with the obstacles would be considered as a collision. When failing to recall the number, they were instructed to wait for the next number and start over. The training session continued until the subject felt ready to perform the experiment. Typically, task instruction and practice took approximately 30 to 45 minutes. 

Participants

A total of 25 subjects with significant peripheral vision loss, including HH and tunnel vision, participated in the study (Table). The causes of HH were either stroke or brain injury, having occurred between 10 months and 25 years prior to participation in the study. None of the HH patients had spatial neglect. 

Table

 

Characteristics of the Study Population

 

The predominant cause of tunnel vision in our study population was RP (12 of 13 subjects). One subject had optic nerve dysfunction, resulting in light perception only in the left eye, a residual VF of 32° in the right eye, and a low visual acuity (VA) and contrast sensitivity (CS) (1.414 logMAR and 0.725, respectively). The VF size for each subject was also quantified for statistical analysis based on the seeing area on the VF plot. After these plots were digitized, VF size was first counted in terms of number of pixels on the plot and then converted to the units of degrees squared. For the tunnel vision subjects, VF was the sum of central field and peripheral islands. Overall, for the tunnel vision patients, the central visual field size ranged from 6° to 32° in horizontal diameter, with three subjects having peripheral islands. Two of the three subjects had an island in the lower right field at approximate eccentricities of 25° and 70° and 29 and 31 deg² in size, respectively. The third had multiple islands scattered 35° away from the center on the top, right, and bottom (total size 19 deg²). Six of the 13 tunnel vision subjects almost always used a long cane when they were by themselves or in unfamiliar areas, and one subject relied on a guide dog for mobility. The remaining six tunnel-vision subjects rarely or never used any mobility aid. Two of the 12 HH patients used support canes. All of the subjects were comfortable walking without their mobility aids in the obstacle course. 

This study was carried out according to the tenets of the Declaration of Helsinki. All participants volunteered for the study and signed the informed consent form approved by the Human Subjects Committee of Massachusetts Eye and Ear. 

Data Analysis

In order to account for the variability in the natural walking speeds of the study subjects, we computed their percentage of preferred walking speed (PPWS) relative to a baseline value.⁶ The baseline walking speed was obtained before starting the trial, as the subjects walked a distance of 27 m in a straight line without any obstacles while performing the secondary task (1-back number recall). The number of collisions and PPWS were compared within subjects when walking with and without the device. Because the low-level obstacles were out of the FOV of the device, collisions caused by them were excluded from the primary analyses and were reported separately. Responses to the secondary task were manually scored from the audio recordings of 22 subjects, and the error rate (incorrect responses/total trials) was computed for each condition. Secondary task responses were not available for three subjects due to recording failures. 

Statistical analysis was performed using SPSS version 11 software (SPSS, Inc., Chicago, IL, USA). PPWS data were normally distributed, whereas collisions were not (Shapiro-Wilk test, P < 0.05). A large variability in the visual field size likely caused the nonnormal distribution of collisions. For analysis of normally distributed variables, repeated measures ANOVA, paired and independent sample t-tests, and Pearson correlation coefficients (r) were used predominantly. Nonnormal data were analyzed using the Wilcoxon signed rank test (paired differences), Friedman test (differences in groups of related samples), Mann-Whitney U test (differences in unrelated samples), and Spearman's coefficient of correlation (r_s). Multiple regressions (linear) were performed to determine the effect of visual functions on mobility. Outcomes with a P value of <0.05 were considered statistically significant. 

Results

Percentage of preferred walking speed did not change significantly between the loops of the obstacle course [overall mean PPWS 51%; F_(3,72) = 0.311, P = 0.818], and there was no interaction between the loop and device factors [F_(3,72) = 1.055, P = 0.374] (Fig. 4a). Similarly, total collisions (with and without the device) did not change significantly between the loops [overall median collisions = 3, interquartile range (IQR) = 9; Friedman χ² test (3) = 3.44, P = 0.329] (Fig. 4b). 

Figure 4

 

Primary mobility outcomes for all subjects with peripheral vision loss broken down by loops of the obstacle course. (a) There was no significant change in the PPWS between the loops of the obstacle course (P = 0.818). (b) There was no significant change in total collisions (collapsed over device conditions) between the loops (P = 0.329). Error bars represent data ranges with outliers excluded. Outliers are the points beyond 1.5 × IQR.

Figure 5 shows the primary mobility outcomes of tunnel vision (TV) and HH subjects separately. Error bars show 95% confidence intervals after correcting for between-subject variances while preserving the within-subject differences.^33,34 Preferred percentage of walking speed did not change significantly for either subject group with and without the device [mean PPWS TV without device = 47% and 44% with the device, F_(1,12) = 1.648, P = 0.224; HH without device = 58% and 54% with the device, F_(1,11) = 2.366, P = 0.152] (Fig. 5a). Homonymous hemianopia subjects walked a little faster than TV subjects, but the difference between the two groups only approached significance [mean PPWS TV = 46%; HH = 56%; F_(1,23) = 3.818, P = 0.063]. The interaction between the device condition and patient groups was not significant [F_(1,23) = 0.075, P = 0.787]. 

Figure 5

 

Mobility outcomes for tunnel vision (TV) and hemianopia (HH) subjects. (a) Percentage of preferred walking speed did not change significantly between device conditions for either subject group (TV: P = 0.224; HH: P = 0.152). Error bars represent 95% confidence intervals of the mean PPWS. (b) Collisions were reduced significantly with the device for both of the groups (TV: P = 0.002; HH: P = 0.011). Tunnel vision subjects had more collisions than HH subjects under both the conditions (without the device: P = 0.005; with the device: P = 0.002). Error bars represent 95% confidence intervals of the median number of collisions.

Collisions were reduced significantly with the device for both of the groups (TV median without device = 16 and a median of 9 with device, P = 0.002; HH median without device = 2.75 and a median of 0.75 with the device, P = 0.011) (Fig. 5b). Overall, TV subjects had significantly more collisions than HH subjects (P = 0.002). 

Considering the high inter- and intragroup variability in the observed data, especially in the number of collisions, we examined the device's effect on each individual subject by using scatterplots (Fig. 6). There was a strong correlation between collisions with and without the device (r_s = 0.946, P < 0.001) when all subjects were included. This was also the case for PPWS (r = 0.88, P < 0.001). According to the slopes of the linear fitting lines, when the device was used, there were approximately 37% fewer collisions (slope = 0.63, P < 0.001), and the PPWS barely changed (slope = 0.93, P < 0.001). Statistically, collisions were significantly reduced with the device from a median value of 6 to 3 (P < 0.001), and the average PPWS was reduced from 52% to 49% when walking with the device, which approached significance [F_(1,24) = 4.16, P = 0.053]. There were no significant differences in collisions due to floor-level objects between the two conditions (mean 3.2 and 2.72 without and with device, respectively; P = 0.553). For stationary obstacles only (not including floor-level objects), the average collisions dropped from 13 to 7.8 with the device. Only 8 of 25 subjects collided with the pedestrians at least once under either condition, and for those 8 subjects, average collisions with the pedestrians dropped from 1.38 to 0.38 with the device. 

Figure 6

 

Between conditions correlation and effect size. (a) Collisions with and without the device were significantly correlated (r_s = 0.946; P < 0.001). All data points lie below the y = x line, indicating no subject had more collisions with the device. The slope of the fitted (dashed) line indicates 37% fewer collisions with the device. Overall, the median collisions were reduced from six without to three with the device. (b) Percentage of preferred walking speed also was significantly correlated between the two experimental conditions (r = 0.88, P < 0.001). Data points are close to the y = x line. The slope of the fitted (dashed) line indicates that the reduction in PPWS with the device was much smaller compared to the reduction observed in collisions. The average PPWS was reduced from 52% without to 49% with the device.

Collisions and PPWS were negatively correlated with (r_s = −0.507, P = 0.01) and without (r_s = −0.48, P = 0.015) the device (Fig. 7a). There was no correlation between change in collisions and change in PPWS (r_s = −0.136, P = 0.516) (Fig. 7b). 

Figure 7

 

Correlations between mobility outcomes. (a) Collisions and PPWS were correlated in both without the device (r_s = −0.48; P = 0.015) and with device (r_s = −0.507, P = 0.01). (b) Scatterplot of the changes in collisions versus changes in PPWS showed no correlation (r_s = −0.136, P = 0.516). This indicates slower walking speed is unlikely to be the cause of the lower number of collisions when walking with the device.

Impact of the secondary task on the mobility of 12 normally sighted subjects in a pilot study was significant. Collisions increased by 25% (P = 0.038), and PPWS decreased from 49 to 44 (P = 0.046) when performing the secondary task. In the case of visually impaired subjects, the secondary task error rates were correlated with collisions both with (r_s = 0.646, P = 0.002) and without (r_s = 0.645, P = 0.002) the device. Secondary task error rates did not change significantly between device conditions, increasing slightly from a mean of 0.31 to 0.33 [t(21) = −1.236, P = 0.24, paired t-test] with device (Fig. 8a). Changes in the secondary task error rates were not correlated with changes in collisions (r_s = −0.23, P = 0.31) (Fig. 8b). A moderate correlation was seen between the change in secondary task error rate and change in PPWS (r_s = −0.45, P = 0.04). 

Figure 8

 

Secondary task analysis. (a) Secondary task error rates increased slightly (but not significantly) when walking with the device (P = 0.3). (b) The change in the secondary task error rates between the two conditions was not correlated with the change in collisions (r_s = −0.23, P = 0.31). (c) Change in secondary task scores were correlated with the change in PPWS (r_s = −0.45, P = 0.04).

A multifactor linear regression analysis was performed to test the effects of age and visual functions including VF, VA, and CS on mobility. We found VF was consistently the most significant factor in predicting collisions (with the device, F = 20.07, P < 0.001, R² = 0.476; without the device, F = 17.795, P < 0.001, R² = 0.447), but VA and CS were not. Similarly, VF was a significant factor in predicting the device benefit, showing a reduction of collisions when subjects used the device [F_(1,24) = 12.16, P = 0.002, R² = 0.35]. 

Discussion

The major objective of this study was to gather mobility data from a group of patients with a large range of VF loss to evaluate the collision warning device. An obstacle course is a commonly used mobility experiment set up, which can be controlled. To make it suitable for our subject sample, some key study design factors were introduced, such as the high density and variety of obstacles; use of a secondary task; and a course design that afforded multiloop walking. Use of a secondary task and the course complexity were important in making the overall task challenging in order to minimize the potential ceiling effect, especially for those subjects who had relatively large residual VF. Only 3 of 25 visually impaired subjects (all hemianopes) did not record a collision under either condition, indicating that it was difficult for many of the subjects to completely avoid colliding with the obstacles. Multiloop walking provided more chances for potential collisions, thus leading to a higher overall collision count in a short duration. Subjects took an average of 6 minutes to complete 4 loops under each condition. Short duration of the experiment meant that we could conduct the study in a single visit, which helped avoid other confounding issues, such as fatigue, or inconsistency across multiple visits. We also did not see any learning effect (learning of obstacle positions) that might have resulted from continuous multiloop walking in the obstacle course (Fig. 4). 

An encouraging finding from this study is that the device was shown to have a substantial effect in reducing collisions for both TV and HH groups (Fig. 5). For the TV and HH groups, the reduction in median collisions was 43% (from 16–9) and 73% (from 2.75–0.75), respectively. As expected, there was not only a large variation between the groups but also a large variation within the groups. Collisions without the device ranged from 1 to 54 in the TV group and 0 to 23 in the HH group. Obviously, the reason for such a high variability is because multiple human factors contribute to the collision avoidance performance and that VF is just one of them. A regression analysis was carried out to evaluate the overall effect size of the device without splitting the subjects into two groups based on VF. It was found that, on average, collisions while using the device were 37% less than those without using the device (Fig. 6a). Although this reduction appears to be smaller than the reduction based on group medians of collision (43% for TV and 73% for HH), it is a conservative estimate of the benefit a user can expect from the device. It should be noted that this effect was achieved with a minimal amount of training. 

As a small reduction in the PPWS (approaching significance) was observed when subjects walked with the device, a question can be raised: did walking more slowly help in reducing collisions when using the device? We think it is unlikely. If there were such a causal relationship, we would have seen at least a correlation between change in walking speed and change in collisions. However, Figure 7b shows this was not the case. Actually, 29% of subjects (n = 7) walked faster with the device. Based on our observation of the experimental process, we believe that slower walking speed with the device was associated primarily with some subjects' need to take time to scan and maneuver themselves to avoid the obstacles when receiving collision warnings from the device. 

Although the number recall task was designated the secondary task, it simulated situations in which patients performed other tasks (not necessarily secondary) while walking, for example, talking on a phone or looking for directions. In this study, we did not find any evidence suggesting that the subjects intentionally paid less attention to the secondary task when walking with the device in order to improve their mobility performance. First, we examined the overall secondary task performance and found that it was not significantly different between walking with and without the device, and the change in secondary task performance was not correlated with changes in collisions (Fig. 8). Then, we specifically examined the secondary task when collision warnings were given versus when no warning was given. It was found that the error rate with interference of the warning (0.39) was just slightly higher than the error rate with the absence of any interference (0.3). These results suggest that our simple auditory warning cues could work well as intended. Certainly, alternative warning strategies, for example, using tactile cues, are also worthwhile experiments for the future. 

The device was not compared with a control condition involving habitual mobility aids, partially because there were only six active long cane users. Also, the current prototype device does not provide any information about the direction of the predicted collision, which can be important for safe navigation. These limitations will be addressed in future work. We plan to conduct an evaluation study in patients' natural environments, while using their habitual mobility aids (if they have any) and doing their daily activities. Then, difference between walking with and without the device will represent the true benefit of the device for patients' mobility. 

Conclusions

Results of this study indicate that the technology readiness level of our single camera–based collision warning device has reached level 6, “prototype demonstration completed in a relevant environment,” according to the US Department of Defense Technological Readiness Guidance definition for biotechnology.³⁵ The next goal will be to achieve Readiness Level 7: “prototype test in actual operational environments.” 

Acknowledgments

The authors thank Amy Doherty for her help with data collection. 

Supported by US Department of Defense Medical Research and Development Program Grant DM090201. 

Disclosure: S. Pundlik, P; M. Tomasi, None; G. Luo, P