Rapid and Reliable Assessment of the Contrast Sensitivity Function on an iPad

Abstract

Purpose.: Letter acuity, the predominant clinical assessment of vision, is relatively insensitive to slow vision loss caused by eye disease. While the contrast sensitivity function (CSF) has demonstrated the potential to monitor the slow progress of blinding eye diseases, current tests of CSF lack the reliability or ease-of-use to capture changes in vision timely. To improve the current state of home testing for vision, we have developed and validated a computerized adaptive test on a commercial tablet device (iPad) that provides an efficient and easy-to-use assessment of the CSF.

Methods.: We evaluated the reliability, accuracy, and flexibility of tablet-based CSF assessment. Repeated tablet-based assessments of the spatial CSF, obtained from four normally-sighted observers, which each took 3 to 5 minutes, were compared to measures obtained on CRT-based laboratory equipment; additional tablet-based measures were obtained from six subjects under three different luminance conditions.

Results.: A Bland-Altman analysis demonstrated that tablet-based assessment was reliable for estimating sensitivities at specific spatial frequencies (coefficient of repeatability 0.14–0.40 log units). The CRT- and tablet-based results demonstrated excellent agreement with absolute mean sensitivity differences <0.05 log units. The tablet-based test also reliably identified changes in contrast sensitivity due to different luminance conditions.

Conclusions.: We demonstrate that CSF assessment on a mobile device is indistinguishable from that obtained with specialized laboratory equipment. We also demonstrate better reliability than tests used currently for clinical trials of ophthalmic therapies, drugs, and devices.

Introduction

The worldwide demographic trends of aging and obesity will increase the global incidence of blinding eye diseases. The leading causes of blindness in the developed world are age-related macular degeneration (20–25 million patients ¹ ), glaucoma (80 million patients by 2020 ² ), and diabetic retinopathy (439 million diabetes patients projected for 2030 ³ ). The costs of failing to treat blindness are severe. In addition to depression and other psychosocial effects on individuals, economic burdens include increased health care costs and lost productivity. ⁴  

To develop new treatments for blinding eye diseases, there is an urgent need for sensitive methods that detect the presence and progression of gradual vision loss. Early detection methods are needed, because treatments are most effective when they are offered in the early stages of the disease, so that significant vision loss can be avoided or delayed. The efficacy of new therapies must be compared against existing clinical interventions, so tests that can detect small changes in progression are essential to demonstrate clinically significant treatment outcomes. The current clinical vision standard is letter acuity, the measurement of the smallest letter size that can be resolved at high contrast (e.g., black letters on white background). Though useful to indicate refractive error, acuity can be insensitive to the neuropathology presented by age-related macular degeneration, ⁵ glaucoma, ⁶ and diabetic retinopathy. ⁷ Acuity may not predict visual quality of life better than questionnaires. ⁸ Because of poor repeatability, acuity scores must be averaged over several measurements, which increases testing times. ⁹ Therefore, there is a pressing need for improving functional or structural assessments to detect disease and monitor its progression. ^10,11  

A more comprehensive measure of visual function is contrast sensitivity, which describes visual sensitivity at different contrasts and spatial scales. Contrast sensitivity deficits significantly affect overall quality of life (for a review see the report of Owsley ¹² ). Relative to acuity, contrast sensitivity is correlated better with target identification in natural images, ¹³ driving, walking, and the ability to see faces. ¹⁴ Poor contrast sensitivity is associated with an increased risk of falls, ¹⁵ which are predicted to cost almost 50 billion dollars annually in the United States alone by 2020. ¹⁶ Contrast sensitivity is impaired in a variety of ophthalmic and neurologic conditions, including age-related macular degeneration, ¹⁷ amblyopia, ¹⁸ dry eye, ¹⁹ glare, ²⁰ glaucoma, ²¹ myopia, ²² ocular hypertension, ²³ cerebral lesions, ²⁴ and multiple sclerosis. ²⁵ Importantly, contrast sensitivity may be impaired in visual neuropathologies that do not affect acuity. ^26,27  

In this project, we present proof of principle for a mobile Health (mHealth) application that provides a rapid, sensitive, and portable assessment of the contrast sensitivity function (CSF), with potential to improve ophthalmic clinical practice and clinical trials. 

Visual function tests may complement current eHealth applications for eye disease, which have focused primarily on acuity ²⁸ and structural assessments of ocular health. ³ While acuity is too insensitive, functional tests for contrast sensitivity ¹² can be more sensitive to the presence of disease states when acuity and fundus images appear normal. ^29,30 Despite these demonstrations, these tests are not used widely, as traditionally they require long testing times and need specialized laboratory equipment or medical devices. 

The Mobile Health solutions may provide the avenue to address these needs, with devices now available that enable the development and deployment of vision tests that provide high-quality assessment outside the laboratory or clinic. Home surveillance of vision, monitoring the potential progression of impaired vision without the need for specialized equipment, could greatly improve clinical practice and clinical trials. ^30,31  

In this study, we described a novel implementation of a vision test on a commodity tablet device (iPad; Apple, Inc., Cupertino, CA). Because digital liquid crystal display (LCD) and thin-film transistor (TFT) displays typically have a grayscale resolution that is insufficient to capture the limits of human visual sensitivity, we also validated its measurements against those obtained with dedicated, specialized laboratory equipment. Our results enabled the wide deployment of rapid and reliable home monitoring of vision, which will improve clinical practice and clinical trials. 

Methods

Efficient Contrast Sensitivity Testing

Despite the recognized importance of contrast sensitivity, ³² it is not used commonly in clinical practice. ¹² Current contrast sensitivity tests use paper cards and charts ³³ that limit the range and resolution of stimulus sampling, and likewise limit test flexibility and precision. Repeated measurements, for example, of both eyes individually, under different luminance conditions, or over time to monitor disease progression, are complicated by the deterministic sequence of stimuli that patients may memorize. Computerized contrast sensitivity tests mitigate some of these issues, and provide more flexibility and precision than paper media. However, computer- and paper-based tests typically are based on the simplifying assumption that CSFs can be described using a single CSF template (a log parabola). Therefore, given assessment of the CSF at one point (the peak), and assessment of acuity as standard care, the clinician could obtain an approximation of the full CSF. Such one-point CS assessment was the goal of Pelli-Robson contrast sensitivity, which measures the peak (or rather an approximation of the peak, because a priori the true peak frequency is unknown) by changing the contrast of a single optotype size. 

While this approximation already provides some information beyond acuity assessment alone, it has been demonstrated in a large population study that not two, but four parameters are needed to describe CSFs appropriately ³⁴ ; at least four parameters also were used to model the ModelFest data set. ³⁵ Measurement of the whole function, however, is valuable because different pathologies can affect different regions of the CSF. For two examples, consider the effect of reduced illumination (e.g., peak sensitivity is reduced and shifted to lower frequencies), or the horizontal shifting of CSFs with increasing eccentricity (peak sensitivity remains the same, but shifts to lower frequencies). 

With traditional psychophysical methods that estimate contrast sensitivities (thresholds) individually at different spatial frequencies, full CSF assessment was very time-consuming (several hundred trials, >30 minutes), which has been a major obstacle to its widespread clinical adoption. 

Recently, Lesmes et al. ³⁶ presented the quick CSF method, a Bayesian adaptive procedure that accelerates estimation of a CSF defined by four parameters that describe peak frequency, peak sensitivity, bandwidth, and low-frequency truncation. The expected information gain ^37,38 for possible grating stimuli (defined by contrast and spatial frequency parameters) is computed before each trial. By choosing a stimulus with high expected gain, the qCSF uses the previous trial history to avoid efficiently uninformative regions of the stimulus space. 

Implementation

The quick CSF method ³⁶ was implemented in Objective-C for use on Apple iOS devices (Apple, Inc.). Wherever possible, results of complex computations were stored in look-up tables to minimize runtime. 

On the iPad 2 (used in Experiment 1), the quick CSF method was set up to evaluate 16 different spatial frequencies, log-spaced from 0.42 to 13.7 cycles per degree (cpd) of visual angle (at a viewing distance of 60 cm). At the highest spatial frequency, one cycle corresponded to four pixels on the screen. In Experiment 2, an iPad 3 with higher pixel density was used, which allowed testing at 19 different frequencies from 0.29 to 18.5 cpd. 

In Experiment 1, stimuli were horizontally-oriented Gabor patches with a support (±4 SD) that corresponded to six cycles, so that half-height was reached at 1.84 cycles. Michelson contrast ranged from 0.2% to 100% in 48 log-spaced steps. In a two-alternative forced-choice paradigm, the task was to report whether a briefly presented Gabor target was presented 4.6° to the left or right of fixation. The CSF represented thresholds defined at the 75% correct performance level. 

In Experiment 2, stimuli were Gabor patches (with four cycles per image and 1.22 cycles at half-height; thus target size, but not bandwidth, covaried with spatial frequency) that were presented at fixation for 1000 ms, and the observer's task was to report their orientation (45° clockwise or counterclockwise); contrast ranged from 0.2% to 100% in 96 log-spaced steps. Overall, therefore, this test implementation could display more than 1800 unique stimuli (3600 including orientation). By comparison, current charts typically display less than 50 stimuli; for example, the MARS chart uses 48 different contrast levels and the Pelli-Robson chart uses only 15 levels. ³⁹  

Two other features of the quick CSF method are the density of a finite grid of possible parameter combinations and the number of Monte Carlo samples from the Bayesian prior used for the pretrial calculation of expected information gain. We here used 333,944 grid nodes and 1000 samples. 

Figure 1 shows an example of trial placement and the resulting CSF estimates over the time course of the test. Each data point shows the spatial frequency and contrast of a test stimulus; correct and incorrect trials are shown in blue and red, respectively. After five trials only (top left), little information has been gained yet, so that confidence intervals (shaded area) of the CSF estimate (solid line) are large. After 30 trials (top right), the test begins to converge; the best estimate after 60 trials (bottom left) already is a very good approximation of the final estimate after 120 trials (bottom right). The filled symbol in each panel corresponds to the most recent trial; note that these trials always are placed in informative regions of the stimulus space (near current best threshold estimate). 

Figure 1

 

Example visualization of the adaptive procedure over the time course of the test after 5 (top left), 30 (top right), 60 (bottom left), and 120 (bottom right) trials. Each data point shows the spatial frequency and contrast of a test stimulus; correct and incorrect trials are shown in blue and red, respectively. The solid line denotes current best estimate of the CSF and the shaded area corresponds to the 66% confidence interval.

Display Fidelity

Typical image processing pipelines represent images with 8 bits precision per color channel, so that 256 unique grayscale tones can be realized. The smallest luminance increment at mean luminance then is greater than 1% (for typical display gamma values ≥ 1.8). Peak human contrast sensitivity, however, can be less than 0.5% ⁴⁰ and, thus, cannot be tested exhaustively with consumer-grade hardware. Even worse, typical TFT and LCD screens provide only 6 bits of grayscale resolution. In the laboratory, CSF measurements, therefore, are taken with cathode ray tubes (CRTs) and hand-built video attenuator boxes that use the bandwidth of the color channels to provide a grayscale resolution of approximately 14 bits. ⁴¹  

One advantage of implementing our test on the iPad is that hardware characteristics are relatively stable across devices. We used a professional photometer to confirm that the iPad displays use the full 8 bits of the input signal and, thus, can show 256 unique grayscale tones. We implemented OpenGL ES ⁴² shaders to increase grayscale resolution further up to 11.5 bits by dithering, that is, adding spatiotemporal luminance patterns outside the limits of human visual perception. ⁴³  

Empirical Validation

Two experiments were performed to validate empirically the reliability, accuracy, and flexibility of the tablet-based qCSF test. In Experiment 1, we evaluated CSF assessment obtained from tablet-based tests and compared it to that obtained for the same observers from laboratory CRT-based tests. In Experiment 2, we evaluated how the tablet-based test tracked CSF changes due to different luminance conditions. 

For comparing the tablet-based with the CRT-based test, the CSF of four observers (aged 28–36 years; three male, one female; all normal or corrected-to-normal vision) was first evaluated by repeated assessment on an iPad 2. All observers gave informed consent, and this and the following experiment were performed in accordance with the tenets of the Declaration of Helsinki. 

Each observer completed four test runs of 120 trials each; during the experiment, observers sat in a darkened room and held the device at 60 cm distance. A fixation marker was displayed in the center of the screen. Each trial was preceded by white markers that indicated the scale (spatial frequency) of the upcoming stimulus and framed the potential locations, which were centered approximately 4.6° from fixation. Stimuli then were presented in one of these locations for 250 ms, where contrast was linearly ramped up and down for the first and last 60 ms, respectively. The subject's response then was registered by tapping the tablet screen in one of the hemifields. 

In a second session, the CSF was assessed with specialized laboratory equipment and a MATLAB-based (MathWorks, Natick, MA) qCSF implementation. The same four observers completed four tests of 120 trials each. The setup comprised an iMac (Apple, Inc.) workstation computer connected to a carefully calibrated, analog CRT display (LaCie 22-inch electronblue IV running at 1024 by 768 resolution and a refresh rate of 120 Hz; LaCie, Paris, France), using a video attenuator ⁴¹ that provides more than 14 bits of grayscale resolution. Despite its analog nature, the CRT has a finite spatial resolution that is imposed by its aperture grill; for the tablet device, resolution (1024 × 768 pixels, 60 Hz refresh rate) is limited by the fixed array of pixel-driving transistors. To keep the retinal angle covered by a single pixel constant across tablet and CRT setups, the test distance for the CRT-based test was doubled to 120 cm. Mean screen luminance also was fixed at 67 cd/m² on both setups; luminance calibration was performed with a PR-655 SpectraScan (Photo Research, Inc., Chatsworth, CA) photometer. For the CRT-based test, observers' localization responses were registered by keyboard response. 

In Experiment 2, the flexibility and sensitivity of the tablet-based test were evaluated, by assessing the CSF under varying luminance conditions, which can be modified easily by programmatically setting the output level of the tablet's backlight. Six subjects (aged 21–46 years; five male, one female; all normal or corrected-to-normal vision) participated in the experiment, and each took the test at 3, 16, and 85 cd/m² (in randomized order). 

The CSF metrics we calculated were the area under the log CSF (AULCSF), ⁴⁴ which provides a broad measure of contrast sensitivity across all frequencies, and sensitivity estimates at individual spatial frequencies, (1, 1.5, 3, 6, 12, and 18.5 cpd), which are specified standards for clinical testing of ophthalmic devices. ⁴⁵  

Results

Reliability and Accuracy of CSF Estimates

Figure 2 presents the CSF estimates obtained in Experiment 1. Each column presents data obtained from one subject, and data from different test-cutoffs (60 or 120 trials) are presented in different rows. For the sensitivity estimates at individual spatial frequencies (1, 1.5, 3, 6, 12, and 18.5 cpd), error bars denote the variability (±1 SD) of sensitivity estimates across four runs. The mean CSFs and individual sensitivities obtained across four runs are presented for the tablet-based test (red) and the CRT-based test (blue). 

Figure 2

 

Results from Experiment 1. CSFs were obtained from four observers (one per column) on a consumer tablet and with specialized CRT-based laboratory equipment. Data analysis was performed for the first 60 (top) and the full 120 trials (bottom row). Solid lines show the average CSF estimate on tablet (red) and with CRT (blue). Error bars on sensitivities estimated at 1, 1.5, 3, 6, 12, and 18.5 cpd denote the standard deviation of contrast sensitivity estimates across four runs. After 120 trials, there was excellent agreement between CSFs obtained with the different systems.

The maximal sensitivities, which were consistently less than 1% and observed at low spatial frequencies, were consistent with previous studies of visual sensitivity. ⁴⁶ Following Bland-Altman analysis, ⁴⁷ we characterized the reliability of tablet-based assessment by the coefficient of repeatability (COR), which describes the 95% confidence limits (2.77 × SD) for repeated measurements. 

For estimates of sensitivities at individual spatial frequencies (see Table), the COR values for mid-range spatial frequencies ranged from 0.21 to 0.39 for 60 trials and 0.14 to 0.37 for 120 trials. The COR value for the highest spatial frequency is artificially low (0.10), due to its nonvisibility to some observers. These values compare favorably to those reported for current contrast sensitivity charts, ³⁹ which range from 0.26 to 0.54 for Vistech and 0.22 to 0.60 for FACT. 

Table

 

Reliability and Accuracy Measures for Individual Spatial Frequencies After 60 and 120 Trials in Experiment 1

SF, cpd	60 Trials		120 Trials
	Reliability COR	Accuracy Δ Sensitivity	Reliability COR	Accuracy Δ Sensitivity
1	0.229	−0.025	0.185	−0.026
1.5	0.205	−0.000	0.183	−0.012
3	0.320	0.038	0.179	0.004
6	0.364	0.094	0.142	0.012
12	0.389	0.141	0.372	0.047
18.5	0.151	0.014	0.100	0.011

 The COR values describe the variability of measurements across repeated runs of the tablet-based test; the values for our test (0.10–0.37) compare very favorably to those reported for current contrast sensitivity charts ³⁹ (e.g., 0.26–0.54 for Vistech, and 0.22–0.60 for FACT). Mean differences in sensitivity between tablet- and CRT-based tests are almost zero, showing that tablet and CRT measurements are indistinguishable.

As Figure 2 further shows, the CSFs obtained from the CRT-based test (blue) demonstrated excellent agreement with those obtained from the tablet (red). Following Bland-Altman analysis, ⁴⁷ we characterized method agreement between CSF metrics obtained with tablet- and CRT-based tests. For individual spatial frequencies, sensitivities obtained with 120 trials (see Table) exhibited almost no differences between tablet- and CRT-based assessments (−0.026 < all mean differences < 0.047 after 120 trials). These differences are considerably smaller than the contrast sensitivity changes that are assumed to be clinically meaningful (0.30 log units ⁴⁵ ). We, therefore, concluded that the tablet- and CRT-based tests provided indistinguishable assessments of contrast sensitivity. 

Effect of Luminance Changes on CSF Estimates

Figure 3 shows average CSF estimates for six subjects under varying luminance levels. Consistent with previous reports, ⁴⁸ peak sensitivity and peak frequency increased with luminance. Mean peak sensitivity for the highest luminance level (85 cd/m²) was 115, that is, threshold was less than 0.9%. Different regions of the CSF were affected differently, with only moderate sensitivity changes (<50%) for low and medium spatial frequencies, and substantial changes for high spatial frequencies (e.g., more than 6-fold sensitivity increase between 3 and 85 cd/m² for 12 cpd). The sensitivity of the test is reflected in Figure 4, which shows area under the log CSF scores for individual subjects. For all subjects, this summary statistic changed meaningfully with changes in luminance. The error bars denote ±1 SD confidence intervals. 

Figure 3

 

Average CSF estimates for six subjects under varying luminance levels. Peak sensitivities are reduced and peak frequencies are shifted towards the left under reduced illumination. High spatial frequencies are particularly affected by varying luminance.

Figure 4

 

Effect of luminance level on area under the log CSF for six subjects. Error bars denote ±1 SD.

These results compared favorably to sensitivity changes measured under different luminance conditions with paper-based contrast charts as reported in the literature ⁴⁹ ; chart scores on average changed by 5.1% (Vistech) or 3.7% (GECKO) for a 31-fold increase in luminance, whereas the qCSF AULCSF scores increased by 19.7% for a 28-fold increase in luminance (10.6% for the increase from 3–16 cd/m²; 8.2% from 16–85 cm/m²). For both charts, the average increase was substantially less than the mean step size of contrast increments. 

To summarize the results of both experiments, the tablet-based assessment provided contrast sensitivity estimates that were more reliable than current tests for normal vision, were sensitive enough to characterize vision changes as a function of changes in luminance, and were indistinguishable from those obtained with specialized laboratory equipment. 

Discussion

The ubiquity of smartphones and internet-enabled tablet devices has led to a rapid growth in development of and research on mobile health applications. For example, aggregated cell phone location data have been used to study the spread of diseases, such as malaria. ⁵⁰ At the individual level, telemonitoring of patients' behavior and health data has shown to have potential to improve care for patients with diabetes. ⁵¹  

Modern smartphones also include a range of sensors, for example, accelerometers that can be used for automatic gait analysis ⁵² or GPS receivers that enable alerts to be sent if dementia patients leave their homes. ⁵³ Additionally, external sensors also can be attached to smartphones, such as low-cost add-ons that may turn a smartphone into a clinical device, for example, for light microscopy ⁵⁴ or cataract assessment. ⁵⁵  

One area in which mHealth solutions still are mostly lacking is sensory assessment, for which development is complicated by the need for the carefully calibrated and precise actuators that have not been available typically on consumer-grade hardware. However, the increasing display resolution of smartphones enables the development of applications (apps) that test visual acuity ²⁸ and estimate refractive error, ⁵⁶ and it is likely that future smartphone development will further enable tests of increasing technical sophistication. 

In this report, we investigated the feasibility of an mHealth application of a computationally intensive and technically demanding contrast sensitivity test. We demonstrated that the same rapid and precise visual assessments obtained with specialized laboratory equipment can be obtained with a mobile tablet device, even at very low contrast levels. This is notable, considering current commercially available sensory test platforms based on flat-panel display hardware, such as the Nike SPARQ Sensory Station (Nike, Inc., Beaverton, OR), ⁵⁷ cannot present stimuli nor measure thresholds lower than 0.8% contrast on its TFT display. Our tablet-based test does not suffer from this ceiling effect, as we here measured thresholds below 0.8% with CRT- and tablet-based tests in Experiment 1. In Experiment 2 (tablet only), peak sensitivity for three subjects was 0.6%, and we confirmed further using a SpectraScan PR-655 photometer (Photo Research, Inc.) that contrast levels of down to 0.2% could be presented reliably. 

Reliable and sensitive assessment of the CSF is useful clinically because various progressive neuropathologies may affect the CSF earlier than acuity, the current clinical standard of visual function assessment. ^5–7,26,27 Until recently, however, characterization of the CSF not only required expensive computer setups, but also was too time-consuming to be practical in clinical settings. The recently developed quick CSF method ³⁶ reduced testing time to a few minutes and has been used successfully to assess, for example, vision in amblyopia. ⁵⁸ In this study, we have implemented this method on a commodity tablet device. We showed that quick CSF estimates on a tablet were stable across several test runs, and that our implementation was sensitive enough to describe accurately and rapidly contrast sensitivity dynamics due to luminance changes. 

Using the quick CSF on a mobile device, we have made it possible to test visual function precisely and rapidly inside and outside the clinic and laboratory. This, in turn, has the potential to provide more individualized, low-cost, and better health care. In this study, we validated our approach with subjects with normal vision, and further studies are required that will determine the ultimate reliability of our approach in clinical practice. However, we already have used iPad-based testing successfully in medically underserved areas in India, in collaboration with Project Prakash. ⁵⁹ The portability and speed of our iPad-based implementation of contrast sensitivity testing allowed us to quantify the development of vision after cataract removal in congenitally blind children. ⁶⁰ In industrialized countries with strong medical infrastructure, clinical practice can be streamlined by testing patients bedside or in waiting rooms. Greater cost savings may be realized in clinical trials for drug development, which currently cost an average of 27 million dollars per year per single new drug. ⁶¹ Increasing the precision of vision tests, and enabling frequent testing without the need for laboratory visits, as the present project does, can reduce significantly the necessary number of clinical trial subjects and, likewise, reduce drug development costs. ⁶² Thus, home monitoring can improve the clinical practice and clinical trials for eye disease and prevent blindness. 

Acknowledgments

Supported by National Institutes of Health Grants EY018664, EY019281 (MD, LAL, and PB), and EY017491 (ZLL). 

Disclosure: M. Dorr, Adaptive Sensory Technology (I, E, S), P; L.A. Lesmes, Adaptive Sensory Technology (I, E, S), P; Z.-L. Lu, P; Adaptive Sensory Technology (I, S), P; P.J. Bex, Adaptive Sensory Technology (I, S), P