Guttman Scale Analysis of the Distance Vision Scale

Abstract

purpose. The Distance Vision Scale (DVS) is a self-assessment of visual acuity (VA). Like VA testing in which letter reading becomes progressively difficult through the test, DVS questions have a hierarchy of difficulty (Guttman scale). The aims were to determine whether the DVS fulfills Guttman scaling criteria and to test the relationship between DVS score and VA in a cataract population.

methods. Responses of 113 participants to the DVS were subjected to Guttman analysis. Standard criteria of scalability were evaluated that included the coefficient of reproducibility (CR), minimal marginal reproducibility (MMR), and coefficient of scalability (CS). The relationship between total item score and binocular visual acuity was determined.

results. Five participants were excluded because of missing data. Regularity in the banding pattern of the scalogram of the 108 participants was suggestive of a deterministic Guttman scale. Analyses showed that DVS satisfies the criteria for classification as a valid unidimensional and cumulative scale, as CS (0.93), CR (0.99), and MMR (0.85) values fall within the desired range. The statistically significant correlation between the total item score and binocular VA was 0.24.

conclusions. The DVS fits the Guttman scale, supporting the deterministic model underlying the scale. It correlates poorly with VA, suggesting it taps aspects of visual performance and other issues beyond high-contrast VA. The DVS could be used as an outcome measure to evaluate change over time and could be used to set achievable treatment objectives because of its hierarchical properties.

It has been hypothesized that as cataract forms in the eye, vision becomes blurred and performance of a range of activities deteriorates. Although clinically apparent, this hypothesis has not been empirically demonstrated. In particular, little is known about the order in which visual activities are affected. If progression occurs in an ordered, predictable, stepwise manner (i.e., hierarchically), then successive tasks will be more difficult; we would expect distance vision activities that are relatively more difficult, such as recognizing people across the street, to be affected initially and to be followed by easier activities, including those at closer distances such as watching TV. 

The application of patient-reported outcomes (or questionnaires) to assess visual disability is increasing in popularity. ¹ To date most visual disability researchers have applied questionnaires with “summated” scales or Likert scales. ² Likert-scaled questionnaires conveniently provide a total score that is obtained by summing responses to individual items in the questionnaire. This total score is assumed to reflect the individual’s level of a given trait (i.e., visual disability). However, Likert scales are limited by the assumption that all items in a questionnaire have an equal level of difficulty. Similarly limited is the assumption that response categories (assigned ordinal values 1, 2, 3…) are equally spaced along an interval level measure. These assumptions are invariably invalid and are problematic because summing of ordinal scores may assign inappropriate abilities to people. ³  

An alternative to Likert scaling is Guttman or “cumulative” scaling. In a Guttman scale, ^{4 5} responses to items are contingent on the amount of the underlying construct (i.e., visual disability) an individual has. Different items in a questionnaire are not assumed to be of the same inherent difficulty; a subject’s response depends on his or her ability and on the difficulty of the item, the latter of which is referred to as the conjoint structure. Establishing a hierarchy with a Guttman scale helps to legitimize the use of a summed score because the rank ordering of scale items is confirmed. In practice, the Guttman scale is used far less frequently than Likert scales, partly because Guttman scales are comparatively more difficult to construct than Likert scales and have reduced reliability and validity because of their brevity. ^{6 7} Consequently, most researchers prefer to construct Likert scales and use Likert scoring or summary scoring. 

Despite these limitations, Guttman scales are advantageous because a single response can be used to predict responses to all items on the scale; therefore, the Guttman scale is deterministic. Guttman scales are characterized by the “implicational” or “scalable” nature of their items. That is, tasks that can be successfully completed only when component subtasks or preconditions are completed in a certain order are considered implicational or scalable in nature. The final score obtained from Guttman scaling is equivalent to the highest item the participant has agreed with or answered correctly. From this final score, one can surmise all other items that the participant has agreed with or answered correctly. Under these conditions the scale is said to be fully scalable or implicational. The Guttman scale is not statistical because it leaves no room for error estimation. To function in this way, the items in a Guttman scale must be ordered from the most to the least difficult; this is usually determined by sorting the items in descending order according to the proportion of people passing or failing the items. According to this approach, the most able person will respond positively to difficult and easy items whereas the least able person will respond positively to easy items only. The Guttman scale is most commonly used when there is a need to develop short questionnaires with good discriminant ability. The Guttman deterministic principle is apparent in the Distance Vision Scale (DVS). ⁸  

The DVS was developed to assess functional vision loss in the elderly and was one of the earliest patient-reported scales in the ophthalmic field. ⁸ The Guttman principle was applied because the researchers wanted to include questions that could, first, identify persons with functional distance vision loss and, second, discriminate between persons with different degrees of vision loss. Almost four decades have passed since the DVS was developed; at that time, it was clear to the authors that items have a hierarchy of difficulty. Since that time, many other vision-related patient-reported outcomes have been developed, yet in many cases the concept of a hierarchy of item difficulty has been ignored. ^{9 10 11 12 13 14} More recently, the importance of item difficulty has been emphasized with the probabilistic approach of Rasch analysis. ^{15 16 17 18 19 20 21}  

The Rasch model assumes that the observations have an underlying deterministic Guttman scale, but the rating scale is corrupted by a random source of homogeneous variability (i.e., variance of the error distribution is the same for every person/item encounter). Rasch models actually are models of the random variance in the Guttman scale that exploit the errors to estimate intervals between items and between persons. In contrast, deterministic Guttman scales are ordinal because they have insufficient information to estimate intervals. We chose to evaluate the Guttman properties of the DVS to see whether the strict hierarchical order of item difficulty is valid. This is important because previous analyses of responses from patients with low vision to visual disability instruments that included the Activities of Daily Vision Scale, NEI-VFQ-25 plus supplement, expanded Visual Activities Questionnaire, and VF-14 have demonstrated some deviation from the Guttman model. ²² The results of these analyses indicated that the relationship between the latent trait and the questionnaire responses was not deterministic but could be probabilistic. However, only the DVS has been developed with the intent of creating a Guttman scale so it is possible that it is valid in this model. Understanding this hierarchical sequence is important for scales such as the DVS that could be useful for tracking amount of visual loss over time in the elderly caused by cataract. The design of the DVS suggests that it effectively contains only one question, and single-item scales cannot be analyzed using Rasch analysis. Furthermore, the DVS has a deterministic structure that is incompatible with Rasch analysis, which uses a probabilistic model. Because the DVS is intended to be a self-reported measure of visual acuity, we also chose to test its construct validity in comparison with measured visual acuity. 

Methods

Distance Vision Scale

The DVS consists of five questions ordered by increasing difficulty (Table 1) . For each question, the participants are asked to respond positively or negatively. For the first 4 items a positive response is scored as a 0 (yes) and a negative response as 1 (no). On the last item, the scoring system is reversed. The response options of “some” or “none” are also different for this item. This should conform to a Guttman scale because the respondent can only theoretically affirm the next question after having affirmed the previous question. This is analogous to a visual acuity chart whereby the reader should only be able to read the next line after having read the previous line. 

Thus responses to each item in a Guttman scale should be capable of being divided into two (“yes” or “no”) categories, which then differentiates whether a participant “passed” or “failed” an item. An item hierarchy is obtained by sorting items from most to least difficult based on the proportion of participants who passed or failed a question. 

Participants

Participants were patients awaiting cataract extraction at the Flinders Eye Centre, Adelaide, South Australia. Patients were 18 years of age or older, spoke English, and had no severe cognitive impairment. Patients were mailed the DVS for self-completion while they were on the cataract surgery waiting list. One hundred thirteen patients completed the DVS. Mean age of the patients was 74.9 years (SD, 9.1). Patients had coexisting ocular and systemic comorbidities that appeared to be representative of the elderly cataract population in Australia. Participant characteristics are detailed Table 2 . 

Ethics approval was obtained from the Flinders Clinical Research Ethics Committee, and each patient who agreed to participate signed a consent form. The study was conducted in accordance with the tenets of the Declaration of Helsinki. 

Clinical Assessment

Routine clinical assessments were performed before cataract extraction. Visual acuity assessments were performed using computerized testing based on LogMAR principles with screen illumination of 150 cd/m². All assessments were performed binocularly because binocular acuity was considered representative of real-world ability. ^{23 24} Therefore, binocular visual acuity was used in all analyses. 

Statistical Analysis

Data were analyzed according to the following steps: (1) ordering of items defined by the total number of endorsements per item, to form a rank ordering of item difficulty; (2) prediction of endorsement order per subject from their total number of endorsements; (3) calculation of total number of errors from mismatch of actual order and predicted order of endorsements; and (4) calculation of statistical values with standard formulas. 

Guttman scales are unidimensional, which implies that component items measure a single underlying dimension. The unidimensionality of a set of questions is assessed by the extent to which a score of 1 to any question is associated with a score of 1 on all other items ranked as less difficult. Second, Guttman scales are cumulative. The cumulative nature of the DVS can be examined by ordering items according to the total score and ordering participants such that a participant who responds positively to a difficult question will respond affirmatively to another less difficult question. 

Three criteria determine whether DVS items conform to a Guttman scale. ^{4 25 26 27} First, we must assess how often responses fit the ideal pattern. This is indicated by the coefficient of reproducibility (CR), which varies from 0 to 1. This value is calculated as 1 minus the result of the total number of errors divided by the total number of responses. A CR value of more than 0.90 is considered acceptable and suggests that it is a valid cumulative and unidimensional Guttman scale. However, if many participants pass or fail all items of the DVS (i.e., “extreme” items), the CR will be spuriously high. This situation can also occur if all participants pass an item or if an item is too difficult for all participants. 

The second criterion is that the minimal marginal reproducibility (MMR) value, which is represented by the average overall frequency of each response, be close to the CR value. MMR corrects for the chance appearance of a hierarchy. ²⁸ The difference between CR and MMR represents the percentage improvement (PI), which is an indication of the extent to which CR reflects the response patterns rather than the inherent cumulative interrelation of the variables used. The third and the most important criterion is the coefficient of scalability (CS). This value indicates the proportion of responses that can be correctly predicted from the total summed score, thereby allowing for the relative frequencies with which different items are passed. Essentially, the CS tests the degree to which data fit the model. The CS is obtained by dividing PI by the difference between 1 and MMR. The CS varies between 0 and 1. A CS value of ≥0.60 is accepted to confirm the validity of the Guttman scale. ²⁹ Table 3represents the ideal Guttman scalogram ³⁰ for the DVS, whereby participants have been ordered by their total scores across all items and the items have been ordered by total scores across all participants. The diagonal banding in the pattern of scores indicates that all participants agree in the rank ordering of items and all items agree in the rank ordering of participants. 

The relationship between DVS score and visual acuity was tested with the Kendall tau correlation coefficient. Nonparametric testing was used because the DVS score data do not conform to a Gaussian distribution given that there are only six different possible results. Testing was done with statistical software (SPSS, version 15.0; SPSS Inc., Chicago, IL). 

Results

One hundred thirteen participants self-administered the DVS. Five patients were excluded from analyses because of incomplete data. Hence, complete data from 108 participants were available for analysis. 

The responses given to the DVS items are presented in Table 4 . Regularity in the banding pattern of the Guttman scalogram (Fig. 1)suggests that that the responses on the DVS do follow a deterministic Guttman scale (i.e., they are essentially ordered from the most difficult to the least difficult). In Figure 1 , green represents a positive response, red represents a negative response, and yellow is applied to responses that are inconsistent with responses to preceding questions. Two participants had inconsistent responses. For all participants who responded affirmatively to only one item, this included item 1 and no others. For all participants who responded affirmatively to two items, this included only items 1 and 2. As evident in Table 5 , the DVS satisfies the criteria for classification as a valid Guttman scale because CS, CR, and MMR values fall within the desired range. 

Participants with poorer visual acuity reported difficulty with a greater number of items (Fig. 2) . There was a monotonic, though not linear, relationship between visual acuity and DVS total score (r = 0.24; P = 0.002). 

Discussion

The Guttman scale described in the present study provides a model for analysis of assessment of patient-reported distance visual acuity through the DVS. The items in the DVS meet all the requirements of a Guttman scale, and logically all the items relate to blurred vision, typical of vision loss in patients with cataract. 

The high CR (0.99) indicates that both patterns of items are cumulative and that the DVS is reliable. It further suggests that a subject’s scores can be legitimately summed. ³¹ A hierarchy of scale item difficulty can be established from the rank order; therefore, it is appropriate to use a cumulative total score. Because the CR exceeds 0.9, we can predict a subject’s response to an “easier” or more frequently passed item when a subject’s most “difficult” item is also passed. For example, if a participant answered affirmatively to “Can you see well enough to recognize a friend who is an arm’s length away?,” we can also predict responses to other questions such as “Can you see well enough to recognize a friend if you get close to his face?.” 

The coefficient of scalability is further evidence of an ideal Guttman scale. The CS is 0.93, which is significantly above 0.6, the generally accepted minimum level of scalability. 

These criteria for fit to the Guttman scale support the deterministic model underlying the scale. Further evidence for the Guttman pattern is evidenced by the distribution of the responses. A triangular pattern of response that fits the model of a Guttman scale is evident, in which the most difficult item is located at the base and the least difficult item at the apex of the triangle (Fig. 1) . 

Our finding of a statistically significant correlation between binocular visual acuity and total item score supports findings by Hasse and Bryant, ⁸ though the correlation is less than one would expect given that the instrument is basically a patient-reported visual acuity test. However, face recognition is different from high-contrast letter recognition; the patient-reported task is complicated by the nature of the task as it incorporates low-contrast attributes. It has long been known that low-contrast visual testing provides information independent of high-contrast vision testing ³² ; it could be that the DVS does also. The limited correlation is consistent with the evidence base for the relationship between subjective and objective vision measurement, which rarely gives a correlation higher than 0.5. ^{23 24} Another reason for the limited correlation may be the limited distribution of the DVS data, with only six possible results and a notable floor effect; 13% of participants had no difficulty with any distance vision task. Similarly, the range of VA data are restricted; the means for each category range over only two lines of VA, which essentially encompasses significant noise given the test-retest reliability of VA measurement. ³³ Additionally, though the Guttman scale does acknowledge differential item difficulty and a person’s ability is predictable from response to a single item, thereby proving the ordinal nature of the total score, it cannot be inferred that the total score is an interval scale. The nonlinearities arising from unequal intervals will also damage the correlation between total score and VA. For these reasons nonparametric correlation was used, but this does not obviate the limited potential for correlation with a restricted distribution. 

Our findings of acceptable Guttman fit for the DVS are in contrast to findings reported for the ADVS, NEI-VFQ, VAQ, and VF-14 visual function questionnaires. ²² None of these questionnaires functioned as deterministic measurement instruments. ²² These visual disability instruments did show a hierarchy of items within a Rasch model but not a Guttman model. However, the DVS contains items related to distance vision only and therefore is not a true “visual disability instrument.” The wording of the DVS represents a surrogate measure of distance visual acuity. The deterministic Guttman principle underlying the DVS is comparable to the logarithmic progression of the rows on a distance visual acuity chart. ³⁴  

The Guttman scale is not without its limitations. First, the Guttman scale seldom contains more than eight items, as is the case with the DVS. This reduced number of items restricts the ability of the Guttman scale to make finer distinctions among participants. Second, the scalogram analysis may be too restrictive, and only a narrow part of content can be used. That is, it does not allow for enough variation in the construct being measured. Finally a deterministic Guttman scale is ordinal. There is no information that can be used to infer the intervals between items and participants. 

As an assessment measure, the DVS provides information about the visual function of a patient. Because is not always feasible for elderly patients with cataract to visit an eye care professional regularly, items in the DVS can act as measure of their distance vision. For example, if difficulty with recognizing a friend even up close is reported, it can be surmised that the patient will also have difficulty with other activities that involve distance vision. This information would indicate that an immediate eye examination is required. Conversely, if the patient reports no difficulty with distance vision, it may be safe to assume the patient likely has no difficulty with easier tasks and that an eye examination is not urgent. Although one can use a single question, the use of multiple questions could help classify patients into several levels of need or urgency. In this way, responses to items in the DVS could be used to determine the need for intervention. 

Although the Guttman scale properties of the DVS have been confirmed, it must be remembered that the DVS is ordinal; hence, it is not possible to compare effect sizes across patients. Furthermore, if the clinician or researcher wants to look at outcomes and change over time, the loss of interval measurement is a significant kill. One of the items (the last one) uses a different rating scale consisting of categories “some” and “none,” which offers the prospect of more detail than found with “yes” and “no” categories. Such detail is lost with the use of a Guttman scale. Alternatively, Rasch models (probabilistic in nature) that use polytomous rating categories for such terms as “not at all,” “a little,” and “moderate” provide more useful information. For this reason, Rasch models are widely used in vision-related instruments. ^{1 12 15 18 19 35 36} The DVS, though perhaps not as useful, has value because it can be used to set achievable treatment objectives as a result of its hierarchical properties. 

Furthermore, because the items can be ordered by difficulty, it is possible for clinicians to identify visual acuity when the response to the first and easiest item (“Can you see well enough to recognize a friend if you get close to their face?”) is negative. The clinician can use this response as a basis for checking the visual acuity data. This might help avoid recording errors not uncommon in busy clinical settings. After observing this response pattern, clinicians might use this as an opportunity to begin a conversation with the patient to confirm difficulties with distance vision activities. In this way, the DVS might enhance the productiveness of patient-clinician interactions and increase the probability of cataract surgery referrals at the appropriate time. 

In conclusion, the DVS can be used as a patient-reported measure to assist the early identification of visual loss in those with ocular conditions such as cataract. The DVS is cost-effective and convenient and may allow for optimal use of health resources. 

 Supported in part by National Health and Medical Research Council (Canberra, Australia) Centre of Clinical Research Excellence Grant 264620. KP is supported by National Health and Medical Research Council (Canberra, Australia) Career Development Award 426765.

 Submitted for publication December 17, 2008; revised March 3, 2009; accepted June 12, 2009.

 Disclosure: V.K. Gothwal, None; T.A. Wright, None; E.L. Lamoureux, None; K. Pesudovs, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Table 1.

 

Items Included in the DVS

Question No.	Question Description
1	(When wearing glasses) can you see well enough to recognize a friend if you get close to his face?
2	(When wearing glasses) can you see well enough to recognize a friend who is an arm’s length away?
3	(When wearing glasses) can you see well enough to recognize a friend across the room?
4	(When wearing glasses) can you see well enough to recognize a friend across a street?
5	Do you have any problems seeing distant objects?

Table 2.

 

Participant Characteristics for the DVS

Characteristic	Value
Age (y), mean ± SD	74.9 ± 9.1
Sex, n (%)
Male	48 (42.5)
Female	65 (57.5)
Binocular visual acuity
Mean ± SD
logMAR	0.26 ± 0.21
Snellen	6/12+2
Range
logMAR	−0.26 to 0.92
Snellen	6/3−2 to 6/48−1
Awaiting second-eye surgery, n (%)	49 (43.4)
Ocular comorbidity,* n (%)
Present	59 (52.2)
Absent	54 (47.8)
Duration of cataract, y
Median	1
Interquartile range	3
Systemic comorbidity,, † n (%)
Present	102 (90.3)
Absent	11 (9.7)

  n = 113 participants.

*  Such as glaucoma, diabetic retinopathy, age-related macular degeneration.

†  Such as diabetes, hypertension, angina.

Table 3.

 

Ideal Response Pattern for a Perfect Guttman Scale on the DVS

Scale Score	Q5	Q4	Q3	Q2	Q1
0	0	0	0	0	0
1	1	0	0	0	0
2	1	1	0	0	0
3	1	1	1	0	0
4	1	1	1	1	0
5	1	1	1	1	1

 Q, question.

Table 4.

 

Responses Given to DVS

Question No.	Question Description	Response (n = 108)
				Yes	No
1	See well enough to recognize a friend if you get close to his face	107	1
2	See well enough to recognize a friend who is an arm’s length away	107	1
3	See well enough to recognize a friend across a room	91	17
4	See well enough to recognize a friend across a street	63	45
5*	Problems seeing distant objects	94	14

*  For Question 5, the response options are Some or None, respectively.

Figure 1.

 

Guttman scalogram for the five items on the Distance Vision Scale. Participants (rows) are ordered by their total score across all items, and items (columns) are ordered by their total score across all 108 participants. Response categories are color coded (green, affirmed; red, rejected), with the lowest score (person with best functional vision or easiest item to endorse) in green and the highest score (person with worst functional vision or hardest item to endorse) in red. Yellow represents an inconsistent response (i.e., error).

Table 5.

 

Evaluation of Guttman Properties of the DVS

Evaluation Criteria	Value
Coefficient of reproducibility	0.99
Minimal marginal reproducibility	0.85
Percent improvement	0.14
Coefficient of scalability	0.93

Figure 2.

 

Error bars showing 95% confidence interval of mean logMAR visual acuity for each score. A single participant with a score of 4 was deleted.