Abstract
Purpose.:
Multi-attribute utility instruments (MAUIs), which contain a descriptive system, including several health dimensions with associated levels of increasing severity, are used commonly to measure utilities. However, the validity of the descriptive systems rarely is examined using modern psychometric theory. Therefore, we evaluated the psychometric properties of the German version of the Vision and Quality of Life Index (VisQol), a six-item vision-related MAUI.
Methods.:
The German VisQol was self-administered to 340 patients and 280 controls. All subjects underwent a full ophthalmologic examination, including best-corrected visual acuity (VA) testing. The psychometric properties of the VisQoL were assessed using Rasch analysis.
Results.:
The VisQoL's descriptive system did not function in controls. In patients, after collapsing response categories to resolve disordered thresholds and omitting misfitting persons, the measurement properties (i.e., precision, unidimensionality, and targeting) of the German VisQoL were satisfactory. Most person misfit related to unexpected responses to item 4 (“organizing assistance”). Rasch-generated person estimates were not different between age categories, sex, or underlying ocular condition, but decreased significantly with presence of visual impairment in the better eye (LogMAR ≥ 0.5, 1.20 ± 4.62 compared to 3.46 ± 3.52, P < 0.001).
Conclusions.:
The German VisQoL's descriptive system displayed adequate fit to the Rasch model after removal of a large proportion of patients with poor fit statistics. However, the wording of item four should be revised to reduce respondent confusion and measurement “noise.” The scale's descriptive system does not function in a sample of visually unimpaired persons, most likely due to a lack of variance in the measured trait.
As demands on healthcare continue to increase, economic evaluations of disease, impairment and interventions also have become increasingly important.
1 A fundamental aspect of every economic evaluation is the collection or inference of utilities that then are used to calculate Quality Adjusted Life Years (QALYs) and, subsequently, the respective impact of disease, impairment, and interventions.
1
One way of calculating utilities is through multi-attribute utility instruments (MAUIs) in which values are elicited indirectly through patient ratings of their health status from a multifeatured classification system, which captures health-related quality of life (QoL) and allows the comparison across different health states. The only vision-specific MAUI available currently is the 6-item Vision and Quality of Life Index (VisQoL), which was developed and validated specifically for vision-impaired populations.
2 To date, the VisQoL is available only in English, and has not been evaluated using modern psychometric theory, such as Rasch analysis, a form of item response theory. The VisQoL has been shown to perform well using classic test theory and its items were chosen based on item response theory different from the Rasch model.
2,3 However, as the scale provides ordinal measurement of vision-related quality of life (VRQoL) based on its Likert scored descriptive system, and is summarized into a single score following conversion, Rasch analysis is useful not only in assessing the VisQoL's measurement properties, but also in transforming the ordinal score into an interval-level linear estimate.
4
Therefore, in our study, we determined the validity, reliability and measurement characteristics of the German VisQoL's descriptive system using Rasch analysis, and investigated the relationship between the severity of vision impairment, main causes of vision loss, and VisQoL measurement in a German sample of patients with and without vision impairment as well as healthy controls.
Participants underwent a complete ophthalmic examination, including presenting and best-corrected visual acuity (VA), biomicroscopy, intraocular pressure measurements, and funduscopy. Further diagnostic tests (visual field assessment, fluorescein angiography, optical coherence tomography, and electrophysiology) were performed as appropriate in each individual case. VA was tested as best-corrected distance VA using a standard retro-illuminated LogMar chart at 4 meters. Ability of participants to complete the questionnaires was assessed by interviewers before recruitment. The questionnaires were self-administered.
The VisQoL was translated into German and back translated into English, and subsequently a final version created based on this process. The final translation then was pilot tested in focus groups of patients, discussing wording, comprehension, and cultural appropriateness of content. This resulted in several changes of the wording of questions, but no change of the overall content was necessary.
Threshold Ordering.
To determine whether the categories used to rate the VisQoL items are valid, we assessed the response category threshold ordering. First, over- or underutilization of response categories, as well as ability of respondents to discriminate between response categories, was assessed. Disordered thresholds, if evident, were addressed by collapsing categories.
Precision of the Instrument.
Unidimensionality.
Targeting.
Differential Item Functioning (DIF).
The SPSS statistical software (Version 19.0; SPSS Science, Chicago, IL) was used to analyze the data. Descriptive statistical analyses were performed to characterize the participants' sociodemographic, clinical, and VisQoL data. VA was categorized into two categories: Normal vision or mild-to-moderate vision impairment in the better eye (<0.5 logarithm of the minimum angle of resolution [–LogMAR]), and moderate-to-severe vision impairment and blindness (–LogMAR ≥0.5). All tests were considered to be statistically significant at an adjusted level of P < 0.05.
The German VisQoL's descriptive system displayed adequate fit to the Rasch model in our sample of patients with and without vision impairment, after collapsing response categories and removal of a considerable number of misfitting persons. Most person misfit was due to unexpected answers to item 4 (“organizing assistance”) which must be rephrased to reduce the amount of noise created in the overall measurement. Following these revisions, the German VisQoL provided satisfactory measurement of VRQoL in patients in our sample, and is responsive to the presence of at least moderate visual impairment (LogMAR ≥0.5) irrespective of the underlying ocular condition. The scale does not function in visually unimpaired, healthy controls.
The VisQoL demonstrated suboptimal targeting of person ability to item difficulty. Item difficulty is determined by item content, but also by wording and response categories.
13 Intuitively, all VisQoL items are quite “easy” (e.g., “friendships,” “fulfill roles,” “confidence”), which explains why they may not apply to a visually unimpaired sample. Similar targeting problems have been found for a large number of vision-related functioning instruments, with no or very few items targeting the most able participants.
13 This was thought to be due to most instruments having been developed with visually impaired persons, thus containing content most appropriate to them. The targeting of the VisQoL may be improved by the addition of more “difficult” items.
Item 4 (“Do I have difficulty organizing any assistance I may need”?) created considerable misfit among persons, which may be due to its ambiguous phrasing, as it remains unclear whether the emphasis is on organizing, the need for assistance, or the level of assistance needed. Future studies should assess whether improvements to the phrasing of item 4, for example, “How often do I need assistance”? reduces the misfit associated with this item. Removal of this item did not improve the Rasch statistics, but removal of misfitting persons, whose misfit was due mostly to item 4, worsened the overall targeting of the instrument. As the proportion of participants who displayed misfit was comparably large, the German VisQoL must be revised to resolve the issues with item 4.
While our study has shown that the VisQoL's descriptive system is valid and reliable in a patient sample, it did not function optimally in a “control” group, that is, those without ocular disease or vision impairment, due to an expected lack of variance in the sample and the observed lack of more difficult items targeting nonvisually impaired participants. The VisQoL's descriptive system was never developed to be used in normals (normally-sighted controls) and, as with other scales measuring an impairment or its impact, the inclusion of a large group of healthy controls, that is, unimpaired persons with no or little understanding of the impairment/underlying trait measured, is likely to obscure measurement. This may have implications for generating utilities from the VisQoL descriptive system, given that utilities are required in patient and control samples for economic assessments. All this raises an interesting theoretical question about the use of MAUI descriptive systems—which must conform to the Rasch or another IRT model to ensure valid measurement of the underlying trait—to generate utilities. Thus, any utility conversion based on MAUIs likely will need to be validated using time trade-off or standard gamble interviews with patients as well as controls to allow for a valid conversion of VRQoL scores to arrive at societal preferences.
Strengths of our study included the relatively large sample size and the use of Rasch analysis to assess psychometric properties of the VisQoL, and to produce interval-level measurements of vision-related utilities. Such a psychometric evaluation has not been done for the VisQoL to date to our knowledge. Our study setting in daily clinical routine better reflects actual participant context and their preferences than highly standardized clinical trials. To obtain adequate fit to the Rasch model, a large number of misfitting persons required removal so as to reduce measurement noise. Although removal of these persons reduced the sample size and potentially could contribute to sample bias, the misfitting persons did not differ according to age, sex, better eye VA, or underlying ocular conditions compared to the rest of the sample. This supports our conclusion that the VisQoL's descriptive system provides valid measurement of VRQoL in this sample of patients with and without vision impairment. Another limitation of our study was the lack of concurrent use of other MAUIs with vision content, which could have been used for cross validation.
A number of different IRT models other than Rasch analysis, could have been used in our study. However, Rasch analysis is considered a gold standard assessment in the field of low vision as it applies a strict model to which the data must conform, which is most appropriate in a validation study. Therefore, we did not use other IRT models in our study.
In conclusion, the German VisQoL's descriptive system is a valid instrument to assess VRQoL in a sample of eye patients with and without vision impairment after a series of minor revisions guided by Rasch analysis, as well as removal of a number of misfitting persons. Future studies should assess whether rephrasing of item 4 reduces person and item misfit, and whether the addition of more “difficult” items improves scale targeting, and also whether the VisQoL allows for the generating of utilities in a sample with patients and controls based on its descriptive system.