Development and Validation of a Vision-Specific Quality-of-Life Questionnaire for Timor-Leste

Abstract

purpose. To develop and determine the reliability and validity of a vision-specific quality-of-life instrument (TL-VSQOL) designed to assess the impact of distance and near vision impairment in adults living in Timor-Leste.

methods. A vision-specific quality-of-life questionnaire was developed, piloted, and administered to 704 Timorese aged ≥40 years during a population-based eye health rapid assessment. Rasch analysis was performed on the data of 457 participants with presenting near vision worse than N8 (78.5%) and/or distance vision worse than 6/18 (69.8%). Unidimensionality, item fit to the model, response category performance, differential item functioning, and targeting of items to participants were assessed.

results. Initially, the questionnaire lacked fit to the Rasch model. Removal of two items concerning emotional well-being resulted in a fit of the data (overall item–trait interaction: χ² (df) = 81 (51); mean (SD) person and item fit residual values: −0.30 (1.02) and −0.32 (1.46), and good targeting of person ability and item difficulty was evident. Poorer distance and near visual acuities were significantly associated with worse quality-of-life scores (P < 0.001). Person separation reliability was substantial (0.93), indicating that the instrument can discriminate between groups with normal and impaired vision. All 17 items were free of differential item functioning, and there was no evidence of multidimensionality.

conclusions. This 17-item TL-VSQOL has high reliability, construct, and criterion validity and effective targeting. It can effectively assess the impact on quality of life of adult Timorese with distance and near vision impairment. The TL-VSQOL could be adapted for use in other low-resource settings.

A population-based cross-sectional rapid assessment of eye health was conducted during 2005 in Timor-Leste, ^{1 2 3 4} a new southeast Asian country with scant resources and very poor health indicators. Data from this have been used to assist with the planning of eye care services. ⁵  

Although objective quantification of vision impairment and its causes are necessary for service planning, ⁶ so also is subjective quantification to provide insight into the impact of vision impairment on the daily activities, social functioning, and emotional well-being of individuals. Subjective visual function and vision-specific quality-of-life assessments that provide an estimate of individuals’ perceptions of their capabilities can be used to obtain such information and thus to show the negative impact of vision impairment on quality of life. ⁷ However, mostly these instruments have been developed for use in resource-rich countries. ^{7 8 9 10 11} They are unsuitable for low-resource settings such as Timor-Leste, where, for example, the impact of loss of television watching and vehicle driving mean nothing to people for whom such activities are not part of everyday life. 

Visual function and vision-specific quality-of-life questionnaires usually assess the impact of visual impairment as a general concept. However, uncorrected presbyopia has also been shown to have a significant impact, ^{8 12} just as its correction improves quality of life. ¹³ Given the high prevalence of presbyopia (for example, 62% of those older than 40 years in Tanzania ¹⁴ and 93% in south India ¹⁵ ), this is a significant population health issue, the correction of which, with that of refractive error, can provide a large and cost-effective contribution to relieving the burden of vision impairment. ^{12 14 15 16}  

Some visual function questionnaires have been designed expressly for low-resource circumstances. ^{17 18 19 20} However, at the time of the Timor survey, no vision-specific quality-of-life questionnaire was available for use in low-resource settings. Therefore, an instrument was designed that incorporated questions hypothesized to show that poor vision-specific quality of life is related to decreasing distance and near vision in a low-resource country. Rasch analysis ^{11 17 18 19 20 21} was then used to evaluate the questionnaire’s validity, reliability, and measurement characteristics. 

This article describes the development and validation of this vision-specific quality-of-life instrument (TL-VSQOL) designed to assess the impact of distance and near vision impairment in adults living in Timor-Leste. 

Methods

The Impact of Visual Impairment ²² and the visual function assessment for use in Africa, ¹⁷ specifically, were assessed for their suitability to meet the goal of this study. Then, using focus groups and interviews, Timorese eye health workers identified activities of daily living specific to the Timor-Leste context that they considered to be affected by poor distance or near vision. This information was used to develop a 28-item quality-of-life questionnaire, of which 10 items from the van Dijk ¹⁷ and 9 items from the Weih ²² scales were retained, some with slight modifications. 

This questionnaire was piloted with 30 adults, 5 of whom completed it twice, to test temporal stability and interobserver reliability. Five items were removed because they showed floor or ceiling effects or were found not to be applicable to most respondents, and probably a large proportion of the Timorese population. The overall rating of distance and near vision was separated, adding an additional item. 

Participants had difficulty understanding response categories of more than four scale items. Therefore, scale items were modified for use in the final questionnaire. Four items with a four-category response scale (1, very poor; 2, poor; 3, good; and 4, very good) related to overall ratings of quality of life, overall health, and distance and near vision. The overall distance vision question was repeated. A three-category response scale (0, no problem; 1, a little; and 2, a lot) and an option of not relevant/not applicable were used for 13 items concerning difficulty in functioning and for six items related to social/interpersonal relationships and concern about vision. 

A revised 24-item questionnaire was administered during a 2005 eye health survey of Timorese aged ≥40 years. This population-based, cross-sectional survey used multistage cluster random sampling. Of the 1470 people enumerated, 1414 (96.2%) were examined. The detailed methodology has been published. ^{1 2 3} Participants had an eye and vision examination. For distance vision, optotypes 6/18, 6/60, and 3/60 were presented. Type sizes N6 to N24 were used for near vision testing. Participants with presenting distance vision worse than 6/18 in the better eye, as well as every third person with presenting vision of 6/18 and better, were asked to answer the quality-of-life questionnaire. The questionnaire was administered by trained interviewers in the local language—Tetum, or another—depending on the participant’s preference. 

The research adhered to the tenets of the Declaration of Helsinki. This study was approved by the Timor-Leste Ministry of Health. Written consent for the survey to occur was obtained from village chiefs. Participants gave informed consent after explanation of the nature of the study. All participants were advised of the availability and location of permanent eye care services. Transport was arranged for those with treatable causes of vision impairment and blindness who were willing to be referred to these services. 

Rasch Analysis

When participants rate the difficulty of performing activities, they are assessing their functional reserves. This subjective assessment is the difference between a person’s perceived visual ability for functional performance and the visual ability required for the particular task. Rasch analysis can be used to estimate interval measures of perceived visual ability for functional performance from the ordinal ratings of difficulty for each item in a questionnaire. Rasch analysis allows the estimation of each person’s visual ability, the required visual ability and the step measure (the functional reserve threshold) for each activity. ^{11 17 18 19 20} Rasch analysis also provides a measure of the validity and reliability of the measurement of the instrument and has been used to develop and validate vision function and vision-specific quality-of-life questionnaires. ^{23 24 25 26} Once the data fit the Rasch model, estimates of measures on an interval scaling are provided that improve the accuracy of scoring and remove measurement noise. ^{21 27 28 29}  

Data were assessed for fit to the Rasch model ³⁰ with commercial software (RUMM2020; RUMM Laboratory Pty Ltd., Perth, Australia). ³¹ Data not included in this Rasch analysis were the data from every third study participant with 6/18 and better vision who had completed the questionnaire and the data from the four global rating questions (Table 1) . The response category rating scale model was used. 

The logit or log-odds unit is the mathematical unit of Rasch measurement. A positive logit item indicates that the activity requires a higher level of ability than the average of the activity items of the questionnaire. Conversely, a negative logit item suggests that a lower level of ability is required than the average. For ease of interpretation, the response scale scoring was reversed for the Rasch analysis, so that participants with little or no difficulty were given the higher scores. 

An item–trait interaction statistic (reported as a χ² value) was used, and a Bonferroni P > 0.003 (0.05/19 items) indicated no substantial deviation between the data and Rasch model. Fit to the model was evaluated by using person and item fit residual statistics which were transformed residuals approximating a z-score and representing a standardized normal distribution, where an optimal fit to the model would have an expected mean value of 0 and variance of 1. 

Individual item or person statistics with fit residual values > 2.5, or a probability less than the Bonferroni-adjusted α < 0.003, were used to indicate misfit. When this occurred, removal of an activity from the questionnaire was considered. A misfit can sometimes be related to disordered threshold. It occurs when participants have difficulty discriminating between the response category options and means that a category expected to be “harder” than an adjacent category was actually “easier.” It often represents interchangeability of response categories. Category collapsing is often the solution to disordered thresholds. Misfit of data to the Rasch model may also be linked with differential item functioning (DIF), when different groups within the sample (i.e., sex), despite equal levels of the underlying trait, respond differently to an individual item. DIF can sometimes be resolved by item removal or by splitting an item by the person factor. 

Targeting was also assessed. This process determines whether items are particularly suitable for assessing difficulty with daily living in this specific population. Poorly targeted measures display floor or ceiling effects, have an uneven spread of items across the full range of respondent scores and show insufficient items to assess the full range of the sample trait. A person-separation reliability score ranging between 0 and 1 was used to indicate how well the items of the instrument separate the respondents. Larger values indicate a greater ability to distinguish between strata of person ability. 

Finally, the unidimensionality of the scale order was assessed by using principal components analysis of the residuals. Unidimensionality is formally tested in RUMM2020 by allowing the pattern of factor loadings on the first component to determine “subsets” of items (“positive” and “negative” loadings subsets). If more than 5% of person estimates derived from these two subsets of items differ significantly (using independent t-test provided in RUMM2020) from the estimates derived from the full scale, a breach of the assumption of unidimensionality is indicated. ³²  

Results

The vision-specific quality-of-life questionnaire was administered to 704 Timorese aged ≥40 years. Their average age was 61 years (range, 40–92). There were marginally more female participants (52.5%). Only 16.2% were literate. More than three quarters of the sample (359/457; 78.5%) were considered to have presbyopia (binocular presenting near vision worse than N8), and 69.8% (319/457) had better eye presenting distance vision worse than 6/18 (Table 2) . Data from the 457 who had impaired distance and/or near vision were included in the Rasch analysis. 

Psychometric Analysis

The data from the 19 items that were analyzed did not fit the Rasch model, with a significant (Bonferroni-adjusted) χ² item–trait interaction statistic (χ² (df) = 192 (76); P < 0.0001). The mean (±SD) person fit residual value was −0.29 (±1.09), indicating no serious misfit among the respondents in the sample. The mean (±SD) item fit residual value was −0.26 (±2.06). A high SD (>1.5) suggested the presence of misfitting items. The person separation reliability value was 0.93, indicating that the scale could discriminate between groups of respondents with several levels of visual disability. There was no evidence of disordered thresholds, as participants could reliably discriminate between the three rating categories of difficulty of the questionnaire. 

Examination of the individual items showed two with an extreme fit residual value (>2.5) or probability below the corresponding Bonferroni-adjusted value (<0.003). This necessitated their removal. Item reduction was an iterative procedure, with one removed at a time and fit again estimated. The item “have you worried about eyesight getting worse” had the highest fit residual value (4.34) and was therefore deleted first, followed by “do you feel frustrated because of your eyesight” (4.12). The item–trait interaction total statistics improved with each deletion, only reaching a statistically nonsignificant (Bonferroni-adjusted) level after both items had been removed (χ² (df) = 81 (51); P = 0.005). There was no evidence of misfitting items after the removal of these two (Table 3) . The final mean person and item fit residual values were −0.30 (±1.02) and −0.32 (±1.46), respectively. The person separation reliability score remained at 0.93. The data that are reported in the remainder of this article pertain to the remaining 17 items that were further analyzed. 

Person-Item Map

The person-item map (Fig. 1)displays participants’ scores on the Rasch-calibrated scale (left) and the relative difficulty levels of each of the questionnaire items (right). Participants with the highest visual ability and the most visually difficult items are at the top of the diagram. Those with the lowest visual ability and the least difficult items are at the bottom. A positive item logit indicates that the required visual ability for that item was higher than the mean required visual ability of all the items. The visual ability needed for an item was less than the mean if the item logit is negative. A positive person logit indicates that the person’s perceived visual ability was higher than the mean visual ability required for the 17 items. If the person logit is negative, that person’s perceived visual ability is less than the mean. Estimates of participant perceived level of visual ability (in logits) were not significantly different from a normal distribution (Kolmogorov-Smirnov z-test score = 0.91; P = 0.37). There was an even spread of items across the full range of respondent scores. This suggested effective targeting of the questionnaire items. Also, the mean person location value (0.51) indicated that overall the perceived visual ability of participants was higher than the mean required visual ability for the 17 items (which was 0). 

The three most difficult items of the questionnaire were associated with near vision and mobility-related activities: “reading newspaper, bible, or book,” “sorting stones from rice,” and “walking around at night,” with logit scores of 1.66, 1.17, and 1.16, respectively. The three least difficult items were associated with personal hygiene and household tasks: “bathing and personal hygiene,” “cooking, chopping vegetables and pouring water,” and “hand-care” (−2.35, −0.96, and −0.75; respectively). 

DIF and Unidimensionality

All 17 items were free of DIF (for age, sex, marital status, literacy, near vision, and degree of visual impairment), with probability values exceeding the adjusted α value for each of the person factors assessed indicating good construct validity. To assess unidimensionality in RUMM2020, the principal components analysis of the residuals first identified two subsets of items consisting of the highest positive (“attend market,” “social activities,” and “recognize faces”) and negative loading items (“rice cleaning”, “money sorting,” “reading,” and “sewing”). Only 0.51% of estimates were found to be significantly different for these participants. Being less than the recommended 5%, there was no evidence of multidimensionality. 

Criterion Validity

The criterion validity of the Rasch-calibrated questionnaire was tested by assessing its ability to discriminate between participants with different levels of distance and near vision. Decreasing distance vision was significantly associated with greater visual disability (6/18 and better, 1.66 logit; 6/60 and better but worse than 6/18, 0.65 logit; worse than 6/60, −1.24 logit: ANOVA; F _(2,258) = 91; P < 0.001). The same held for decreasing near vision (N8 or better, 0.94 logit; N10–N14, 1.0 logit; N24 or worse, −0.94 logit: ANOVA; F _(2,258) = 48; P < 0.001). 

There was a moderately significant linear relationship between presenting visual acuity in the better eye and person measures (r = −0.50), which accounted for 25.0% of the variance in the visual ability person measure (Spearman’s rho P < 0.001). Near vision explained 10.0% (P < 0.001). Combined, near and distance vision accounted for 33.6% of the variance (P < 0.001). 

Agreement

The question “how would you rate your overall distance eyesight now?” was asked before and after the administration of the instrument. There was a strong positive association between the two responses. This result indicated substantial reliability of participant responses (κ = 0.74; P < 0.001). 

Discussion

Rasch analysis enabled a detailed examination of the operation of the items of the vision-specific quality-of-life questionnaire. The response rating scale model was used to evaluate the ordering of response categories (threshold ordering). The response scale used was valid, and, unlike other instruments that used more than three response category options, ^{11 33 34 35 36 37} no category collapsing was required. 

During analysis of the questionnaire, two items were found to be misfits and were removed. Of interest, these (“worried about eyesight getting worse” and “feeling frustrated because of eyesight”) were the only items relating to emotional well-being. That this was the case may be because there are cultural limitations to naming and discussing emotions in Timor-Leste. Most negative emotions are identified only as “sad.” Also, responses may have been affected by years of oppression and the recent violent upheaval that has, not surprisingly, left some of the elderly population with a generally pessimistic outlook. 

Substantial construct validity of the 17-item TL-VSQOL was supported by the absence of DIF for age, sex, marital status, literacy near vision, and degree of visual impairment. Unidimensionality verified that the instrument measured the underlying trait that it aimed to assess: vision-specific quality of life. Criterion validity confirmed its ability to discriminate between participants with different levels of distance and near vision. A greater perceived visual disability was significantly associated with greater impairment of distance vision. The same was true of near vision. These differences reflected meaningful, discriminating changes in quality of life. 

According to the person-item map, there was outstanding targeting of the response scale, with no apparent floor or ceiling effect as has been found with other instruments. ^{25 33} Only a few participants had no trouble in performing the most difficult items. Others had substantial difficulty with the easiest activities. Also, the good targeting of item difficulty to respondent level of participation suggests that the 17 item TL-VSQOL should be suitable to assess the difficulty of performing daily activities by individuals living in the community, across the spectrum of distance and near vision impairment. This breadth of assessment is a unique feature of the TL-VSQOL. “Reading” has consistently been shown by other instruments to require substantial visual ability. However, the TL-VSQOL revealed that activities such as “sorting stones from rice” and “walking at night” are equally difficult for visually impaired Timorese. 

The purpose of this study was to develop a valid and reliable vision-specific quality-of-life instrument to assist in assessing the impact of distance and near vision impairment on Timorese aged ≥40 years. The application of Rasch analysis confirmed that this has been achieved. The 17-item TL-VSQOL meets the quality assessment criteria that have recently been proposed for appropriate development, refinement, and validation of a vision-specific quality-of-life instrument. ³⁸ It has measurement properties that make it an effective screening tool. Rasch analysis also allows the comparison of these scores with those derived from different questionnaires that all measure visual disability as the same underlying trait. ²⁹ Responsiveness of the instrument will be assessed after blindness-prevention interventions in Timor-Leste. The TL-VSQOL could be adapted for use in other low-resource settings. 

Authors’ Note

Investigators wishing to use the TL-VSQOL questionnaire can apply the validation data to convert raw scores into Rasch person measures, without having to perform a Rasch analysis. This conversion mainly holds for participants with complete data. Raw scores are calculated by first reversing response scores (0, 1, 2) to (2, 1, 0) to give a higher score to the less impaired. The average for the 17 items equates to the TL-VSQOL raw score. The relationship between the raw and Rasch person scores is double asymptotic, because the average raw rating has a floor and a ceiling (0 and 2, respectively). ³⁹ The following equation can be used to convert raw scores to Rasch person measures: TL-VSQOL_{person measure} = 2.594 log [TL-VSQOL_{raw score}/(2-TL-VSQOL_{raw score})] - 0.119. 

 Submitted for publication February 17, 2008; revised May 9, 2008; accepted August 22, 2008.

 Disclosure: R. du Toit, None; A. Palagyi, None; J. Ramke, None; G. Brian, None; E.L. Lamoureux, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Table 1.

 

The 24 Items of the Vision-Specific Quality-of-Life Questionnaire Trialed in Timor-Leste

1. Do you have any difficulty avoiding potholes, stones, or branches when you are walking because of your eyesight?

2. Do you have any difficulty walking around at night because of your eyesight?

3. Do you have any difficulty attending the market because of your eyesight?

4. Do you have any difficulty with bathing or personal hygiene because of your eyesight?

5. Do you have any difficulty cooking, chopping vegetables, or pouring water because of your eyesight?

6. Do you have any difficulty sorting stones from rice because of your eyesight?

7. Do you have any difficulty meeting with family and friends because of your eyesight?

8. Do you have any difficulty participating in social activities like weddings or funerals because of your eyesight?

9. Do you have any difficulty recognizing faces of people across the street?

10. Do you have any difficulty reading things such as a newspaper, Bible, or book?

11. Do you have any difficulty telling the difference between a $1 and a $5 note because of your eyesight?

12. Do you have any difficulty cutting your fingernails because of your eyesight?

13. Do you have any difficulty sewing or making baskets because of your eyesight?

14. Do you have any difficulty tending your garden or repairing your house because of your eyesight?

15. Do you have to rely on other people for help because of your eyesight?

16. Do you feel frustrated because of your eyesight?*

17. In the past month, how often has your eyesight made you go carefully to avoid falling or tripping?

18. In the past month, how often have you worried about your eyesight getting worse?*

19. In the past month, how often has your eyesight stopped you from doing the things you want to do?

Global rating questions:

How would you rate your quality of life?, †

How would you rate your overall health?, †

How would you rate your overall distance vision now? (question repeated), †

How would you rate your overall near vision now?, †

*  Data analysis revealed items 16 and 18 to be misfit, and so these were removed from the final vision-specific quality of life questionnaire for use in Timor-Leste (TL-VSQOL)

†  Global rating questions not included in this analysis.

Table 2.

 

Characteristics of the 457 Participants with Impaired Presenting Distance and/or Near Vision

	n	%
Age (y)
40–49	90	19.7
50–59	94	20.6
60–69	139	30.4
≥70	134	29.3
Sex
Male	217	47.5
Female	240	52.5
Marital status
Married	265	58.0
Not married	192	42.0
Literacy
Literate	74	16.2
Illiterate	383	83.8
Location
Urban	194	42.5
Rural	263	67.5
Presenting distance vision (better eye)
6/18 or better	138	30.2
6/60 or better, but worse than 6/18	211	46.2
Worse than 6/60	108	23.6
Presenting near vision (binocular)
N8 or better	98	21.4
N10–N14	246	53.8
N24 or worse	113	24.7

Table 3.

 

Response Category Frequencies and Fit Indices for the 17-Item TL-VSQOL

Items	Response Category Frequencies*					Location	Standard Error	Fit Residual	χ2	P
		0	1	2
Read	26	30	15	1.66	0.2	0.83	3.6	0.31
Walk at night	166	135	102	1.17	0.09	0.31	2.44	0.49
Clean rice	81	113	51	1.16	0.12	1.73	10.69	0.01
Sew	79	83	69	0.93	0.11	−0.46	3.84	0.28
Recognize faces	119	197	115	0.65	0.09	−1.06	4.12	0.25
Avoid falls	117	193	122	0.52	0.09	−0.88	2.73	0.43
Do things	110	186	135	0.37	0.09	1.47	1.52	0.68
Avoid potholes	99	179	153	0.17	0.09	1.05	5.25	0.15
Social activities	99	94	192	0.01	0.09	−2.42	6	0.11
Attend market	97	89	213	−0.21	0.09	−2.03	6.58	0.09
Meet people	68	157	205	−0.52	0.09	−0.16	7.74	0.05
Garden	61	88	183	−0.54	0.11	−2.33	8.61	0.03
Sort money	67	132	220	−0.56	0.09	−0.97	2.54	0.47
Hand care	73	107	247	−0.75	0.09	−1.98	4.73	0.19
Rely on others	75	109	248	−0.75	0.09	−1.38	4.94	0.18
Cook	24	58	157	−0.96	0.13	2.08	3.91	0.27
Personal hygiene	15	75	337	−2.35	0.12	0.66	1.28	0.73

 All items showed fit residual values < 2.5 and Bonferroni-adjusted P > 0.003.

*  Categories: 0, no problem; 1, a little; 2, a lot.

Figure 1.

 

Person-item location map of the 17 item Rasch-scaled TL-VSQOL: the distribution of calibrated participants’ scores (left) and item locations (right). The mean (SD) person location score was 0.51 (1.91) logits.

The authors thank the Timor-Leste Ministry of Health, District Health Management Bobonaro, and the survey teams of the Dili and Bobonaro districts for their assistance in the study.