Validation of the National Eye Institute Visual Function Questionnaire-25 (NEI VFQ-25) in Age-Related Macular Degeneration

This letter is in reference to the article by Orr et al. ¹ in the May 2011 issue. The title implies that the article provides validation of the NEI VFQ-25 for use in neovascular age-related macular degeneration (AMD); however, the evidence presented provides, at best, only very weak support for the validity of the NEI VFQ-25. 

The authors report only three aspects of validity: convergence (testing the association between NEI VFQ-25 scores and visual performance measures), known-groups (comparing mean questionnaire scores between subgroups stratified by visual performance results), and internal consistency (demonstrating that a series of items measure a common theme). They demonstrate no (R ² = 0.00) to moderate (R ² = 0.31) relationships between NEI VFQ-25 scores and visual acuity (VA), contrast sensitivity, and reading speed. They also demonstrate that people with VA better than 20/32 have better NEI VFQ-25 scores than those with VA worse than 20/100. Finally, they tested internal consistency with Cronbach's α, which is the weakest of the suite of methods for testing internal consistency (versus factor analysis and principal components analysis of Rasch residuals). ² Although Cronbach's α is easy to calculate, it is confounded by questionnaire length and may fail to identify the multidimensionality made evident by more sophisticated techniques. ³ Indeed, several studies have shown that the NEI VFQ-25 in its original format is a multidimensional scale, and investigators have recommended reorganizing the items into two scales and abandoning most of the subscales. ^{4 –6}  

Taken together, the results of Orr et al. provide some of the absolute minimum requirements for a questionnaire, ⁷ but are a long way from suggesting “Validation of the NEI VFQ-25 in AMD.” The study did not show whether the NEI VFQ-25 is valid in the functioning of the response categories, measurement precision, unidimensionality, targeting of the scale to the study population, and differential item functioning (whether measurement properties are stable across populations). ² Indeed, the NEI VFQ-25 has been shown to lack validity for a number of these properties. ^{4 –6} This is important because these psychometric properties have more serious implications for measurement validity than those presented by Orr et al. Therefore, sending the message that the NEI VFQ-25 has been validated for use in AMD by this study appears inappropriate and carries the danger of encouraging use of the instrument despite published evidence questioning its validity. ^{4 –6}  

We also suggest that any paper that describes the validation of an instrument be more comprehensive, ideally including modern psychometric methods such as item-response theory models or Rasch analysis, given the deep insight that these provide into the psychometric properties of an instrument, ² and comparison of the findings against established frameworks of questionnaire quality. ^8,9 The other important advantage of Rasch analysis is that it enables interval level estimates from ordinal questionnaire responses. ¹⁰ The reduction of measurement noise provided by interval scaling can halve the sample size needed for outcome measurement, with substantial potential cost savings for major clinical trials such as those from which the data in Orr et al. were taken. ¹¹