Rasch Analysis for QoL Questionnaires

Vianya-Estopa et al. ¹ have criticized the Amblyopia and Strabismus Questionnaire (A&SQ), in contrast with the very favorable outcome of our recent factor analysis. ² We would like to comment on the measurement model used in their appraisal. 

In clinimetrics, test administration is seen as a probability experiment in which the essential attributes of interest, here quality of life, as well as incidental effects such as errors due to the measurement experiment itself, are thought to influence the item responses. ³ The purpose of a measurement model is to describe this experiment and provide estimates of the latent variable of interest and the variance of its measurement errors. The authors analyzed A&SQ data with Andrich's rating scale model ⁴ which assumes that (1) the subject is making a series of consecutive choices between neighboring categories, ⁵ (2) the items measure in the same scale units, ⁶ and (3) a subject's responses uniquely indicate the attribute of interest and do not depend on each other. ⁴ None of these assumptions, however, seem to adequately describe the measurement experiment or have much relevance for the meaningfulness of statements based on A&SQ scores. Lack of fit of this model should, however, not be interpreted as evidence for multidimensionality but as a lack of fit to all the assumptions of the rating scale model. Therefore, the second dimension found by the authors may well have been the result of a shortcoming in the description of the measurement experiment. 

In our analysis ² we took a pragmatic approach and used principal component analysis (PCA) to assess whether a substantial amount of the response variance can be explained by a small number of factors (preferably one), instead of detecting deviations from a restrictive one-dimensional model. PCA allows items to be measured on different interval scales, ⁷ and controls for correlated residuals by introducing additional factors that explain negligible amounts of response variance. We searched for effect size rather than adherence to a (virtually nonattainable) normative standard such as Rasch homogeneity. For this reason PCA is generally considered the model of choice in assessment of measurement instruments for attitudes and personality like the Big Five. ^8,9  

If there is reason to believe that our methods have been too crude, the usual method is to compare the fit of a one-dimensional discrete item response model against a well-fitting multidimensional model. ¹⁰ Given the number of parameters to be estimated, multidimensional models require a large sample of subjects. We predict, however, that even when a two-dimensional model is preferred over a one-dimensional one, the correlation between the factors will be too high in most populations to be of much practical interest. If uncorrelated, the second dimension may be an effect of the measurement experiment and picks up only small factors due to artifacts such as priming, which will generally not have a systematic influence on the A&SQ score. Summarizing, we do not think the analysis of Vianya et al. ¹ undermines the practical usefulness of the A&SQ or threatens the meaningfulness of statements based on A&SQ scores.