Abstract
Purpose :
The Andrich rating scale model is widely used to estimate person and item measures from rating scale data collected from visual function questionnaires such as the Activity Inventory (AI), which has 510 items and 5 rating categories. A well-known problem is that the Andrich model often estimates disordered rating category thresholds. To ensure ordered thresholds are estimated, users must manually merge neighboring rating categories (rescore data). We show that rescoring data changes the scale on which the Andrich model estimates parameters, making comparisons across studies difficult or impossible. A new method is needed that estimates measures on a scale invariant to the number of rating categories.
Methods :
We have developed a new method for estimating measures from rating scale data called the Method of Successive Dichotomizations (MSD). MSD applies the dichotomous Rasch model as many times as there are thresholds and uses the averages of the person and item measures from each dichotomization as the final estimated person and item measures. Thresholds can be estimated by looking at the differences between the mean person measure from each dichotomization to the average of all person measures from all dichotomizations. MSD is well grounded in theory, as we show that the method can be derived from two assumptions: 1) thresholds are ordered on every trial, and 2) the distribution of threshold error terms is identical for every threshold.
Results :
We show through simulations that the correlation between true and MSD-estimated person measures, item measures and thresholds is close to 1. We then apply MSD to the AI and compare the results to parameter estimates of the Andrich model. We show that the advantage of MSD over the Andrich model is that while estimated person and item measures are similar for both models, MSD-estimated thresholds are always ordered, no manual rescoring is required and the scale on which parameters are estimated remains invariant to the number of rating categories.
Conclusions :
The Method of Successive Dichotomizations improves on the Andrich model by estimating ordered thresholds on a scale that remains invariant to the number of rating categories. This enables parameter estimates across different studies to be directly compared to each other.
This is an abstract that was submitted for the 2018 ARVO Annual Meeting, held in Honolulu, Hawaii, April 29 - May 3, 2018.