The data were evaluated for fit to the Rasch model using the RUMM2020 (Rasch unidimensional measurement models; RUMM Laboratory, Perth, WA, Australia) software,
37 with the goal of assessing how well the observed data fit the expectations of the measurement model. The partial-credit approach
38 (which allows each item to have its own threshold parameters) was used because the likelihood-ratio test was statistically significant (
P < 0.001) indicating that the rating scale model (which requires equivalent thresholds across all items) was not appropriate. The likelihood-ratio test was still statistically significant (
P < 0.001) when applied to the two subsets of items (19 and 13 items), suggesting that the partial-credit approached was more suitable. Three overall fit statistics were considered. Two were fit residuals statistics, which represent the residuals between the expected estimate and actual values for each person-item, summed over all items for each person and over all persons for each item. The residuals are transformed to approximate a
z-score and represent a standardized normal distribution where perfect fit to the model would have a mean of approximately 0 and an SD of 1. An item–trait interaction score reported as a χ
2, which reflects the property of invariance across the trait, was also provided. A statistically nonsignificant probability value (
P > 0.05) indicates no substantial deviation from the model. Individual item or person statistics where fit residuals values >2.5 or probability values below the Bonferroni adjusted α value (i.e., 0.05/32 = 0.001) are also used to indicate misfitting to the model. In addition to these overall fit statistics, the RUMM2020 program provides an indication of person-separation reliability using the person-separation index (PSI range, 0–1) which indicates how well the items of the instrument separate, or spread out, the subjects in the sample. A person-separation reliability value from RUMM of 0.7 is the equivalent of a G value of 1.5, representing the ability to distinguish two distinct strata of person ability.
39 40 A value of 0.9 is equivalent to a G value of 3, with the ability to distinguish four strata of person ability.