Subscale Assessment of the NEI-RQL-42 Questionnaire with Rasch Analysis

Abstract

Purpose.: To explore the psychometric properties of the 13 subscales of the NEI-RQL-42 questionnaire using Rasch analysis.

Methods.: The NEI-RQL-42 is a refractive error-related quality of life (QoL) questionnaire with a complex design; its 13 subscales contain 42 questions, which include 16 different question/response category formats. It was completed by 100 laser refractive surgery subjects (spectacle and contact lens wearers) pre- and postoperatively. Rasch analysis was used to assess the use of response categories, success in measuring a single trait per subscale (unidimensionality), ability to discriminate persons (precision), and targeting of the questions to person QoL.

Results.: Response categories were misused in four subscales (clarity of vision, diurnal fluctuation, symptoms, and appearance), which required repair before further analyses. Six subscales contained items that did not contribute to a single trait measurement (multidimensional). All subscales were found to be inadequate at distinguishing between persons (person separation >2.0), and targeting of the questions to QoL was poor for six subscales.

Conclusions.: The NEI-RQL-42 questionnaire is deficient for all psychometric properties tested. Clinicians or researchers wishing to measure QoL related to refractive error correction should consider other questionnaires that have been rigorously developed and meet standard psychometric properties.

Patient reported outcomes (PROs) have become a fundamental element of clinical practice and research. ^{1 –5} It is therefore vital that PROs are well developed, of high quality, and meet standard psychometric properties for appropriate use. Superficially, PROs resemble questions a clinician may ask during a consultation, but the purpose is distinctly different. Clinical history taking needs to elicit a presenting complaint or clues to a diagnosis; that is, things that need to be fixed. A PRO, on the other hand, needs to include questions that allow measurement to occur on a latent trait of importance to a patient, such as visual disability or vision-related quality of life. 

The National Eye Institute Refractive Error Quality of life instrument (NEI-RQL-42) is a commonly used questionnaire that seeks to measure refractive error-related quality of life (QoL). ^{6 –9} The assessment of refractive error-related QoL is an important outcome measure for the assessment of the many refractive surgery procedures. The NEI-RQL-42 was developed as an NEI-sponsored project to better capture the more subtle effects of functioning associated with refractive error and its correction in patients with visual acuity of 20/30 or better. ⁹ The scored questionnaire consists of 42 items (questions) across 13 subscales. These subscales are conceptual domains that were created by the developers of the NEI-RQL-42, but whether these represent independent latent traits requires testing. The psychometric properties of each subscale needs to be assessed individually. 

Two studies in the literature have investigated the questionnaire with traditional validation methods. ^6,10 However, traditional validation criteria are superficial and do not assess key issues such as whether response categories are used as intended (response category ordering), whether a single subscale score represents a single construct (dimensionality), ability of the instrument to discriminate between people (person separation), and targeting of questions to persons. It is therefore imperative that these issues are assessed comprehensively to gauge whether the subscales measure what they purport to measure. Secondly, like many questionnaires, the NEI-RQL-42 uses Likert scaling with its simple summary scoring method. ¹¹ This method is limited by the assumption that all items are of equal difficulty and that the steps between response options are equal. Likert-type scoring methods are susceptible to bias from missing item responses and to scale distortions from instrument-specific nonlinearities that result from unequal intervals between items and response categories. Thus, scaling may not be additive or linearly related to the latent trait, which interferes with the validity of statistical analyses of the results. ¹² Scored questionnaires should meet the conditions of noninteractive conjoint structure, which means that the manifest variable must exhibit the same ordered relationship as the added latent variables. ¹³ In comparison to summary scoring methods, Rasch models have the advantage in that the measurement is linear with the latent variable. This is true when the raw scores are found to be monotonic with the latent variable the instrument is purporting to measure, which is assessed via Rasch analysis. ¹⁴ All measurements must also conform to a Guttman scale where responses are contingent on the amount of the underlying construct. ¹⁵ Establishing a hierarchy with a Guttman scale helps to legitimize the use of a summed score because the rank ordering of scale items is confirmed. The Rasch model assumes that the observations have an underlying deterministic Guttman scale, but the rating scale is disturbed by a random source of homogenous variability. Rasch models are actually models of the random variance in the Guttman scale that exploit the errors to estimate intervals between items and between persons. Rasch analysis has been used widely for the development of new questionnaires ^{16 –18} and the re-engineering of legacy questionnaires. ^{19 –27}  

The aim of this study was to explore the psychometric properties of the 13 subscales of the NEI-RQL-42 using Rasch analysis to examine the assumption that the underlying construct was refractive error–related QoL. 

Methods

The National Eye Institute Refractive Error Quality of Life Instrument (NEI-RQL-42)

The questionnaire consists of 42 items grouped into 13 subscales with 16 different question/response category formats scattered throughout the questionnaire and subscales (Table 1). There are multiple response options for some subscales, such as the “Near vision” subscale. For item 2 there are six response options, for items 7 and 8 there are five response options, and for item 11 there are four response options. The scoring of the questionnaire comprises two steps. In the first step, the original numeric values are recoded after a set of scoring rules across a 0%–100% range, with higher scores indicating better QoL. Secondly, subscales are scored by averaging the ranks of responses to items within each subscale. The number of items in each subscale varies from one to seven. 

Table 1.

 

The 42 Items in the NEI-RQL-42 Questionnaire with Its 13 Subscales and 16 Different Question/Response Category Formats

Subscales	Items	Response Options
		1	2	3	4	5	6
Clarity of vision	23: At this time, how clear is your vision using the correction you normally use, including glasses, contact lenses, a magnifier, surgery, or nothing at all?	Perfectly clear	Pretty clear	Somewhat clear	Not clear at all	—	—
	Have you experienced any of the following problems in the last 4 weeks? If yes, how bothersome has it been? Please respond for problems in either or both eyes. If yes, how bothersome has it been?	Yes, very	Yes, somewhat	Yes, a little	Yes, not a lot	No	—
	37: Distorted vision?
	39: Blurry vision with your eyesight or the type of vision correction you use?
	40: Trouble seeing?
Expectation	1: If you had perfect vision without glasses, contact lenses, or any other type of vision correction, how different would your life be?	No difference	Small difference for the better	Large difference for the better	I have this already	—	—
	28: If you had perfect vision without glasses, contacts, or any other type of vision correction, how much do you think your life would change?
Near vision	2: How much difficulty do you have doing work or hobbies that require you to see well up close, such as cooking, fixing things around the house, sewing, using hand tools, or working with a computer?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	Never try to do these activities because of my vision	Never try to do these activities for other reasons
	7: How much difficulty do you have reading ordinary print in newspapers?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	Never try to do this because of my vision	—
	8: How much difficulty do you have reading the small print in a telephone book, on a medicine bottle, or on legal forms?
	11: Because of your eyesight, how much difficulty do you have with your daily activities?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	—	—
Far vision	4: How much difficulty do you have judging distances, like walking downstairs or parking a car?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	—	—
	5: How much difficulty do you have seeing things off to the side, like cars coming out of driveways or side streets or people coming out of doorways?
	6: How much difficulty do you have getting used to the dark when you move from a lighted area into a dark place, like walking into a dark movie theatre?
	9: How much difficulty do you have driving at night?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	Never drive at night because of my vision	Never do this for other reasons
	10: How much difficulty do you have driving in difficult conditions, such as in bad weather, during rush hour, on the freeway, or in city traffic?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	Never drive in these conditions because of my vision	Never do this for other reasons
Diurnal fluctuation	3: How much difficulty do you have seeing because of changes in the clarity of your vision over the course of the day?	Don't have changes in the clarity of my vision	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	—
	20: How often are you bothered by changes in the clarity of your vision over the course of the day?	Never	Rarely	Occasionally	Sometimes	All of the time	—
Activity limitations	12: Because of your eyesight, how much difficulty do you have taking part in active sports or other outdoor activities that you enjoy (like hiking, swimming, aerobics, team sports, or jogging)?	No difficulty at all	A little difficulty	Moderate difficulty	A lot of difficulty	Never try to do these activities because of my vision	Never try to do these activities for other reasons
	33: Because of your vision, do you take part less than you would like in active sports or other outdoor activities (like hiking, swimming, aerobics, team sports, or jogging)?	Yes	No	—	—	—	—
	34: Are there any recreational or sports activities that you don't do because of your eyesight or the type of vision correction you have?	Yes, many	Yes, a few	No	—	—	—
	35: Are there daily activities that you would like to do, but don't do because of your vision or the type of vision correction you have?
Glare	17: How often when you are around bright lights at night do you see starbursts or halos that bother you or make it difficult to see?	All of the time	Most of the time	Some of the time	A little of the time	None of the time	—
	Have you experienced any of the following problems in the last 4 weeks? If yes, how bothersome has it been? Please respond for problems in either or both eyes. If yes, how bothersome has it been?	Yes, very	Yes, somewhat	Yes, a little	Yes, not a lot	No	—
	38: Glare?
Symptoms	18: How often do you experience pain or discomfort in and around your eyes (for example, burning, itching, or aching)?	All of the time	Most of the time	Some of the time	A little of the time	None of the time	—
	19: How much does dryness in your eyes bother you?	Don't have dryness	Not at all	Very little	Moderately	Quite a lot	A lot
	24: How much pain or discomfort do you have in and around your eyes (for example, burning, itching, or aching)?	None	Mild	Moderate	Severe	Very severe	—
	25: How often do you have headaches that you think are related to your vision or vision correction?	Never	Rarely	Occasionally	Sometimes	All of the time	—
	Have you experienced any of the following problems in the last 4 weeks? If yes, how bothersome has it been? Please respond for problems in either or both eyes. If yes, how bothersome has it been?	Yes, very	Yes, somewhat	Yes, a little	Yes, not a lot	No	—
	36: Tearing?
	41: Itching in or around your eyes?
	42: Soreness or tiredness in your eyes?
Dependence on correction	13: Do you need to wear glasses or bi-focal lenses or use a magnifier when you are reading something brief, like directions, a menu, or a recipe?	Yes, all of the time	Yes, some of the time	No	—	—	—
	14: Do you need to wear glasses or bi-focal lenses or use a magnifier when you are reading something long, like a book, a magazine article, or the newspaper?
	15: When driving at night, do you need to wear glasses or contacts?	Yes, all of the time	Yes, some of the time	No	Don't drive at night because of vision	Don't drive at night for other reasons	—
	16: At dusk, when it is just starting to get dark, do you need to wear glasses or contacts for driving?	Yes, all of the time	Yes, some of the time	No	Don't drive at dusk because of vision	Don't drive at dusk for other reasons	—
Worry	21: How often do you worry about your eyesight or vision?	Never	Rarely	Occasionally	Sometimes	All of the time	—
	22: How often do you notice or think about your eyesight or vision?
Suboptimal correction	31: How often did you use a type of correction or treatment that was uncomfortable in the last 4 weeks because it made you look better?	All of the time	Most of the time	Some of the time	A little of the time	None of the time	—
	32: How often did you use a type of correction that did not correct your vision as well as another correction would have in the last 4 weeks because it made you look better?
Appearance	27: In terms of your appearance, how satisfied are you with the glasses, contact lenses, magnifier, or other type of correction (including surgery) you have?	Completely satisfied	Very satisfied	Somewhat satisfied	Somewhat dissatisfied	Very dissatisfied	Completely dissatisfied
	29: In terms of your appearance, is the type of vision correction you have now the best you have ever had?	No change	Small change for the better	Large change for the better	I have this already	—	—
	30: In terms of your appearance, is there a type of vision correction that is better than what you have now?	Yes	No	—	—	—	—
Satisfaction with correction	26: How satisfied are you with the glasses, contact lenses, magnifier, or other type of correction (including surgery) you have?	Completely satisfied	Very satisfied	Somewhat satisfied	Somewhat dissatisfied	Very dissatisfied	Completely dissatisfied

Subjects

The NEI-RQL-42 was self-administered by 100 preoperative laser refractive surgery patients who had refractive correction by means of spectacles or contact lenses. The same 100 patients completed the questionnaire six weeks postoperatively (mean age, 36.3 ± 10; age range, 22–58; 59 female, 41 male; 70 underwent laser-assisted subepithelial keratectomy [LASEK] and 30 underwent laser in situ keratomileusis [LASIK]). All patients were 18 years or older, English speaking, and had no severe cognitive impairment. The study was approved by the Cathedral Eye Clinic Ethics Committee, and research was conducted in accordance with the Declaration of Helsinki. 

Rasch Analysis

Rasch analysis was performed on both pre- and postoperative data as one dataset. A separate Rasch analysis was performed for each of the different subscales of the questionnaire. One of the subscales (“Satisfaction with correction”) contains only one item and hence cannot, by definition, be subjected to Rasch analysis. Therefore 12 separate Rasch analyses were performed, one for each subscale. Items that contained the response options “Never do these activities for other reasons,” “Don't drive at night for other reasons,” and “Don't drive at dusk for other reasons” were treated as missing data. The response polarity was matched as per the recoding scores in the NEI-RQL-42 manual. Items 36 to 42 have an “a” and “b” part. Part “a” has a yes/no response, and part “b” is completed only if the response to “a” is yes. We considered this as a five-response (four-threshold) scale analogous to the NEI-RQL-42 manual. 

The Rasch model is based on a probabilistic relationship between item difficulty and person ability. This difference is known as the functional reserve or ability, ²⁸ which expresses the probability of any person being successful on any item. A polytomous (multiple response options) Andrich rating scale model (ln[p(x)/p(x − 1)]) was used in this study via commercial software (Winsteps v. 3.70.0.2; Winsteps, Chicago, IL). For the individual, rating scale responses are mutually exclusive. So, if we are given the information that person n responded to item i with rating category x or x − 1, then the posterior probability of that person responding with any other rating category is zero. ²⁹ Rasch analysis was used to assess each subscale for response category performance, item fit statistics, precision, and targeting. 

Category Threshold Order

The performance of response categories in terms of being used in the order intended was evaluated by observing if the category calibration increased in an orderly fashion in the category probability curves (a graphical display of the likelihood of each category being selected over the range of the scale). The threshold is the midpoint between response categories and indicates the point where the likelihood of choosing either response category is the same. Items in the questionnaire have between two and six choices, which translates to one to five thresholds, respectively. Each threshold has a location on the logit scale, and each item has a mean location. Hence, one would expect that with decreasing ability, the probability of selecting each statement would increase in an ordered fashion from least to most difficult. There are a number of reasons for disordered thresholds, such as an underused category, unclear descriptive wording, or if the number of categories exceed the number of levels the participants can distinguish. ^30,31 Disordered thresholds are a symptom of noise due to confusion over categories. Therefore, in cases of disordered thresholds, response categories were collapsed (combined together) until thresholds were ordered to reduce noise. This was done before further analysis. Category collapsing was performed according to the following criteria. In the presence of disordered thresholds, category probability curves were inspected to identify the category (categories) candidate for combining with adjacent categories. Categories could be considered for combining only when category labels made it logical to do so (e.g., a positive and a negative descriptor could not be combined). Categories that showed the greatest overlap of curves are usually the most appropriate to be combined; however, where an underutilized category could be combined with two adjacent categories, both were performed in turn with the impact on the fit to the model assessed. The combination that provided the largest improvement in fit was accepted. Ideally, categories should be evenly spaced and advance step calibrations by at least 1.4 logits. ³¹  

Dimensionality

A fundamental element of measurement is unidimensionality; a score produced by a measure should represent a single concept. Item fit statistics are used in the assessment of unidimensionality. Fit statistics (infit and outfit) focus on two aspects, which can be reported as a mean square (MNSQ) with an expected value of 1. The infit statistic is less sensitive to distortion from outliers and is thus considered the more informative fit statistic. The infit MNSQ is the ratio of the observed variance of the residuals to the variance expected by the model. This ratio is expected to be distributed as χ² divided by its degrees of freedom. The MNSQ residual statistic is normalized to the average expected variance such that a residual of <0.70 indicates at least 30% less variance than expected, which can occur with large numbers of extreme values (ceiling or floor) or if the sample is very noisy with a lot of misfits. Residuals greater than 1.30 indicate at least 30% more variance than expected, suggesting items may be measuring something different from the overall scale. Therefore, an acceptable infit and outfit are within the range of 0.70–1.30. ³²  

Unidimensionality was further assessed by principal components analysis of the residuals. The contrasts in the residuals report unexplained variance by the principal component. The first contrast is the second principal component, whereas the second contrast is the third principal component, and the third contrast is the fourth principal component. This study used the criterion that the contrast with an eigenvalue >2.00 units (i.e., strength of at least two items) is suggestive of a second construct being measured, thus indicating a multidimensional instrument. Similarly, items loading on first contrast by a minimum of 0.4 are identified as contrasting items and tap different constructs. 

Precision

Precision is a fundamental aspect of measurement; a measure needs to be able to discriminate along its scale. Rasch-derived person separation statistics indicate the overall precision of the instrument. It is used to illustrate how many groups or strata of person ability an instrument can discriminate and equates to the ratio of the true variance in the estimated measures to the observed variance. ³³ The greater the value of person separation, the greater the precision enabling distinction between levels of function. ³⁴ A minimal acceptable cutoff value for the person separation ratio was set at 2.0 for this study. ³⁵  

Targeting

Targeting refers to the extent to which the difficulty of the items matches the abilities of the persons in the sample. This can be assessed visually, by observing the person-item map. This map also shows item hierarchy and enables the identification of redundant items or large gaps between items. Inadequate targeting occurs when items are clustered at certain points along the logit scale, leaving large gaps, and when many persons have a higher or lower ability than the most or least difficult item threshold. Targeting may be measured by comparison of the person and item mean values. A perfect targeting instrument would have a difference of zero, whereas a difference of more than one logit indicates significant mistargeting. ³⁶ Precision and targeting are related concepts. The precision indices are global, and targeting refers to local precision. An excellent method for illustrating targeting is the Fisher information function. The Fisher information function was produced to overlie the person-item map. This is a related index to the distribution of item measures but incorporates the standard error of the estimate for each item and in a perfectly targeted instrument would indicate the highest level of information at the mean of person measures. ³⁷  

Differential Item Functioning

An important characteristic of a good instrument is that all items function similarly for persons at the same level of ability. Differential item functioning (DIF) occurs when subgroups of people with comparable levels of response respond differently to an item, which implies a response to some characteristic other than item difficulty. Surgery-specific DIF was assessed to determine whether items function differently in a surgery group compared with a preoperative group (spectacles and contact lenses). Notable DIF was classified as >1.0. 

Results

The first step in the analysis of each subscale was to check the category response thresholds for each of the 16 different question/response category formats. Five of these formats were disordered, which required reordering by category collapsing before further analysis. Four subscales (“Clarity of vision,” “Diurnal fluctuation,” “Symptoms,” and “Appearance”) were affected by the disordered question/response category formats, which indicates that response options were not used as intended, as illustrated by disordered structure calibrations (Table 2). In the “Clarity of vision” subscale, the disordered formats affected items 37, 39, and 40. For the “Diurnal fluctuation” subscale, one item (item 20) was affected, and for the “Symptoms” subscale, six items (items 18, 19, 25, 36, 41, and 42) were affected. Finally the “Appearance” subscale had one item (item 27) affected by the disordered formats (Fig. 1). Figure 2 demonstrates an appropriately ordered format with category calibration increasing in an orderly fashion (Fig. 2). 

Table 2.

 

Overall Performance of the NEI-RQL-42 Questionnaire Using Rasch Analysis

	Subscales
	Clarity of Vision	Expectations	Near Vision	Far Vision	Diurnal Fluctuation	Activity Limitations	Glare	Symptoms	Dependence on Correction	Worry	Suboptimal Correction	Appearance	Satisfaction with Correction
Number of items	4	2	4	5	2	4	2	7	4	2	2	3	1
Number of question/response category formats	2	1	3	2	2	3	2	5	2	1	1	3	1
Number of item formats needing reordering	1	None	None	None	1	None	None	3	None	None	None	1	N/A
Items with thresholds needing reordering	37, 39, 40	All ordered	All ordered	All ordered	20	All ordered	All ordered	18, 19, 25, 36, 41, 42	All ordered	All ordered	All ordered	27	N/A
Number of misfitting items	2	None	None	None	2	2	1	2	4	None	None	None	N/A
Misfitting items	23, 40	N/A	N/A	N/A	3, 20	33, 34	38	19, 42, 41	13, 14, 15, 16	N/A	N/A	N/A	N/A
Person separation	0.00	0.00	0.71	0.00	0.00	0.00	0.00	0.00	0.41	0.00	0.00	0.59	N/A
Mean item location	2.1	−0.50	0.71	2.34	1.09	0.72	1.93	1.65	−0.33	2.21	0.93	0.52	N/A

 N/A, not applicable.

Figure 1.

 

Response category probability curves for item 27. The x-axis represents the difference between item and person calibration, and the y-axis represents the probability of the category being chosen.

Figure 2.

 

An example of ordered response category probability curves. The x-axis represents the difference between item and person calibration, and the y-axis represents the probability of the category being chosen. Each response option is appropriately used, and category calibration increases in an ordered fashion.

After the collapse of the response options, unidimensionality was assessed. Six of the 12 subscales contained misfitting items, classified as outside the range of 0.70–1.30 (Table 2). This resulted in 13 misfitting items across the questionnaire. These items, by definition, are confounding the measurement of the underlying measurement trait. Precision was assessed by person separation. None of the 12 subscales had adequate person separation. The largest person separation subscale was “Far vision” with a value of 0.71; however, this is significantly less than the minimum acceptable value of 2.0 (Table 2). Targeting of items to person QoL was poor in six subscales, defined as a difference between the mean item difficulty and mean person ability of one logit (Table 2). Figure 3 illustrates the person-item map for the subscale “Far vision.” The mean difference between items and persons for the “Far vision” subscale is 2.34 logits. Good targeting would be indicated with differences <1.00 logit. The overlying curve is the Fisher information function for the items. 

Figure 3.

 

Person-item map for the subscale “far vision.” Subjects appear on the left (each # in the person column represents five persons, and each dot is one to four persons), and items appear on the right. Person and item means are denoted by M with the large distance (2.34 logits) between the means illustrating poor targeting. The overlying black curve is the Fisher information function for the subscale, which shows the information provided by the items does not inform the status of the people.

DIF was assessed by comparing pre- and postoperative responses. Notable DIF was found in eight subscales (Table 3). The largest DIF was for item 13 in the “Dependence on correction” subscale. No notable DIF was found for subscales “Near vision,” “Far vision,” “Glare,” “Suboptimal correction,” and “Appearance.” 

Table 3.

 

Notable DIF (>1.0) Comparing Pre- and Postoperative Responses

Subscale	Differential Item Functioning	Mantel-Haenszel Test P Values	Item
Clarity of vision	1.97	<0.0001	23
	−1.26	<0.0001	40
Expectation	2.67	<0.0001	1
	−3.34	<0.0001	28
Diurnal fluctuation	1.39	0.0001	3
	−1.94	0.0001	20
Activity limitations	2.88	<0.0001	12
	−1.13	0.0054	33
	−2.18	<0.0001	34
Symptoms	−1.86	<0.0001	19
	1.52	0.910	24
	2.08	<0.0001	25
Dependence on correction	5.53	0.0016	13
	4.49	0.0016	14
	−4.50	0.0016	15
	−2.82	0.0016	16
Worry	1.48	<0.0001	21
	−1.41	<0.0001	22
Appearance	1.39	0.0247	29

Subscale precision is usually poor due to a small number of items. In an attempt to investigate if the NEI-RQL-42 could measure latent variables that have not been predetermined based on the developers' domain classification, a principle components analysis (PCA) was performed including all 42 items in one analysis. The PCA is performed on the residuals with the first principal component (indicating the measure itself) explaining 32.8 eigenvalues of the 74.8 eigenvalues of variance in the observations (43.9%). This analysis revealed that the unexplained variance explained by the first contrast was 7.2 eigenvalue units (9.6%); by the second contrast 2.6 (3.4%) and by the third 2.1 (2.9%). The remaining contrasts were <2.0 eigenvalue units. This demonstrates that the core measure of the NEI-RQL-42 explains a small part of the overall variance observed in this questionnaire, which also contains at least one, and possibly three, additional dimensions representing latent traits different from that represented by the core measure. Seven items loaded positively (>0.4) onto the first contrast (items 15, 16, 28, 30, 18, 26, and 1), three items (items 13, 14, and 21) onto the second contrast, and three items onto the third contrast (items 40, 37, and 39). These three contrasts provided the framework to investigate if they functioned as three separate domains. The first domain consisting of seven items revealed an inadequate person separation of 1.34 and six misfitting items outside the range 0.70–1.30. Only item one was found to fit the model (infit 0.85, outfit 0.73). The second domain containing three items had a person separation of 0.00, and all items were found to misfit. The third domain also with three items had a person separation of 0.00 with two items misfitting. Only item 39 fitted with an infit of 0.90 and an outfit of 0.83. Since DIF could be the source of dimensionality, an analysis after the removal of the 19 items with DIF found that the unexplained variance explained by the first contrast was 4.3 eigenvalue units (9.9%), and the second contrast was 2.2 (5.2%). This suggests two possible dimensions, with three items loading positively (>0.4) onto the first contrast (items 15, 16, and 18) and two items onto the second contrast (items 13 and 14). However, analyses of each dimension revealed inadequate person separation of 0.23 and 0.00, respectively. 

Discussion

The purpose of this article was to investigate the psychometric properties of the 13 subscales of the NEI-RQL-42 using Rasch analysis. The questionnaire seeks to measure QoL related to refractive error correction. The sample of subjects used in this study was ideal to investigate these properties as it covered three forms of refractive error correction; spectacles, contact lenses, and laser refractive surgery. The results of this analysis indicate that all the subscales are inadequate on a number of counts. 

The first issue is the response category ordering. Five of the 16 question/response category formats were disordered, which affected four subscales. One of the main reasons this can occur is due to the use of too many response options. Some of the questions had six response options, but it has been shown that respondents typically tend to only use four or five categories. ^30,31 Figure 1 shows the response categories for item 27 (“In terms of your appearance, how satisfied are you with the glasses, contact lenses, magnifier, or other type of correction [including surgery] you have?”). There were six response options for the item: “Completely satisfied,” “Very satisfied,” “Somewhat satisfied,” “Somewhat dissatisfied,” “Very dissatisfied,” and “Completely dissatisfied.” It can be seen that the second and third category responses were underutilized. At no point were these categories likely to be chosen over categories one or four. Second, multidimensionality was found in terms of misfitting items in six subscales, which equates to half the number of subscales assessed. This is a problem because it means that these items are contributing noise rather than information about the latent trait under measurement. It is imperative that any questionnaire scale or subscale seeking to make a measurement of a latent trait is unidimensional. The same is true for clinical measurement; for example, a device that measures central corneal thickness (CCT) and intraocular pressure (IOP) might be very useful if it produces two unidimensional scores (one for CCT and one for IOP). However, if it produces a single multidimensional score, e.g., 600, this might have something to do with glaucoma, but it has no clinical utility. The six multidimensional subscales of the NEI-RQL-42 have the same problem. 

Person separation was found to be inadequate for all 12 subscales. All subscales had person separation values significantly less than the minimum accepted value of 2.0. ^{33 –35} This indicates that the questionnaire subscales could not adequately discriminate between the individuals in the sample population. The cause of inadequate person separation is too few items (the SE of the person separation is inversely proportional to the square root of the number of items) and items that poorly target the people under measurement. Targeting in general was poor, with six subscales indicating significant mismatch between the mean item difficulty and mean person ability estimates. By way of example, Figure 3 illustrates the person-item map for the subscale “Far vision.” This subscale has five items with a difference of 2.34 logits between the mean item difficulty and mean person ability. In particular, item 6 (“How much difficulty do you have getting used to the dark when you move from a lighted area into a dark place, like walking into a dark movie theater?”) is not as important as item 10 (“How much difficulty do you have driving in difficult conditions, such as in bad weather, during rush hour, on the freeway, or in city traffic?”). Poorly targeted items such as item 6 should be removed from instruments. These items are not a problem for patients with refractive error; there are too many people with not enough problems for these items to be part of the measurement of refractive error–related QoL. These underutilized items may be more useful in disease groups such as cataract, keratoconus, or macular degeneration who may have, for example, activity limitations. However, for an instrument that aims to measure QoL related to refractive error correction, these items are off scale and not relevant and so simply increase respondent burden without contributing to measurement. Similar questionnaires, namely, the NEI-Visual Functioning Questionnaire and the Refractive Status Vision Profile, have similar problems in terms of a lack of person separation, item misfit, and poor targeting. ^25,26,38 Large DIF was also found when preoperative questionnaire responses were compared to postoperative questionnaire responses, particularly for items that relate to spectacle correction. The subscale “Dependence on correction” may by definition become irrelevant after surgery. This illustrates that the relative impact of some items on the latent trait from pre- to postsurgery; this is undesirable, as it confounds measurement. The analysis found surgery-dependent DIF for 19 items, indicating serious problems with the use of this instrument when interpreting surgery outcomes. Attempts were made to repair the NEI-RQL-42 questionnaire first by using PCA results from a complete analysis of the 42 items together. This revealed three potential domains for which their psychometric properties were assessed. Unfortunately none of the newly identified domains functioned adequately. 

Question format was highly variable in the NEI-RQL-42 with a range of question syntax. There were 16 different question/response category formats, which were scattered throughout the questionnaire and subscales. Having a range of question formats and category responses within one subscale not only increases respondent burden, it can cause detrimental psychometric effects, such as poorly functioning category responses. For example, the “Symptoms” subscale has four different question styles and three different types of category response options. Item 18 asks about the frequency of discomfort, whereas item 19 asks how much does dryness bother you and item 24 asks about the severity of pain or discomfort. This was associated with six disordered items from a total of seven items in the subscale. Other instruments in the literature such as the Quality of Vision (QoV) questionnaire demonstrate that items assessing frequency, severity, and bothersome nature should be assessed individually in their own subscales. ¹⁶  

Other issues with the questionnaire can be seen in a study by the developers, Hayes and colleagues. ⁶ They reported a significant (>50%) ceiling effect (all maximum responses) for two subscales. The “Activity limitations” subscale had a 53.5% ceiling effect, and the “Suboptimal correction” subscale had a 81.5% ceiling effect. Similarly, there are 42 items across 13 subscales, meaning that each subscale has few items; this leads to poor measurement precision. 

In conclusion, the NEI-RQL-42 questionnaire has a serious lack of precision in measures based on the subscales, outcome-dependent DIF, and scale distortions from misfitting items. Clinicians and researchers wishing to measure QoL related to refractive error correction should consider other questionnaires that have been rigorously developed and meet the standard psychometric properties. 

Footnotes

 Disclosure: C. McAlinden, None; E. Skiadaresi, None; J. Moore, None; K. Pesudovs, None