Outcome Preferences in Patients With Noninfectious Uveitis: Results of a Best–Worst Scaling Study

Abstract

Purpose: To estimate patient preferences regarding potential adverse outcomes of local versus systemic corticosteroid therapies for noninfectious uveitis by using a best–worst scaling (BWS) approach.

Methods: Local and systemic therapies are alternatives for noninfectious uveitis that have different potential adverse outcomes. Patients participating in the Multicenter Uveitis Steroid Treatment Trial Follow-up Study (MUST FS) and additional patients with a history of noninfectious uveitis treated at two academic medical centers (Johns Hopkins University and University of Pennsylvania) were surveyed about their preferences regarding six adverse outcomes deemed important to patients. Using “case 1” BWS, patients were asked to repeatedly select the most and least worrying from a list of outcomes (in the survey three outcomes per task).

Results: Eighty-two patients in the MUST FS and 100 patients treated at the academic medical centers completed the survey. According to BWS, patients were more likely to select vision not meeting the requirement for driving (individual BWS score: median = 3, interquartile range, 0–5), development of glaucoma (2, 1–4), and needing eye surgery (1, 0–3) as the most worrying outcomes as compared to needing medicine for high blood pressure/cholesterol (−2, −4 to 0), development of cataracts (−2, −3 to −1), or infection (sinusitis) (−3, −5 to 0). Larger BWS scores indicated the outcomes were more worrying to patients.

Conclusions: Patients with noninfectious uveitis considered impaired vision, development of glaucoma, and need for eye surgery worrying adverse outcomes, which suggests that it is especially desirable to avoid these outcomes if possible. (ClinicalTrials.gov number, NCT00132691.)

The Multicenter Uveitis Steroid Treatment (MUST) Trial has compared fluocinolone acetonide implant versus systemic corticosteroids, supplemented with immunosuppressive drugs, as alternative treatments for intermediate uveitis, posterior uveitis, and panuveitis and shown that both treatments, on average, are effective in preserving vision for these patients.^1,2 A corticosteroid implant is placed into a patient's eye to deliver corticosteroid locally for the management of uveitis.³ It is designed to reduce the need for systemic use of corticosteroids, which may increase the risk for high blood pressure, diabetes mellitus, and infections.⁴ However, several studies have indicated that use of the implants leads to a higher risk of developing cataracts and glaucoma,^2,3,5 both well-known ocular adverse effects of corticosteroid therapy. While the MUST Trial has demonstrated that risks of systemic adverse effects do not differ greatly between groups and has found that benefits are similar,² in clinical practice, the problem of trading off local versus systemic adverse effects and benefits of alternative routes to deliver corticosteroids in eye disease remains of interest in specific cases. 

The scenario of comparing qualitatively different outcomes of treatments is challenging in the clinical setting. Understanding how patients may interpret the trade-offs between the adverse outcomes would help to inform benefit–harm assessments and treatment decisions of this nature.^6,7 The aim of a quantitative benefit–harm assessment is to compare treatments through a comprehensive assessment of their effects on multiple benefits and harms,⁷ which requires learning about patient preferences (i.e., the importance or weights that patients place on outcomes⁸) to estimate the benefit–harm balance.^7,9,10 Besides benefit–harm assessment, preference data also play a key role in other health care research such as economic evaluation and decision analysis.^11,12 They often are incorporated into such studies to properly reflect patients' needs and to generate findings that support patient-centered care.^13,14 

The aim of our study was to learn how patients with noninfectious intermediate uveitis, posterior uveitis, and panuveitis value adverse treatment outcomes of corticosteroid implant and systemic corticosteroids by conducting a preference-elicitation survey. In addition, we explored whether patients' characteristics and their experiences with treatment outcomes were associated with their preferences. 

Methods

Study Design

We conducted a cross-sectional survey using best–worst scaling (BWS), a technique introduced by Finn and Louviere¹⁵ in marketing research, to elicit outcome preferences. We chose a “case 1” BWS design, in which a set of three or more “objects” are shown to patients, who are asked to choose the best and worst of the multiple objects.^16–18 By studying the probability of subjects choosing one object over others, we can elicit the preferences that patients have for that object. 

Patient Preferences Questionnaire

We were interested in how patients as a group make trade-offs between adverse treatment outcomes that are commonly seen in practice. Through a literature review and consultation with investigators of the MUST Trial and Follow-up Study, we selected adverse outcomes that are associated with corticosteroid implant and systemic corticosteroid therapy for noninfectious uveitis.^2,19,20 These outcomes included major ocular adverse events such as loss of visual acuity, development of cataracts and glaucoma, and needing eye surgery, as well as major systemic adverse events such as prescription-requiring hypertension/hyperlipidemia and systemic infections (e.g., sinusitis). We then developed descriptions for the six outcomes included in the survey (see Table 1) with help from clinical experts and methodologists. 

Table 1

 

Objects for the Best–Worst Scaling Tasks

 

The questionnaire started by asking patients to read the descriptions of the six outcomes and complete a visual analogue scale (VAS) task for each outcome. After patients read the description of each outcome, they assigned a score from 0 (least serious outcome) to 100 (most serious outcome) on a VAS to indicate how serious they perceive the outcome to be. These VAS tasks were designed to help introduce patients to the outcomes. 

Then the patients were asked to complete 10 BWS tasks. In each BWS task, three outcomes (“objects” in BWS) were shown to patients and they were asked to choose the one that would worry them most and the one that would worry them least. To cover all 6 outcomes equally in the 10 BWS tasks, we used a balanced incomplete block design¹⁶ to generate 10 sets of outcomes (3 outcomes per set). The balanced incomplete block design assured each outcome appeared as an option equally often (five times) in the questionnaire and also coappeared with each other equally often. We did pilot-testing of the questionnaire with clinicians and students to ensure that the instructions were clear. The questionnaire that we administered can be found in the Supplementary Material. 

Study Patients

This study consisted of two populations of patients with noninfectious uveitis. For one population, 23 clinics in the Multicenter Uveitis Steroid Treatment Trial Follow-up Study (MUST FS) were contacted about their willingness to administer the Patient Preferences Questionnaire. This cohort of patients is being followed up for an additional 5 years after completion of the MUST Trial for long-term treatment outcomes as part of the MUST FS. Twelve clinics agreed to participate and administered the questionnaire to patients at their next scheduled follow-up visits between July 2013 and May 2014. Institutional review boards (IRBs) at the coordinating center and at all 12 participating clinical centers provided their approvals for administering the questionnaire. 

For the other population, outpatients with uveitis who were treated in ocular inflammation subspecialty clinics at the Scheie Eye Institute, University of Pennsylvania (referred to as PENN) or the Wilmer Eye Institute, Johns Hopkins University (referred to as JHU) were recruited. Between September 2013 and April 2014, patients at each clinic were recruited to complete the Patient Preferences Questionnaire while waiting for their ophthalmologist appointment. Eligible patients had a history of noninfectious uveitis and were at least 18 years of age. Patients with anterior uveitis were included along with intermediate uveitis, posterior uveitis, and panuveitis cases (those studied in the MUST FS). In addition to the survey, 14 respondent-specific questions about demographic and clinical characteristics were answered. All study procedures were approved by IRBs at both institutions. 

All research conducted adhered to the tenets of the Declaration of Helsinki. 

Data Analysis

To analyze BWS data, we computed an “individual BWS score” for each of the six outcomes, defined as the number of times an outcome “object” was picked as the most worrying by a participant minus the number of times it was picked as the least worrying among the presented objects. Because each object appeared five times across 10 BWS tasks, the individual BWS score for each object was on a scale bounded by −5 and 5, and the larger the score, the more worrying the outcome to the individual. Additionally, we counted the occurrence of best and worst choices across all participants to calculate the “aggregate BWS score” for each outcome. Best–worst scaling scores of this nature are easy to interpret and have been shown to provide sufficient statistics for various regression models.¹⁶ 

We compared the six individual BWS scores among the three patient groups (MUST FS implant therapy group, MUST FS systemic therapy group, and outpatients receiving ocular inflammation care at PENN/JHU), using nonparametric tests and simple linear regression. In addition, we explored potential associations between patient characteristics (including sex, age, race, education, time since diagnosis, location of uveitis, and experiences with treatment outcomes) with each of the six BWS scores. To account for potential confounders and to identify independent predictors, we constructed six multiple linear regression models with BWS scores of each outcome as the dependent variables. In each model we adjusted for patient characteristics simultaneously. In sensitivity analyses, we multiplied imputed missing data regarding patient characteristics to assess if the associations would differ significantly. We conducted analyses by using SAS 9.3 (SAS, Inc., Cary, NC, USA) and Stata 11.2 (StataCorp LP, College Station, TX, USA). P values less than 0.05 were considered statistically significant. 

Results

Patient Characteristics

As of July 2013, a total of 210 patients were under follow-up in the MUST FS. Between July 2013 and May 2014, a total of 89 patients across 12 MUST FS clinical centers completed the Patient Preferences Questionnaire at their study visits. Seven patients who did not provide valid responses on BWS tasks (in which the most and least worrying outcomes were not clearly indicated) were omitted from our analysis. Among the 82 patients included in our analysis, 36 were originally assigned to the implant therapy group and 46 to the systemic therapy group. Between September 2013 and April 2014, another 107 outpatients with noninfectious uveitis (73 from PENN and 34 from JHU) were also recruited for this study, among whom 100 patients (68 from PENN and 32 from JHU) completed the questionnaire and provided valid responses. Taken together, our final analysis was based on responses of 182 patients with noninfectious uveitis. 

The sociodemographic, disease, and treatment-related characteristics of the 182 study patients are shown in Table 2. Most were female (74%), white (58%), employed with income (57%), and were high-school graduates (30%) or had some college education (29%). The mean age was 52 years. The distributions of age and employment status were not significantly different between patients recruited from the MUST FS and patients recruited from PENN/JHU, but there were more male (15% vs. 35%) and black (29% vs. 44%) participants and more people having high educational level (college graduate or higher: 24% vs. 47%) in the latter population. The MUST Trial did not include patients with anterior uveitis, while approximately half of patients recruited from PENN/JHU had anterior uveitis (52%), with the result that the distribution of sites of uveitis involvement was approximately balanced. Patients from the MUST FS were more likely to have bilateral uveitis (88% vs. 72%), to have been diagnosed with uveitis for a longer time (median years: 10 vs. 5), and to have more experiences with treatments including corticosteroid injections, systemic corticosteroids, and eye surgery. But their experiences with certain diseases (associated systemic diseases, high blood pressure, high cholesterol, and glaucoma) were not significantly different from those of patients recruited from PENN/JHU, except for cataracts (more patients had cataracts in MUST FS). The vision status of these patients can also be found in Table 2. 

Table 2

 

Sociodemographic, Uveitis, and Uveitis Treatment–Related Characteristics of the Study Patients

 

Preferences Scores

Estimates of treatment outcome importance, using BWS, are presented in Table 3. We counted the total number of times across all surveys that each object (treatment outcome) was chosen as most or least worrying. The number of counts was divided by the availability of each object (each was presented a total of 910 times, five times for each of 182 participants) to calculate a proportion. The aggregate BWS score for glaucoma was the largest (387) and the score for cataract was the smallest (−333). The median and interquartile range of the individual BWS scores (bounded by −5 and 5) are also presented. 

Table 3

 

Estimates of Treatment Outcome Importance Using BWS

 

Box plots of the individual BWS scores (on a scale from −5 to 5) and the VAS scores (on a scale from 0–100) are shown in the Figure. According to the individual BWS scores of the six outcomes, we identified that vision not meeting the requirement for driving (median: 3), development of glaucoma (median: 2), and needing eye surgery (median: 1) were considered more worrying by study patients than the other outcomes. Distributions of the scores of vision not meeting the requirement for driving and infection (e.g., sinusitis) had the largest variability as shown by interquartile ranges. 

Figure

 

Individual BWS scores, VAS scores, and rankings for the treatment outcomes. The box plots represent the median, interquartile range, and 95% confidence interval for the scores of each outcome. Rankings are based on medians of the scores.

With regard to the VAS scores, the median score of each outcome was 80 (vision not meeting the requirement for driving), 85 (glaucoma), 70 (needing eye surgery), 70 (needing medicine for high blood pressure/cholesterol), 70 (cataracts), and 60 (infection, e.g., sinusitis). The ranking of the six outcomes by median of the individual BWS scores was different from the ranking by median of the VAS scores. Comparison of the distributions of medians of these two types of scores (see Fig.) demonstrated less overlap of the distributions with the BWS methods than the VAS methods. 

Associations Between Patient Characteristics and Preferences Scores

Table 4 shows that, in simple linear regression analysis, there were no significant differences in each score when comparing patients from MUST FS implant group or MUST FS systemic group to patients from PENN/JHU. Table 4 also shows the results of the six multiple linear regression models to identify independent predictors. For sociodemographic variables, male sex was associated with lower BWS scores of needing medicine for high blood pressure/cholesterol and older age was associated with lower scores of cataracts. White race was associated with higher scores of vision not meeting requirement for driving and lower scores of cataracts and infection (e.g., sinusitis). Higher educational level was associated with higher scores of vision not meeting requirement for driving and lower scores of infection (e.g., sinusitis). Disease characteristics such as time since diagnosis and location of uveitis (anterior versus other) were not significantly associated with BWS scores of any outcome. Furthermore, when we focused on the predictors indicating if patients had the respective outcome previously (cataracts, glaucoma, eye surgery, or high blood pressure/cholesterol), none of the associations reached the 0.05 level of significance. 

Table 4

 

Associations Between Patient Characteristics and Individual BWS Scores of Each Treatment Outcome

 

Discussion

Using a BWS approach to assessing patient outcome preferences, we found that patients with noninfectious uveitis considered some ocular adverse outcomes, that is, impaired vision, development of glaucoma, and need for eye surgery, more worrying than systemic adverse outcomes. These results help us distinguish between more and less worrying outcomes from the patients' perspective and may inform decision making on treatment of patients having noninfectious uveitis. Although the preference data had high variability, they were shown to be comparable across patients recruited from different sources in our study. 

Our study used a novel and easily implemented preference-elicitation approach to learn patient ratings of adverse treatment outcomes and provided quantitative data. This may help clinicians understand how patients themselves perceive the relative importance of treatment outcomes that are crucial for decision making. We saw that patients, as a group, do value adverse outcomes differently. For example, both glaucoma and cataracts are common complications in patients with uveitis as well as side effects from corticosteroid therapy,²¹ but on average, development of glaucoma was considered much more worrying by patients than the development of cataracts. Perhaps knowing the different disease prognoses and different abilities of the medical community to manage cataracts and glaucoma, patients had distinct views on these two complications.^22,23 Or, it may be that cataracts were more commonly known and less frightening than glaucoma to our study patients, so they rated them differently. 

One important methodologic question in a preference-elicitation study is if patients' prior experiences with the outcomes affect their ratings on such outcomes.^24,25 When designing the study, we chose to define broad inclusion criteria and enrolled two populations in order to have the opportunity to explore patient characteristics as an explanation for expressing different preferences. One of our patient sources (those participating in the MUST FS) included more severe cases of uveitis and more patients having long-term experiences with treatments and outcomes than the other source (those treated at PENN/JHU). According to the regression analyses, factors significantly associated with preferences were mainly sociodemographic factors. In contrast, we did not find outcome preferences to be significantly dependent on their prior experience with adverse outcomes. 

One of the challenges we encountered while designing the questionnaire was to ensure that patients had a common understanding of the adverse outcomes when they were doing preference-elicitation tasks. To achieve this goal, we asked patients to read the description of each outcome carefully and complete the VAS tasks before they did the BWS tasks. In the description of outcomes, we related vision to driving standard since vision is a rather abstract concept to describe. We restricted infections to sinusitis (most common one) since the term “infections” may be perceived very differently by individual patients. However, the interquartile ranges for the outcomes vision not meeting the requirement for driving and infection (e.g., sinusitis) were still the largest. We found that the heterogeneity of the preference scores for these two outcomes was mainly explained by race and education. This may be because patients with different races and educational levels do have different ratings of importance regarding vision not meeting the requirement for driving and infection (e.g., sinusitis). Or, this may reflect a methodologic issue that some patient groups (e.g., patients with lower educational level) were less consistent in their choices when doing the BWS tasks, which then led to the high heterogeneity found in the preference scores. 

As for comparing the results from VAS with BWS, it seems that using BWS is easier to differentiate the relative importance of the six outcomes than using VAS. One advantage of using BWS to elicit patient preferences is that it allowed us to ask patients in a way that they can make trade-offs between outcomes.¹⁶ In contrast, in doing VAS tasks, there are no trade-offs involved, so that method may be less sensitive to detect differences in the scores of the outcomes being rated.²⁶ Moreover, the anchors (e.g., most or least serious outcome in our case) of VAS may have various meanings to individuals, which affect our findings. We demonstrated in this study that most patients can complete BWS tasks to indicate their preferences without major difficulty. Investigators who plan to elicit patient preferences may consider using this approach for their future studies. 

Preference data on adverse outcomes, such as what we collected, are essential to benefit–harm assessment. With patient preference data, we are better informed of how patients, the most important stakeholders, consider the trade-offs between treatment benefits and harms. There is a growing interest in both US Food and Drug Administration and European Medicines Agency in incorporating patients' perspectives into their regulatory process, such as review of prescription drugs for marketing.²⁷ The current practice is somewhat limited to consulting patient groups or including patients on advisory committees but rarely uses such information quantitatively and explicitly.¹⁴ Our study demonstrated an alternative and more explicit approach to incorporating the patients' perspectives. We surveyed the key patient population who are most familiar with the condition and elicited their preferences for outcomes quantitatively. This also provides an effective way to engage patients in the process of developing evidence that informs preference-sensitive decisions. 

A number of limitations were identified. Our study elicited the “stated preferences” from patients, in which patients completed preference-elicitation tasks from their judgment of hypothetical descriptions of adverse outcomes. Thus, the study results may be dependent on how we described the outcomes that were rated. For example, the description for outcomes such as cataracts or glaucoma may be found oversimplified by some clinicians and such description may not grasp the complexity of the clinical cases seen in real-life practice. We might have underrated cataracts since we did not mention the possibility of loss of accommodation, or we might have overrated glaucoma since we did not emphasize that the type of glaucoma induced by the implant is not a “normal” case of glaucoma. Furthermore, sinusitis was chosen as the example for infection since it was the most common one. Had we chosen a more serious example for infection (e.g., pneumonia), the construct of the questions would have been different, which may lead to different patient preference findings. 

We included patients with anterior uveitis for whom either systemic or implant therapy may not be indicated; however, we did not observe significantly different preferences in patients with anterior uveitis compared to patients with intermediate uveitis, posterior uveitis, or panuveitis. The preference scores, though with high variability among individuals, were quite comparable across patients recruited from different sources in our survey. This is similar to previous BWS studies done in health- or non–health-related fields, where the studies also found heterogeneous individual preferences but comparable results across groups.^28,29 

Finally, our study aim was to elicit preferences regarding the adverse outcomes that are commonly seen in practice and measured in clinical studies. We developed our questionnaire by using review of the existing literature and consultation with clinical experts. To increase the feasibility of doing the survey among patients, we did not include a large number of potential adverse outcomes in the BWS tasks but the common and important ones. Many systemic adverse events, such as fracture, diabetes, and weight gain, though important, were not common in this group of patients according to the trial findings.² Others were not considered because we did not have data from the trial on their occurrence, for example, insomnia or irritability. A better way to develop the questionnaire would be to conduct qualitative research with patients (e.g., focus group³⁰) at the beginning. This would ensure that outcomes meaningful to patients are captured in the preference-elicitation tasks and included in clinical trial design. 

In summary, we have demonstrated a feasible approach to eliciting patient preferences regarding treatment adverse outcomes. Patients with noninfectious uveitis considered vision not meeting the requirement for driving, development of glaucoma, and need for eye surgery worrying adverse outcomes, suggesting that it is desirable to avoid these outcomes if possible. Future research may focus on learning how results of preference-elicitation study of this type can be applied in clinical settings to improve the communication between patients and clinicians regarding adverse outcomes, or finding effective ways to combine these results with clinical trial data to help patients and clinicians work together and develop preference-sensitive treatment decisions. 

Acknowledgments

We thank Tonetta Fitzgerald, Nancy Prusakowski, and all study coordinators who were involved in the data collection. We thank John Kempen, MD, PhD, for conduct of the survey and for commenting on the manuscript. 

Supported by cooperative agreements from the National Eye Institute to Mount Sinai School of Medicine (U10 EY 014655), The Johns Hopkins University Bloomberg School of Public Health (U10 EY 014660), and the University of Wisconsin, Madison, School of Medicine (U10 EY 014656). 

Disclosure: T. Yu, None; J.T. Holbrook, None; J.E. Thorne, None; T.N. Flynn, None; M.L. Van Natta, None; M.A. Puhan, None