Functional Vision Assessment Over 4 Years in USH2A Using the Veteran Affairs Low-Vision Visual Functioning Questionnaire

Abstract

Purpose: The purpose of this study was to evaluate the validity of the Veterans Affairs Low Vision Visual Functioning Questionnaire (VALVVFQ-48) functional vision scores (FVS) in patients with USH2A-associated retinal degeneration. In addition, to correlate the change in the VALVVFQ-48 FVS with the change in visual function (VF) measurements.

Methods: The VALVVFQ-48 was administered verbally to participants ≥18 years of age at baseline, and after 2 and 4 years. Associations among changes in FVS and changes in VF measures were assessed using the Spearman correlation coefficients. Mixed effects regression models with a random intercept were used to estimate annual rates of change of FVS and their responsiveness to change in VF measurements.

Results: All domain scores (except visual motor) showed significant decline over 4 years. Changes in the VALVVFQ-48 domain scores were negligibly to strongly correlated with changes in clinical measures of vision function over 4 years (|r| = 0.02 to 0.61). All domains showed evidence of responsiveness to changes in some VF measures, particularly the visual field. Participants with improvement and worsening in FVS beyond the coefficient of repeatability (CoR) ranged between 17% and 46% across all domains. Ceiling effects at baseline precluded accurate calculation of change over time in 19% to 36% of participants in 3 domains.

Conclusions: The VALVVFQ-48 may not be a sensitive measure for evaluating longitudinal outcomes in all persons with USH2A-associated retinal degeneration.

According to the Global Retinal Inherited Disease dataset, retinitis pigmentosa (RP) is the most common inherited retinal disease (IRD), and USH2A is the most common implicated gene in RP and the second most common mutated gene in IRDs overall.^1–3 Pathological variants of the USH2A gene result in non-syndromic autosomal recessive RP (ARRP), characterized by progressive irreversible loss of visual function, or Usher syndrome type 2 (USH2), characterized by loss of hearing in addition to loss of visual function.⁴ Unfortunately, no treatments exist for the loss of visual function in USH2A-associated ARRP or USH2.⁵ A preliminary step toward development of therapeutics is to document and validate the characteristics of measures that will quantify the efficacy of future treatments. One category of measures that can fulfill this need is questionnaire-based patient reported outcome measures (PROMs).⁶ 

RUSH2A is a multicenter, longitudinal, natural history study designed to characterize progression of USH2A-related IRD using a variety of clinical functional tests, such as visual acuity (VA); static, kinetic, and micro-perimetry; and full-field stimulus testing, in addition to collecting PROMs.⁷ The aim of RUSH2A is to identify measures most suitable for use as outcome measures in future treatment trials. At the time of RUSH2A initiation, no IRD-specific PROMs were available, and the Veteran Affairs Low-Vision Visual Functioning Questionnaire (VALVVFQ-48) was chosen as an existing patient reported outcome (PRO) potentially suitable for use in patients with USH2A-related retinal degeneration with some modifications.^7,8 

The VALVVFQ-48 was developed to measure functional vision (FV) of veterans enrolled in low-vision programs during rehabilitation.⁹ It quantifies function based on 48 questions split over 4 domains, each assessing a participant's self-reported difficulty performing vision-related tasks.¹⁰ The scores for each of the four domains – reading, mobility, visual information processing (VIP), and visual motor – are calculated independently. An overall score based on all 48 questions also is calculated. 

In this study, we report functional vision of the RUSH2A cohort over 4 years as measured using the VALVVFQ-48 and evaluate its validity by examining cross-sectional and longitudinal associations between the VALVVFQ-48 scores and quantitative clinical measurements of visual function, including best-corrected visual acuity (BCVA), full-field stimulus threshold (FST), optical coherence tomography (OCT), microperimetry (MP), and static perimetry (SP). Henceforth, “FVS” is used to refer to the functional vision scores from VALVVFQ-48 questionnaire (overall and/or by domain), “overall FV” is used to refer specifically to the overall VALVVFQ-48 score, and “domain FV” is used to refer to a specific VALVVFQ-48 domain score. “VF,” or visual function, is used to refer to the clinical measurements of various aspects of vision, such as VA or visual field. 

Methods

RUSH2A Study Design

The RUSH2A study and methods have been described previously.¹¹ Briefly, the study is being conducted at 16 clinical sites and is registered on www.clinicaltrials.gov (NCT03146078). The protocol and informed consent adhered to the tenets of the Declaration of Helsinki and were approved by the ethics boards of each participating institution. Informed consent was obtained from each participant (or parent or other legally authorized representative for minors) before study enrollment. Participants were at least 8 years old and had disease-causing homozygous or compound heterozygous USH2A variants inherited in trans. The hearing loss history and baseline audiology examinations were reviewed by an audiologist to assign a clinical diagnosis of either USH2 or ARRP to each participant.^11,12 

Patient Cohorts

All participants from the RUSH2A cohort 1 (N = 105) were considered for inclusion in the analysis cohort. Cohort 1 participants had at least one eye with baseline Early Treatment of Diabetic Retinopathy Study (ETDRS) VA letter score of 54 or more (20/80 or better), stable fixation, and a clinically determined kinetic visual field III4e diameter 10 degrees or greater in every meridian of the central field. The study eye was defined as the eye having better VA at baseline. Ultimately, 3 cohort 1 participants aged ≤ 18 years were excluded from analyses as they completed a different PRO, as were 3 additional participants receiving investigational treatment in the non-study eye in clinical trials. 

Outcomes

At baseline, year 2, and year 4, participants scored their perceived ability to perform each activity on the VALVVFQ-48 on a scale from 0 (impossible to do) to 5 (no difficulty). Hence, a higher FVS equates to higher self-reported functional vision. As described in a previous publication,¹³ the method of successive dichotomizations (a polytomous Rasch model that allows for estimation of ordered response category thresholds) was used to recalibrate the VALVVFQ-48 items and category thresholds, and to estimate FV measures for each participant using the baseline data from the cohort. This was necessary to account for the substitution of two original questionnaire items with two alternative items and allow for the possibility of differential item ordering between the original low vision cohort and USH2A-associated RP cohort.¹⁴ 

Statistical Methods

The distributions of overall and domain FVS at each visit were summarized using means, standard deviations (SDs), medians, interquartile ranges (IQRs), and ranges. Spaghetti plots were used to visualize FVS of each individual study participant from baseline to year 4. Boxplots were used to display the distribution of domain FVS in the study cohort by clinical diagnosis subgroups, and histograms were used to visualize change in FVS from baseline to year 4. Duration of disease was computed based on the age of onset (participant reported) and date of enrollment in the study. 

Annual rates of change of the FVS with 95% confidence intervals (95% CIs) were estimated using mixed effects models with a random intercept. Baseline factors were assessed for their effect on rate of change by including an interaction term between each baseline factor and time from baseline visit in the models. A coefficient of repeatability (CoR) for each domain was calculated using the within-participant variance from a mixed model with random slope and intercept.¹⁵ This approach assumes a linear change over time on a person-level and provides a conservative estimate of CoR as deviations from the linear model are assumed to be solely from measurement error. 

Associations among change in FVS and change in other measures (BCVA, FST, OCT, SP, and MP) from baseline to 4 years were assessed with Spearman correlation coefficients and longitudinal mixed models. The latter contained terms to estimate both the cross-sectional association between the baseline clinical vision measurements and baseline PRO scores, and the longitudinal association between changes in the clinical vision measurements and changes in the PRO scores over time.¹⁶ As there was no correction of P values for multiple testing, it is possible some significant findings are due to chance. 

Correlation coefficients were interpreted as suggested by Evans: 0.80 to 1.0 = very strong; 0.60 to 0.79 = strong; 0.40 to 0.59 = moderate; 0.20 to 0.39 = weak; and 0 to 0.19 = negligible to very weak.¹⁷ Mixed models and correlation analyses were conducted using SAS/STAT software version 9.4 (SAS Institute, Cary, NC, USA). 

Results

Study Population

The flowchart showing the numbers of participants included in the analysis cohort and completing the VALVVFQ-48 at baseline, year 2, and year 4 are shown in Figure 1. Of the 105 participants in the vision cohort 1, there were 99 that were included in the analysis cohort, 99 of whom completed the VALVVFQ-48 at baseline, 95 completed the VALVVFQ-48 at year 2, and 91 completed the VALVVFQ-48 at year 4. Mean (SD) of BCVA at baseline was 81 (7) and mean (SD) of BCVA at year 4 was 77 (8). 

Figure 1.

 

Flowchart for participants.

As shown in Table 1, among the analysis cohort, the clinical diagnosis was USH2 for 59 (60%) participants and ARRP for 40 (40%) participants. Fifty-six participants (57%) were women and 89 (90%) were White. The median age was 38 years (IQR = 30 to 45 years) and the median duration of disease at enrollment was 13 years (IQR = 7 to 20 years). 

Table 1.

 

Participant Characteristics at Baseline for the Analysis Cohort (N = 99)

 

Distribution of FV Measures

FVS from baseline through year 4 and by disease duration are reported in Figure 2 and Supplementary Figure S1. There is a notable ceiling effect on reading, visual motor, and visual information domains. The distributions of FVS (without imputation of missing values) at each visit are reported in Table 2. The mean FVS across all domains at study baseline ranged from 2.1 to 3.1 logits, at year 2 from 2.0 to 2.8 logits, and at year 4 from 2.1 to 2.7 logits. There were no notable differences in FVS change by clinical diagnosis (Fig. 3, Supplementary Table S1). 

Figure 2.

 

FV score (FVS) for each participant by years from baseline. The range of scores in each plot includes the maximum and minimum possible scores for that domain, highlighting the ceiling effects of the population. The blue line represents the regression line for each domain FV from baseline to 4 years.

Table 2.

 

Distribution of FV Score (FVS) Measures at Baseline, 2, and 4 Years

 

Figure 3.

 

Distribution of changes in FV score (FVS) from baseline to 4 years by clinical diagnosis. The median is denoted by the line diving each box into two parts. Each box represents the middle 50% of scores for that group. The upper and lower whiskers represent scores outside the middle 50%. Mean (inside the boxes) and outliers (outside the boxes) are denoted by o and + markers for USH2 and ARRP, respectively.

When imputing missing values in sensitivity analyses, domain FVS were higher in the reading, VIP, and visual motor domains, suggesting less impairment in those domains than indicated by the original scores (Supplementary Tables S2A–S2C). 

Estimated Annual Change From Models

Overall and individual domain FVS showed a significant decline over 4 years, except for the visual motor domain. The overall FVS declined by −0.09 (95% CI = −0.15 to −0.04) logits/year, mobility FVS decline was −0.09 (95% CI = −0.15 to −0.02) logits/year, reading FVS decline was −0.09 (95% CI = −0.17 to −0.01) logits/year, VIP FVS decline was −0.08 (95% CI = −0.15 to −0.01) logits/year, and visual motor FVS decline was −0.02 (95% CI = −0.10 to 0.07) logits/year (Table 3, see Fig. 2). 

Table 3.

 

Estimated Annual Rates of Change in FV Score (FVS) Based on Random Intercept Models

 

Table 4.

 

Spearman Correlations Among Changes in FV Scores (FVS) and Changes in Clinical Vision Measures from Baseline to 4 Years (Preserved Cohort)

 

Associations Among Changes in Visual Function Measures and PRO Scores

Changes in VALVVFQ-48 overall and domain scores were negligibly to strongly correlated with changes in clinical measures of VF over 4 years (|r| = 0.02 to 0.61). There was a weak statistically significant (P < 0.05) correlation between changes in FST Red with changes in overall FVS (r = −0.27), changes in mobility FVS (r = −0.31), and changes in reading FVS (r = −0.34). MP mean sensitivity (MS) was weakly correlated with changes in reading FVS (r = 0.38) and changes in VIP FVS (r = 0.34), whereas changes in the SP pattern SD were very weakly or weakly correlated with all domains (r = 0.13 to 0.28). Changes in VIP scores were weakly correlated with changes in BCVA (r = 0.33), moderately correlated with changes in FST White (r = −0.58), and strongly correlated with changes in FST Blue (r = −0.61) (Table 4). 

Estimated changes in VALVVFQ-48 overall and domain FVS (logits/year) per unit change in the measures from the longitudinal mixed model analysis are shown in Table 5. Decline in any of the visual field measures was associated with increased difficulty in the overall, mobility, and VIP domains, whereas decline in central macula structure (OCT ellipsoid zone [EZ] area) and visual function measures (MP MS, V30, and center-weighted SP MS), although not BCVA, was associated with increased reading difficulty. Finally, a decline in BCVA was associated with increased difficulty in VIP and visual motor tasks. 

Table 5.

 

Estimated Change in VALVVFQ-48 Scores in Logits Per Unit Change in Functional Vision Measure

 

Coefficient of Repeatability

The CoR for each domain was used to calculate the percent of participants with improvement and worsening beyond CoR. Participants with improvement or worsening beyond CoR ranged from 17% to 46% across all domains (Fig. 4, Supplementary Table S3). 

Figure 4.

 

Histograms of change in FV and percent of participants with changes beyond CoR at 4 years. Dotted lines show coefficient of repeatability (CoR) for each domain, and the percent of participants with improvement or worsening beyond CoR.

Association of Changes in FVS With Baseline Characteristics

Table 6 shows the effect of baseline characteristics on the change in FVS over time as characterized by the difference in slope. The baseline score was found to have a significant association with change in FVS in all domains and in the overall FVS. Individuals with higher baseline FVS (better FV) experienced a greater decline in score over 4 years. Visual motor scores of individuals with a clinical diagnosis of USH2 declined more over 4 years than those with ARRP (difference = 0.22 logits, P = 0.009), possibly due to disease duration. The estimated difference by clinical diagnosis was smaller (0.14 logits) after removal of two high outliers in the ARRP group (see Fig. 3) but remained statistically significant (P = 0.048). 

Table 6.

 

Results for Differences in Slope for FV Score (FVS) Over Time by Baseline Characteristics

 

Discussion

All the VALVVFQ domains, except visual motor, significantly declined over 4 years, although standardized rates of decline per year (−0.37 to −0.13) were generally lower than those seen for the clinical measures as reported in a previous paper (−1.58 to −1.01).¹⁸ Moderate to strong correlations were found in change values over 4 years between the VIP domain score and FST White, the Reading domain score and MP MS, and the VIP domain score and FST Blue. The correlation between FVS and other clinical measures were generally negligible to weak. 

Using baseline FST data from RUSH2A, we previously found that FST White threshold below −30 decibel (dB) and a difference < −20 dB between blue and red thresholds was indicative of remaining rod function, with the converse indicative of thresholds that were cone-mediated.¹⁹ It is possible that the VIP scale is detecting differences in VIP associated with rod- versus cone-mediation of retinal sensitivity. 

Regression analysis using mixed model methods to estimate the change in FVS per unit change in the clinical VF measures suggested that the VALVVFQ-48 scores were responsive to changes in clinical measures of VF. The associations found, for example, the mobility score changes associated with change in visual field measures and the reading score changes associated with changes in central vision measures, lend support to the criterion validity of these domains in the study population. 

The regression analysis included both year 2 and 4 data, whereas the correlations used only year 4 data, likely improving the regression analysis power for detecting associations, and possibly explaining some of the differences between the correlation and regression analysis results. Both analyses found associations among change in MP MS and the overall and reading scales, and between several of the clinical measures (BCVA, FST Blue, and MP MS) and the VIP scale, suggesting that the VIP scale might be of use for monitoring changes in difficulty of visual tasks due to progression of USH2A-associated RP. 

To aid in clinical interpretation of the observed changes in functional vision, as measured by the VALVVFQ-48, we used 0.6 logits as a benchmark of change in greater than that expected by chance.¹⁸ This value corresponds to an estimated probability of 37.5% of responding an average of 1 category higher (or lower) on the item-level difficulty (none, moderate, extreme, and impossible) across all items in any VALVVFQ-48 scale between any two time points. We found that substantial proportions of participants (30–35%) had worsening of this magnitude at years 2 and 4 (35–46%), whereas 15% to 25% and 17% to 27% had improvement at years 2 and 4, respectively. Use of assistive devices and other adaptations to declining vision may have resulted in participants reporting improvement in visual task difficulty over time; unfortunately, this study did not collect data on device usage. 

We also found that, other than baseline FVS, other baseline characteristics largely had no detectable impact on change in FVS over the 4-year study, aside from a greater decline in visual motor scores in those with an USH2 diagnosis as compared to ARRP; this was partly attributable to two individuals who reported an unusually large gain in visual motor function in the ARRP group. In all domains, individuals having greater functional vision at baseline had greater decline, possibly as they have a greater ability for change, or “more function to lose,” when compared with individuals who started with poorer functional vision. 

One limitation of this study is the presence of a ceiling effect; multiple participants could not be assigned a score for 1 or more VALVVFQ-48 subscales because they answered “no difficulty” to all subscale questions. These participants accounted for most of the missing data for the affected subscales. This precluded obtaining accurate measurements of change for 19% to 27% of individuals for those subscales, decreasing precision, and introducing bias into estimated rates of change in the study cohort. 

Conclusions

The VALVVFQ-48 demonstrated the ability to measure changes in functional vision in many, but not all, individuals in our cohort due to ceiling effects in some subscales at baseline. All domains showed evidence of responsiveness to change in some visual function measures, particularly visual field. Given ceiling effects and weak to moderate correlation between functional vision measures and the VALVVFQ-48, it may not be a sensitive measure for evaluating longitudinal outcomes for all patients with USH2A-associated retinal degeneration. 

Acknowledgments

Supported by funding from the Foundation Fighting Blindness. 

The comprehensive list of FFB Clinical Consortium Investigator Group members participating in this protocol was previously published in Duncan JL, Liang W, Maguire MG, et al. Baseline Visual Field Findings in the RUSH2A Study: Associated Factors and Correlation with Other Measures of Disease Severity. Am J Ophthalmol. 2020;219:87–100. 

Other Writing Committee Acknowledgments: Jacque L. Duncan's work was made possible, in part, by NEI P30 EY002162 - Core Grant for Vision Research, and by an unrestricted grant from Research to Prevent Blindness, New York, NY. 

Disclosure: B. Parekh, None; N. Peck-Dimit, None; J.L. Duncan, Acucela (F), AGTC (F), Allergan/Abbvie (F), Biogen/NightstaRx (F), ProQR (F), PYC Therapeutics (F); Spark Therapeutics Choroideremia trial (S), and AGTC X-linked retinoschisis trial (S); L. Samarakoon, None; M.F. Abalem, None; C.A. Andrews, None; I. Audo, Novartis (C), and Janssen (C); A.R. Ayala, None; C. Bradley, None; J.K. Cheetham, Allergan/AbbVie (I); G. Dagnelie, None; T.A. Durham, None; R.M. Huckfeldt, BlueRock (C), Janssen (C), Sanofi (C), Sunovion (C), clinical trial support from Beacon Therapeutics (F), Biogen (F), Janssen (F), MeiraGTx (F), ProQR (F), and Spark Therapeutics (F); G.D. Lacy, University of Michigan (R); B. Malbin, None; M. Michaelides, MeiraGTx (C), (I), (O), Belite Bio (C), Saliogen (C), and Mogrify (C); D.C. Musch, University of Michigan (R), NEI STEM trial (C), and Belite Bio (C); K. Stingl, Novartis (C), ProQR (C), ViGeneron (C), Santen (C), Janssen (C) with all fees paid to Center for Ophthalmology, University of Tuebingen to support research; C.Y. Weng, Alcon (C), Alimera Sciences (C), Regeneron (C), Allergan/AbbVie (C), Novartis (C), REGENXBIO (C), Opthea (C), DORC (C), Genentech (C), Iveric Bio (C), EyePoint (C), Apellis (C), AGTC (F), DRCR Retina Network (F), Alimera Sciences (F); Springer Publishers (R), Women in Ophthalmology (S), American Society of Cataract & Refractive Surgery (S), American Society of Retina Specialists (S), and Macula Society (S); A.Z. Zmejkoski, None; M. Melia, None; K.T. Jayasundera, University of Michigan (R)