Predictors of Discordance Between Dry Eye Symptoms and Signs: Insights From the Sjögren's International Collaborative Clinical Alliance

Abstract

Purpose: The diagnosis and management of dry eye disease (DED) could be complicated by the discordance between DED-related symptoms and signs. We performed a cross-sectional study to investigate the factors of and develop predictive models for the discrepancy in DED symptomatology.

Methods: We used data from 3455 participants, 21 to 89 years old, from the Sjögren's International Collaborative Clinical Alliance study. We performed a multivariable stepwise linear regression model with backward elimination and Bayesian information criteria to select predictors for the discordance in DES symptomatology, which was defined as the difference between the rank score of Ocular Surface Disease Index 6 (OSDI-6) and the rank score of ocular staining score (OSS).

Results: Ten predictors, such as “vitality,” “immunomodulating drugs,” sensory symptoms,” and “ethnicity,” remained in the final models, achieving an adjusted R² (aR²) of 0.35 (95% confidence interval [CI], 0.32–0.39). Specifically, medication use explained 19% (95% CI, 0.17–0.22) of the variance in the outcome, followed by medical history (aR² = 0.18; 95% CI, 0.15–0.21). Health-related quality of life contributed 16% to the variance in the outcome (95% CI, 0.13–0.19), and, last, demographics contributed 11% (95% CI, 0.09–0.13).

Conclusions: Our results suggest that individuals of Asian descent and those using immunomodulating medications often present with severe ocular signs that necessitate regular ophthalmological evaluations, even in the absence of proportionate ocular symptoms. Additionally, ocular symptoms, when accompanied by abnormal sensations in other parts of the body, could indicate systemic conditions that require further investigation and medical care.

Dry eye disease (DED) is a multifactorial disease of the ocular surface characterized by an imbalance in the quality and quantity of the tear film.¹ DED affects a significant portion of the world population, with prevalence ranging from 5% to 50%, depending on the area studied and the disease definition used, and it is a leading cause of ophthalmologic visits.² Despite this, many people with DED remain underdiagnosed and undertreated, and DED has been less investigated for youth and populations south of the Equator.³ The diagnosis of DED relies on a combination of symptoms and clinical signs. However, it is well-known that the DED symptoms and signs are notoriously discordant, and the discordance between ocular symptoms and signs could complicate the clinical diagnosis and management.^4,5 Given the potential detrimental effects that DED can have on vision, quality of life (QoL), work productivity in individuals, and the burden on society at large, it is essential to investigate the factors that contribute to the inconsistent correlation between ocular symptoms and signs of DED. 

A previous study investigated the discordance between symptoms and signs in DED within the Groningen LOngitudinal Sicca StudY (GLOSSY), a single-center clinic-based cohort.⁶ The study identified several factors associated with the discordance, including age, the presence of chronic pain syndromes, Sjögren's disease (SjD), atopic diseases, diabetes, and graft-versus-host disease. However, the model developed in that study only explained a limited portion of the variance in the outcome (15.4%). To provide more comprehensive insights and evidence-based recommendations for individuals experiencing DED, we explored patient characteristics associated with discordance between ocular symptoms and DED using data from a large, international multicenter cohort while incorporating additional variables including demographics and self-reported physical and mental health, alongside medical history. 

Methods

Study Population

This study was a secondary analysis of data from the Sjögren's International Collaborative Clinical Alliance (SICCA) cohort, a multisite research initiative conducted across diverse geographical regions. Participants were included from six international and three US research sites between September 2004 and September 2012. The detailed methodology and implementation of SICCA have been documented previously.⁷ Briefly, 3514 participants between the ages of 21 and 89 years at the time of their initial assessment were included if they were referred by their clinicians with suspicions of SjD or exhibited symptoms, signs, or abnormal test results suggestive of SjD. In this analysis of the factors influencing the discordance between DED symptoms and signs, we excluded individuals with more than 20% missing values or those lacking data for the outcome measures, namely the Ocular Surface Disease Index 6 (OSDI-6), which assesses ocular symptoms, or ocular staining score (OSS), which assesses ocular signs (n = 59). The remaining cohort of 3455 participants constituted the study population. 

Candidate Predictive Indicators

We extracted predictive variables from participants’ baseline questionnaires and findings from rheumatology and ophthalmology examinations. These variables covered a wide spectrum of information, including demographics, medical history, DED, systemic medications, and health-related QoL. Variables with rare or empty categories were combined or excluded. For example, rheumatologic diseases (other than being classified as SjD), such as rheumatoid arthritis, lupus, or amyloidosis, were combined into one variable because each was rare in the population. Because 1517 participants (43.9%) were classified as SjD based on the 2016 American College of Rheumatology (ACR)/European League Against Rheumatism (EULAR) classification criteria, we used SjD status as an individual predictor in the analysis.⁸ 

Outcomes

The discordance between DED symptoms and signs was measured using the difference between OSDI-6 (symptoms) and OSS (signs). OSDI-6, ranging in values from 0 to 24 with a cutoff of 4 or higher indicating symptomatic, is a shortened questionnaire of the full OSDI.⁹ It has demonstrated a significant correlation with the original OSDI and strong repeatability.⁹ OSS, ranging from 0 to 12 with a threshold of 5 or greater representing severe signs and compatibility with keratoconjunctivitis sicca (aqueous deficiency), is assessed through the combination of corneal fluorescein staining and conjunctival lissamine green staining.⁸ 

We quantified the discordance between DED symptoms and signs by calculating the difference between the rank scores of OSDI-6 and OSS, both ranging from 0 to 1.⁶ A consistent framework for the variables makes it easier to compare OSDI-6 and OSS on the same scale regardless of the absolute range magnitude. Further, the normalized rank score difference can promote faster convergence during gradient-based training in our models and improve model performance and accuracy. Additionally, the method allows for comparisons across studies with different sample sizes, as the normalization adjusts for the number of observations. 

Additionally, we explored two alternative methods to measure discordance. The first method was a raw-data discordance whereby we converted the OSDI-6 to the same numerical scale as the OSS and estimated the difference between the converted OSDI-6 and OSS. The second method was a three-level categorical discordance. Participants could be categorized into one of the following groups. The first group was symptomatic with mild signs, where the OSDI-6 was above the threshold and the OSS was below it. The second group was the concordant group, where the participants’ OSDI-6 and OSS were both normal or both abnormal. The third group was asymptomatic with severe signs, where the OSDI-6 was normal and the OSS was abnormal. 

Statistical Analysis

The missing values of the predictors were imputed 1000 times using a non-parametric multivariate imputation by combining random forest and multivariate imputation by chained equations (MICE) method.¹⁰ We then checked multicollinearity among predictive features. Variables were removed if they exhibited high correlation (>0.70) with others, as measured by correlation matrix, or had a substantial variation inflation factor (>5.00) in the linear regression model with all variables. 

We randomly partitioned the dataset into a training set (80%) and a testing set (20%). We selected predictors and developed the prediction models in the training data and examined model performance in the testing data. For continuous outcomes (namely, rank score and raw data), multivariable stepwise linear regression models with backward elimination including all variables were fitted. To streamline the model for this extensive cohort, we employed Bayesian information criteria (BIC) for model selection with a penalty parameter of log(n).¹¹ Subsequently, the selected predictors were included in a forward stepwise regression based on their contribution to the fit of the model. Model performance was assessed by mean squared error, R², and adjusted R² (aR²). The assumptions of linearity in the relationship between selected variables and outcomes, homoscedasticity of the residual variance, normality of residuals, and endogeneity between residuals and independent variables were verified. 

For the categorical discordance, we used a random forest model to identify the most important features for prediction accuracy. The number of selected features matched the number of variables selected by the multivariable stepwise linear regression model on rank-score discordance. The model was trained in the training set with 10-fold cross-validation, and the model performance was evaluated in the testing set using accuracy, recall, precision, F score, and area under the receiver operating characteristic curve (AUC). 

Given the focus of SICCA on studying SjD, we conducted a subgroup analysis defined by SjD status for the rank-score discordance. Finally, we performed two exploratory analyses: (1) including ocular symptoms other than the questions in the OSDI-6 in the model to measure the effect of those ocular symptoms to assist DED patients with symptoms to estimate their signs; and (2) including ocular pathologies in the prediction model to assess whether ocular pathologies would contribute to the discordance between ocular symptoms and signs. 

Results

Table 1 presents the demographics of the participants included in this analysis. A total of 3455 participants with a mean ± SD age of 52.9 ± 13.2 years comprised the study population, among them 3151 were female (91.2%) and 1517 were classified as SjD (43.9%). Among the participants, 848 (24.5%) reported having at least one form of rheumatologic disease. However, a significant proportion of participants (n = 3206; 92.8%) reported experiencing musculoskeletal or nervous system symptoms, and 2081 participants (60.4%) had sensory abnormality. Also, 1043 participants (30.2%) were taking anticholinergic drugs, and 1452 participants (42.0%) were on immunomodulating drugs. Notably, the study population exhibited lower physical component summary (PCS) and mental composite summary (MCS) scores than the US general population (PCS and MCS are standardized to a mean ± SD of 50 ± 10 using a 2009 US general population normative sample provided by Quality Metric). Moreover, 471 participants (13.6%) indicated moderately severe to severe depression, and 305 participants (8.8%) reported that bodily pain had severely interfered with normal work. 

Table 1.

 

Characteristics of 3455 SICCA Participants by the Discordance Status Between Dry Eye–Related Ocular Symptoms and Signs

 

Out of the total study population, 1845 participants (53%) had concordant symptoms and signs, 479 participants (14%) were asymptomatic but with severe signs, and 1131 participants (33%) were symptomatic but with mild signs (Table 1). Compared to participants in the asymptomatic with severe signs group, those in the symptomatic with mild signs group were more likely to be from American and European areas, be Caucasian, have higher education levels, be current or past smokers, experience oral pain, frequently encounter interference of bodily pain with normal work, have a higher composite pain score, have sensory abnormalities, and use artificial tears and anticholinergic drugs. Moreover, participants in the symptomatic with mild signs group were less likely to come from Asia, be classified as SjD, or be taking immunomodulating drugs, and they tended to have lower PCS and MCS scores. 

Consistent with high levels of discordance in the overall scores, the correlation between OSDI-6 and OSS was negligible, and there were no significant correlations between individual ocular symptoms and signs (Fig. 1A). Regarding the preselected predictive indicators, ethnicity, sensory symptoms, rheumatologic symptoms, items in the health-related QoL survey, indicators of neuropathic pain, and some systemic and topical medications showed some associations with discordance outcome variables (Fig. 1B). 

Figure 1.

 

Correlations between (A) ocular symptoms with signs examined in the SICCA study, and (B) selected predictors with ocular symptoms and signs collected from SICCA questionnaire. Irritation1, eye irritation while reading or driving a car for a long period; irritation2, eye irritation while watching TV or working on a computer for an extended period; Uncomf1, eye discomfort in wind or air drafts; Uncomf2, eye discomfort at places with low humidity such as air-conditioned or heated buildings or airplanes.

Figure 1.

 

Continued.

After removing predictors with multicollinearity, 31 predictors remained in the analysis. The participants’ characteristics were comparable between the training and testing datasets (Supplementary Table S1). Utilizing a multivariable stepwise linear regression with backward elimination on the rank-score discordance, we identified 10 predictors in the final model, achieving an aR² of 0.36 (95% confidence interval [CI], 0.33–0.39) (Fig. 2, Table 2). Vitality, an item in the Short Form 12-Item, version 2 (SF-12v2), of the self-report health survey, emerged as the most influential contributor to the variance in the rank-score discordance, followed by the use of immunomodulating drugs.¹² Sensory symptoms, such as tingling and sharp pain sensation over the body surface and oral pain, and demographic characteristics, such as ethnicity, were also the main contributors of the discordance between ocular symptoms and signs. Regarding the predictor groups, medication use explained 19% (95% CI, 0.16–0.22) of the variance in the outcome, followed by medical history (aR² = 0.18; 95% CI, 0.15–0.21). Health-related QoL contributed 16% to the variance in the outcome (95% CI, 0.13–0.19), and, last, demographics contributed 11% (95% CI, 0.09–0.13). Upon stratifying the analysis by SjD status, the aR² values of the prediction model were 0.40 (95% CI, 0.36–0.44) in SjD participants and 0.27 (95% CI, 0.23–0.32) in non-SjD participants (Supplementary Fig. S1, Supplementary Table S2). Across the overall and SjD stratified analyses, the use of immunomodulating drugs, ethnicity, use of artificial tears, and neurologic symptoms were the common predictors of rank-score discordance. 

Figure 2.

 

(A) The cumulative adjusted R² of model-selected predictors on rank-score discordance between ocular symptoms and signs. (B) The adjusted R² contributed by each predictor category.

Table 2.

 

Prediction Performance of the Rank-Score Discordance Model by Individual Predictor Category

 

When we included in the model additional ocular symptoms that are not queried in the OSDI-6, its prediction performance improved significantly, yielding an aR² of 0.43 (95% CI, 0.40–0.46). Gritty sensation contributed most to the prediction. Sensory symptoms, fluctuating vision, self-reported ability to make one's own tears, ocular pain, and eyes feeling dry were further selected by the model. In contrast, when including slit-lamp examination pathologies in the model, none was selected by the model, and the output remained the same as the main analysis (Supplementary Fig. S2, Supplementary Table S1). 

With respect to the outcome of the raw-score discordance, the same set of predictors, except for social function, was selected by the model despite the sequence differing modestly, and the model achieved a similar aR² of 0.34 (95% CI, 0.31–0.37) (Fig. 3, Supplementary Table S1). To assist patients in qualitatively self-assessing their DED condition, we conducted additional analyses on the categorical outcome that categorized participants as asymptomatic but with severe signs, concordant, and symptomatic but with mild signs. The nine-item Patient Health Questionnaire (PHQ-9) for depression emerged as the most predictive feature of the categorical discordance (Fig. 4). The trained model yielded an AUC of 0.70 (95% CI, 0.69–0.72) and an accuracy of 0.59 in the test data. 

Figure 3.

 

The cumulative adjusted R² of model-selected predictors on raw-score discordance between ocular symptoms and signs.

Figure 4.

 

Predictive accuracy contributions of model-selected predictors in the random forest model for categorical discordance between ocular symptoms and signs.

Discussion

This research represents one of the largest clinical studies exploring the factors predicting the discordance between DED symptoms and signs. Incorporating a broad range of predictors, our prediction model explained 36% of the variance in discordance between DED symptoms and signs. Our results suggest that health-related QoL, medication use, ethnicity, and abnormal sensations in other body parts may be related to the discordance in DED symptomatology. 

Despite the model explaining just over one third of the variance in the discordance of signs and symptoms, we identified several variables that may contribute to the discordance between DED symptoms and signs. Notably, the vitality health concept from the SF-12 questionnaire is the most influential factor in the main analysis on rank-score outcome. Vitality is a scale of mental health that measures the energy and fatigue levels of an individual, which can reflect physical and mental health and significantly impact daily functioning and overall QoL.¹² Previous research has consistently indicated that patients with DED often experienced measurable reductions in reported vitality and QoL.^13–15 In this study, we observed that participants with symptomatic DED but mild signs exhibited a significantly worse mean vitality score than those who were asymptomatic with severe signs (41.3 vs. 50.9). Further, participants with DED symptoms but mild signs reported worse scores across health-related QoL dimensions. In addition, participants with symptomatic DED but with mild signs demonstrated a higher prevalence and more severe depression, which was a significant predictor of the categorical discordance. This finding has been consistently supported by previous research indicating that depression is associated with more severe DES symptoms but not necessarily with DES signs, and the DES symptoms can in turn exacerbate depression and anxiety.^16–19 This highlights the importance of increased attention to overall QoL and mental health for DED patients at ophthalmologic clinics. 

The use of medication also contributed significantly to predicting the discordance in DED symptomatology. Individuals in the group characterized by symptoms with only mild signs were more likely to take anticholinergic drugs (41.6% vs. 12.5%), yet less inclined to use immunomodulating drugs (22.6% vs. 42.0%) compared to those exhibiting signs but without severe symptoms. Most anticholinergic drugs, which are antimuscarinic, are known to reduce both aqueous and mucus secretions from lacrimal glands and conjunctival goblet cells, respectively, thereby contributing to DED.²⁰ In addition, these drugs can also lead to other ocular complaints such as blurred vision, sensitivity to light, narrowing of the anterior chamber, and glaucoma.^21,22 Systemic immunomodulating drugs are not specifically prescribed for treating ocular signs of DED. However, ocular signs might be one of the manifestations of systemic rheumatologic diseases that require immunomodulating drugs. Although we did not ascertain the participants’ systemic burden of rheumatologic diseases, those on immunomodulating drugs showed a higher prevalence of positive anti-Sjögren's syndrome–related antigen A (SSA) antibodies (73.2% vs. 5.1%) and labial salivary gland with focal lymphocytic sialadenitis and focus score ≥ 1 (80.0% vs. 5.3%) compared to non-users. 

Further, medical history, such as tingling and sharp pain sensation over the body surface, neurological symptoms, and oral pain, also played a significant role in the discordance between ocular symptoms and signs. Previous studies have indicated that emotional and psychological stress, certain conditions such as migraines, medication side effects, and environmental factors can lead to symptoms in the eye or other parts of the body without corresponding signs.^23–26 It remains uncertain whether these symptoms co-occur with ocular symptoms or if they collectively indicate an underlying shared disorder. Consequently, ophthalmologists should consider a comprehensive approach when patients report ocular symptoms. 

Moreover, ethnicity stands out as another significant predictor of discordance in DED symptomatology. Asian ethnicity is a largely recognized DED risk factor, and previous studies have consistently reported a higher DED prevalence and more severe DED symptoms and signs among Asians than Caucasians.^3,27 In this study, though, we observed that Asian participants demonstrated milder symptoms but much worse signs compared to the Caucasian and other groups of participants. Notably, 14.9% of Asian participants exhibited normal OSDI-6 but abnormal OSS, whereas this proportion was 71.2% in the Caucasian participants. These observations raise concerns for potentially undiagnosed DED in Asian individuals, especially because DED patients often seek care at ophthalmologic clinics primarily due to their symptoms. 

When comparing DED symptoms and signs by SjD status, our previous study using the SICCA dataset showed that SjD participants demonstrated similar ocular symptoms but worse signs compared to non-SjD participants.²⁸ This discordance in DED symptomatology could be caused by increased ocular inflammation, ocular surface dysbiosis, and altered corneal nerve density and morphology in SjD patients.^29,30 Additionally, health conditions such as anxiety, depression, and self-perceived health may influence patients’ perception of ocular symptoms.³¹ Our results provide evidence regarding the impact of comorbid conditions, including abnormal sensory and neurological symptoms, and self-perceived health, such as vitality, on DED manifestation. Furthermore, the discordance of DED symptomatology in SjD may vary by ethnicity. The predictors of rank-score discordance among non-SjD participants are similar to those of SjD participants. However, the model demonstrated a significantly reduced predictive performance for the non-SjD group. It is possible that various factors contribute to the discordance in ocular symptoms and signs in the non-SjD population. Nevertheless, it is worth noting that non-SjD participants were required to present symptoms or signs suggestive of SjD at baseline to be included in SICCA. This selection bias could have exaggerated the inconsistency between ocular symptoms and signs among the non-SjD participants, making it less predictable. 

In addition to measuring the discordance between DED symptoms and signs using the difference between the rank scores, we propose an alternative method of measuring the difference between the raw scores of OSDI-6 and OSS which only requires simple transformation to the same scale. Though rank-score discordance has the advantage of disregarding data distribution, it does not retain the original data, thus resulting in a loss of information. Further, it is difficult to practice in a prospective cohort, and the interpretation of the model coefficients is less straightforward. Given that the outcomes of rank-score discordance and raw-score discordance showed similar patterns of distribution and that the selected predictors as well as the model performance were also close, it is reasonable to use raw-score discordance in future research, as it is easy to calculate and implement, especially in a prospective study. Future research is warranted to investigate and compare the results of these two outcomes. 

To date, only one other study, by Vehof et al.,⁶ has explored the predictors of the discordance in DED symptomatology using the GLOSSY cohort, and their model yielded an R² of 15%. The improved predictive performance in the current study compared with the previous work likely resulted from several significant factors. First, although the previous study primarily focused on DED-specific risk factors (mainly medical history and ocular and systemic therapy usage) as the predictors, we took advantage of the multidisciplinary effort of SICCA by incorporating additional demographics, self-reported physical and mental health, ocular symptoms, and pathologies into this analysis to uncover potential factors that could be related to the discordance between ocular symptoms and signs but had been previously unknown. Second, we utilized the shorter OSDI-6 rather than the OSDI, as it has been validated to have significant correlation with the original OSDI but stronger repeatability.⁹ In addition, instead of using the composite signs severity scores of six independent tests, including tear osmolarity, Schirmer test without anesthesia, staining of the cornea with fluorescein, tear breakup time (TBUT), staining of the nasal and temporal conjunctiva with lissamine green, and meibomian gland dysfunction, we opted for the OSS (a composite score of corneal fluorescein staining and conjunctival lissamine green staining) as the indicator for ocular signs. We did not adopt the composite signs severity score because tear osmolarity testing is not universally available, and only 2.2% participants in this study had tested tear osmolarity. In addition, the results of the Schirmer I test and TBUT can be influenced by various factors and may vary within the same individual over time. It was a concern whether a one-time assessment of the Schirmer I test and TBUT would represent the true state of DED. OSS, as supported by previous research, could be a more reproducible and sensitive indicator of ocular surface disturbance compared to other common clinical tests³²; however, it is not perfect and is subject to interrater and intrasubject variability.³³ Therefore, objective biomarkers with better repeatability are needed for DED. It is worth noting that the discordance, influenced by the inconsistent results of commonly used clinical tests, the inherent variability of the disease process, subjective symptoms perception, and cognitive responses to questions about ocular symptoms, could not be perfectly predicted.^1,34 Therefore, in clinical practice, it is essential to conduct thorough and separate assessments of both symptoms and signs, as they may represent two relatively independent types of DED indicators. 

Despite these differences, there are several important limitations of this study worth noting. First, the selection of participants who had symptoms and/or signs compatible with SjD could have exaggerated the discordance between ocular symptoms and signs among the participants and made it less predictable. Second, to assess the discordance between ocular symptoms and signs, we used OSDI-6 as the indicator of ocular symptoms and OSS for ocular signs. However, DED is characterized by great variability, and OSDI-6 and OSS each represent only a single facet of DED symptoms and signs. For example, we did not incorporate meibomian gland dysfunction, the leading cause of evaporative DED, in the model. Consequently, the outcome we used may not have fully represented the spectrum of DED symptoms and signs. Third, the participants were referred from clinical centers, with a large proportion being female and classified as having SjD. In addition, we did not validate this model in an external dataset; therefore, the results should be interpreted with caution when being generalized to male or other populations. Finally, there may be other potential factors, such as environmental triggers, medications, and lifestyle and dietary factors, that were not included in this model. Nevertheless, this study is the largest investigation thus far into predictors of the discordance between DED symptoms and signs. The SICCA cohort offered a unique chance to explore a broad range of variables across nine research sites and a wide range of ages. By identifying the factors that contribute to the discordance in DED symptomatology, our findings provide valuable insights into identifying underdiagnosed and undertreated patients, developing individualized treatment approaches, and enhancing patient management and education. Moreover, these factors may shed light on disease mechanisms and inspire future research. 

Conclusions

Our results suggest that individuals of Asian descent and those using immunomodulating medications often present with severe ocular signs that necessitate regular ophthalmological evaluations, even in the absence of proportionate ocular symptoms. Additionally, ocular symptoms, when accompanied by abnormal sensations in other parts of the body, could indicate systemic conditions that require further investigation and medical care. It is essential to treat ocular symptoms and signs as separate yet significant indicators of DED and to assess each thoroughly in clinical settings. Further research should aim to explore additional markers and develop more robust outcome measures that can comprehensively capture and understand the discordance between ocular symptoms and signs of DED. 

Acknowledgments

Supported by grants from the National Eye Institute, National Institutes of Health (EY026998 to JAG); by the National Institute of Dental and Craniofacial Research, National Institutes of Health (contract HHSN26S201300057C); and by the Office of Research on Women's Health, National Institutes of Health (contract N01 DE-32636). This work was also supported in part by an unrestricted grant from Research to Prevent Blindness. Its contents are solely the responsibility of the authors and do not represent the official views of the National Institutes of Health. 

Disclosure: F. Xiong, None; B.F. Arnold, None; T.M. Lietman, None; J.A. Gonzales, Dompé (C), Bristol-Myers Squibb (C)