Estimating a Just-Noticeable Difference for Ocular Comfort in Contact Lens Wearers

Abstract

Purpose.: To estimate the just-noticeable difference (JND) in ocular comfort rating by human, contact lens–wearing subjects using 1 to 100 numerical scales.

Methods.: Ostensibly identical, new contact lenses were worn simultaneously in both eyes by 40 subjects who made individual comfort ratings for each eye using a 100-point numerical ratings scale (NRS). Concurrently, interocular preference was indicated on a five-point Likert scale (1 to 5: strongly prefer right, slightly prefer right, no preference, slightly prefer left, strongly prefer left, respectively). Differences in NRS comfort score (ΔC) between the right and left eyes were determined for each Likert scale preference criteria. The distribution of group ΔC scores was examined relative to alternative definitions of JND as a means of estimating its value.

Results.: For Likert scores indicating the presence of a slight interocular preference, absolute ΔC ranged from 1 to 30 units with a mean of 7.4 ± 1.3 (95% confidence interval) across all lenses and trials. When there was no Likert scale preference expressed between the eyes, absolute ΔC did not exceed 5 units.

Conclusions.: For ratings of comfort using a 100-point numerical rating scale, the inter-ocular JND is unlikely to be less than 5 units. The estimate for the average value in the population was approximately 7 to 8 units. These numbers indicate the lowest level at which changes in comfort measured with such scales are likely to be clinically significant.

Although technological advances permit ever more sophisticated data to be gathered from biological systems, subjective responses continue to be a vital component in many evaluations involving human subjects, whether these are performed for research or therapeutic purposes. Studies of the ocular surface are no exception to this; particularly when physical discomfort is involved, such as may occur during contact lens wear or in dry eye. Among the tools available for the purpose of assessing ocular comfort responses, numerical rating and visual analog scales are commonly chosen options ^{1 –7} ; however, there is little information in the literature pertaining to the limits these tools have in detecting differences in ocular sensation between presentations. In psychophysical terms, the just-noticeable difference (JND) for sensations of ocular discomfort is unknown. 

Usually described as the minimum change in stimulus intensity needed to produce a noticeable variation in perceived sensation, the concept of the JND is fundamental to psychophysics. Originally proposed by Weber, ⁸ JNDs have since been determined for a wide range of subjective phenomena including atmospheric haze ⁹ and refractive error, ¹⁰ as well as more abstract concepts such as meaningfulness. ¹¹ In a clinical context, JNDs are of particular interest where an intervention is primarily intended to provide some kind of symptomatic relief. It seems reasonable to suppose that any such intervention would be deemed useful only if the change in symptoms it produced was actually apparent to the patient. Thus, an effective treatment, product, or strategy for reducing the ocular discomfort caused by dry eye, or contact lens wear, for example, would have to produce a change in response in excess of the JND for that sensation. Knowledge of the JND would thus be a valuable aid during product or treatment development, as it sets a minimum level of performance that must be attained before subjective efficacy is recognizable to users. Looked at another way, the JND may be seen as representing a level of change that has clinical relevance or significance. 

Typically, JNDs are determined by asking subjects to make forced choices between two simultaneously presented stimuli of different magnitude, as the interstimulus difference is varied over a series of trials. The 50% point of the resulting discrimination probability curve may then be taken as the JND, although other values can be used to suit the needs of the individual researcher. 

It will be readily apparent that this approach does not easily lend itself to situations, such as ocular comfort, where the stimulus cannot be readily manipulated. While it is quite possible to extract a subjective estimate of the amount of comfort associated with any given situation by using an instrument such as a visual analog ^1–2 or numerical rating scale, ^{3 –7} the lack of an obvious way to incrementally vary the magnitude of comfort precludes determination of the JND by traditional means. 

This article describes an attempt to overcome this difficulty by using the random variations that are invariably introduced during the manufacturing process between contact lenses that are intended to be identical. It was hypothesized that a range of these differences exists, such that when identically labeled lenses are worn by any individual, there is some probability that an unequal physical sensation will be perceived. Hence, the subjective, interocular difference in comfort will vary between trials, sometimes being evident and other times absent. By asking subjects to rate their comfort for both eyes across a series of such wearing trials, it was anticipated that an estimate of the rating difference corresponding to the smallest, subjectively perceptible sensation would be forthcoming. This may then be taken to represent the JND for ocular comfort during contact lens wear, as measured by the relevant rating tool. 

Methods

The protocol was approved by the Vision CRC (Cooperative Research Centre) Human Ethics Committee and complied with the Declaration of Helsinki 1975, as revised in 1989. All subjects signed a record of informed consent. 

Forty subjects took part in the study. As we were not interested in making comparisons between groups of subjects in terms of their responses, a traditional sample size calculation was not appropriate to the study design, and this number was chosen so that there would be a reasonable distribution of cases across the various possible judgment categories. All were in good health and were examined to ensure that they were free of ocular abnormality. 

At the beginning of the study, a pair of apparently identical, new contact lenses of the same type was placed on the eyes of each subject. The eye into which the first lens was inserted was chosen at random, and subjects were masked to lens type and were not informed that the lenses were identical. Immediately after initial insertion, the lenses were manipulated by gentle lateral displacement to remove any debris from the postlens space. Subjects were then instructed to blink normally. After 15 minutes, subjective comfort was assessed for each eye, using a 1 to 100 numerical rating scale (NRS) with anchors at 100 (“perfect, I cannot feel the lens on my eye”) and 1 (“intolerable, lens must be removed immediately”). Subjects then indicated their preference between the two eyes using a five-point Likert scale (1 to 5: strongly prefer right, prefer right, no preference, prefer left, strongly prefer left, respectively). The specific question put was: “Of the two lenses in your eyes, which do you prefer in terms of comfort?” 

After this, both lenses were removed and discarded. 

The whole procedure was repeated three times: once using the same lens type and twice with a different lens type. This method increased generalizability and permitted the consistency of the outcome to be assessed. The two lens types used were:

•  
  Lens A: 8.4/14.0/-3.0/ACUVUE OASYS (Johnson & Johnson Visioncare, Jacksonville, FL)
•  
  Lens B: 8.6/14.2/-3.0/Air OPTIX (Ciba Vision, Duluth, GA)
•  
  At least 15 minutes elapsed between the end of one trial and the start of another, and no more than two wearing events took place on any given day. All the trials took place in the same airconditioned, windowless, illumination-controlled clinic room.

Results

The mean age (± SD) of the 40 subjects who participated was 33.1 ± 8.6 years, with a range of 21 to 60 years. There were 27 women and 13 men, and most (n = 28) were not habitual contact lens wearers. As a means of demonstrating the general similarity between the eyes of individuals in the study, Table 1 summarizes corneal curvature and refraction data, expressed in terms of the difference between the eyes of each subject. Note that data were available for only 39 subjects. As can be inferred from the means and their associated 95% confidence intervals (CIs), within-subject differences in curvature and best vision sphere were not significant, with the maximum values being 0.75 and 1.00 D, respectively. 

Table 1.

 

Summary Statistics for Eye Shape and Refraction, Expressed as Differences between the Right and Left Eyes of Subjects

	Best Vision Sphere (D)	Corneal Curvature (D)
		Average of Flat and Steep Meridians	Steep Meridian	Flat Meridian
Mean	−0.01	−0.04	−0.05	−0.03
95% CI	0.10	0.06	0.07	0.08
Max	0.75	0.58	0.40	0.75
Min	−1.00	−0.45	−0.55	−0.55

  n = 39.

The modulus (absolute value) of the difference between NRS comfort scores from right and left eyes (Mod ΔC), at each wearing event, was calculated. These values were then arranged into three groups according to the strength of preference between right and left eyes as indicated by the corresponding Likert scale response. Thus, events where subjects reported a strong preference between the eyes (Likert 1 or 5) were placed in one category, while those where preference was slight but still evident (Likert 2 or 4) were collected into a second. A third group held those events where no preference was expressed (Likert 3). 

Figure 1 shows the distribution of Mod ΔC scores across the various Likert scale categories. 

Figure 1.

 

Distribution of absolute comfort differences between right and left eyes (Mod ΔC) according to the corresponding Likert scale score: strong preference, Likert 1 and 5; slight preference, Likert 2 and 4; no preference, Likert 3.

Table 2 gives summary statistic for each of the 12 possible groups, all but four of which were not normally distributed (Shapiro-Wilk, P > 0.05). 

Table 2.

 

Summary Statistics for the Modulus of Differences between Reported Comfort Ratings for the Right and Left Eyes (Mod ΔC)

	Lens A		Lens B
	Trial 1	Trial 2	Trial 1	Trial 2
Strong Preference Expressed (Likert 1 and 5)
Mean	13.0	5.0	25.3*	22.7
95% CI	0.1	—	0.2	0.4
Max	20	5	40	90
Min	10	5	14	5
25th Percentile	10	5	20	10
50th Percentile	10	5	25	15
75th Percentile	18	5	30	28
95th Percentile	18	5	30	28
Count	5	1	9	12
Slight Preference Expressed (Likert 2 and 4)
Mean	4.1*	6.8*	6.5	6.6
95% CI	0.1	0.1	0.1	0.1
Max	10	25	30	20
Min	0	0	0	0
25th Percentile	2	2	2	3
50th Percentile	4	5	5	5
75th Percentile	5	10	10	10
95th Percentile	5	10	25	20
Count	14	19	26	21
No Preference Expressed (Likert 3)
Mean	0.5	0.3	1.6*	0.1
95% CI	0.0	0.0	0.1	0.0
Max	5	3	5	1
Min	0	0	0	0
25th Percentile	0	0	0	0
50th Percentile	0	0	1	0
75th Percentile	0	0	4	0
95th Percentile	5	3	4	0
Count	21	20	5	7

 Data are shown for two replicates each of two different lens types and are stratified on the basis of the strength of preference between right and left eye as expressed on a five-point Likert scale at the time of rating (1, strongly prefer right; 2, prefer right; 3, no preference; 4, prefer left; 5, strongly prefer left).

*  Normally distributed data (Shapiro-Wilk P > 0.05).

For the slight- and no-preference groups, there was good consistency between repetitions of the same lens type, as well as between different lenses. More variability was evident in the strong preference group, as would be expected, given the openendedness of this classification on its upper side. By definition, data falling into this group came from judgments of markedly dissimilar ocular sensations that would be well above the JND at threshold. Permitting participants to choose this category served to remove the more extreme responses from the slight preference group, thus reducing bias. We will not consider the strong preference data beyond this point. 

Fundamental to this analysis is the proposition that, by using Likert scales, we can classify responses as being either above the JND for ocular comfort between the eyes (i.e., Likert scores of 1, 2, 4 or 5) or below the JND (i.e., Likert score of 3), while simultaneously accessing ratings on an interval scale for the individual comfort levels of each eye. Having made this classification, how do we interpret the data to estimate where the JND occurs? 

While several approaches are possible, perhaps the most obvious would be to define the JND as the smallest Mod ΔC for which participants reported a R versus L preference. Unfortunately, a problem arises because in certain trials, some individuals seemed unable to express a differential comfort rating between the eyes. That is, they had Mod ΔC = 0, even when they preferred the way one eye felt, as indicated by choosing a Likert score ≠ 3. The converse situation also occurred, where non-0 Mod ΔC scores were sometimes returned when there was no interocular preference (i.e., Mod ΔC > 0, when Likert score = 3). We refer to these phenomena as “hyporating” and “hyperrating,” respectively, and subjects displaying these traits are listed in Table 3. Note that neither of these groups differed significantly from each other, or from normal subjects, in terms of the sex (P > 0.61), contact lens–wearing status (P > 0.38; both χ² test), corneal curvature (P = 0.76), best vision sphere (P = 0.07), or age (P = 0.55; all one-way ANOVA) of the component individuals. 

Table 3.

 

Absolute Comfort Differences between Right and Left Eyes (Mod ΔC) and Likert Scale Scores for Subjects Classified as Hyporaters and Hyperraters

Subject	Lens	Replicate	Mod ΔC	Likert Score
Hyporaters
3	2	2	0	2
9	1	1	0	2
13	2	1	0	2
33	1	1	0	2
33	2	1	0	2
33	2	2	0	4
34	1	2	0	4
34	2	1	0	4
37	1	2	0	2
39	1	2	0	4
40	2	1	0	2
Hyperraters
2	1	1	5	3
4	2	1	2	3
16	1	2	3	3
22	2	1	5	3
27	1	1	1	3
28	1	2	2	3
30	1	1	2	3
36	1	1	3	3
38	2	1	1	3
38	2	2	1	3

Because of the phenomenon of hyporating, the “minimum Mod ΔC required to create a preference” approach resulted in an unhelpful JND estimate of 0. Thus, the working definition for the JND was modified to be the average Mod ΔC for which participants reported a slight R versus L preference. Data relevant to this criterion are given in Table 2 and ranged between 4.1 and 6.8 units, depending on the lens and trial. Note, however, that because of the presence of 0 scores from the hyporaters in the sample, this outcome is likely to be biased to some degree. Accordingly, a supplementary calculation was made after excluding these individuals, to obtain a better estimate. Table 4 shows the results, which ranged from 4.8 to 8.1 units across the four conditions. Having four replicates permitted comparison of the JND estimate from each as a means of assessing repeatability, and the similarity of the resulting values is encouraging in this respect. 

Table 4.

 

Summary Statistics for Absolute Comfort Differences between Right and Left Eyes (Mod ΔC)

	Lens A		Lens B		Overall
	Trial 1	Trial 2	Trial 1	Trial 2
Mean	4.8	8.1	7.7	7.3	7.2
95% CI	1.5	3.0	2.7	2.1	1.3
Max	10	25	30	20	30
Min	2	2	1	2	1
25th Percentile	3	4	4.25	5	4
50th Percentile	4.5	5	5	5	5
75th Percentile	5	10	10	10	10
95th Percentile	10	17.5	24	15.5	15
Count	12	16	22	19	69

 Data are shown for subjects who expressed an R Versus L preference (i.e., Likert Score ≠ 3, after excluding hyporater responses.

Discussion

While necessarily indirect, the method described above for estimating the JND for ocular comfort has the major advantage that the outcomes should not be sensitive to the lens type used, because when subjects make comfort judgments, they do so for both eyes concurrently, presumably according to the same internal criteria. While it is possible that these may vary according to the applied stimulus, our analysis uses only the relative magnitude of these judgments to make inferences about the JND. It should not matter then, if we were to use different lens types, or for that matter a completely different stimulus, provided that the presentation occurs to both eyes simultaneously. 

Interpretation of the data in this study was complicated by the discovery of the hypo-/hyperrater phenomenon. Each of these behaviors manifested in approximately 6% of all trials, with 20% of the total sample hyporating and 23% hyperrating. Although it is, of course, possible that these anomalous responses indicate nothing more than random noise within the experiment, it is interesting to note that there was no overlap between the subjects in these two groups. Thus, hyperraters constituted a different set of people from hyporaters. Furthermore, some individuals replicated the specific response of hypo- or hyperrating on more than one trial, suggesting that it is a repeatable phenomenon. Based on these observations, it may be that these are distinct characteristics of different portions of the population. Alternative explanations include the prospect that there is multidimensionality in the perception of ocular comfort, ¹² which is not amenable to capture by these simple psychophysical scales, or simply that these subjects did not fully understand the task allotted to them. 

Whatever the actual explanation may be, the existence of hyporating in particular, clouds the picture emerging from an attempt to estimate the JND by finding the minimum interocular comfort difference score for which a preference was expressed. Even if this were not the case, that approach would probably still be viewed unfavorably by many, as it is dependent only on the most sensitive individual in the sample. Single-point estimators such as this are likely to be significantly biased, and alternative statistics that are more representative of the whole sample and do not incorporate 0 responses will generally be more acceptable. This was the reasoning underpinning the choice of the sample mean to represent the JND, as outlined above. 

It is acknowledged that some readers may wish to choose statistics other than the mean to define the JND, particularly given the non-normality of several data groupings, and several alternatives have been added to Tables 2 and 4 in an effort to assist them. Note, however, that whichever statistic is chosen, the four trials are quite consistent in the values that they return. 

Additional insight can be gained by looking at the lower end of the range of possible values and considering a different criterion—namely, the maximum Mod ΔC—for which no interocular preference could be discerned. Although it will be noted that this is again a single-point estimator, on this occasion, it is one with a useful and meaningful interpretation because it indicates where the absolute floor value of the JND lies. Inspection of Table 2 shows that when it was not possible for subjects to choose between their eyes, as shown by a Likert score = 3, none of them had an interocular comfort difference of more than 5 units. Thus, because differences below this level were not discernible by anyone in this sample, a fair conclusion seems to be that the JND is likely to be greater than 5 units. 

While the working value of JND used depends on the philosophy and needs of the user, it is clear from the foregoing that there is no justification for using any number lower than 5. Practically speaking, higher values are likely to be needed to appropriately reflect the behavior occurring in proportions of any real population that would, of course, be composed of several individuals and so present a range of responses. 

For those comfortable with the average, a JND estimate of ∼7 or 8 units seems reasonable. The associated 95% CIs are tolerably narrow, suggesting that sufficient subjects were available to make the assessment, despite it being necessary to segment the original sample to do so. Nevertheless the working value may have to more than double if, for example, it is necessary to capture activity up to the level of the 95th percentile in the population. This being said, it is worth remembering that the experimental paradigm used here did not permit control over the magnitude of the discomfort stimulus. Interocular differences perceived on any exposure to the lenses were therefore caused by effects that were random and of unknown magnitude, meaning that some of the larger responses were quite possibly due to stimuli substantially above the JND; even though these still fell within the slight preference Likert category. Using a statistic, such as the 95th percentile, that includes responses toward the higher end of the recorded range will thus tend to overestimate the JND. 

From a practical point of view, these results set out how large any change in comfort must be before it can be said to matter to an individual. Put another way, it defines the level of clinical significance for this phenomenon, a quantity that is often important to gauge. For example, during clinical study design, it is usually necessary to calculate the required sample size, and, among other things, this calculation relies on knowing the smallest difference that it is desirable to detect. ¹³ In many situations, appreciating the level at which the sensation becomes clinically significant will serve that purpose well. At the other end of the experimental process, when interpreting results where an analysis has indicated statistical significance, investigators may ask what the practical relevance of the finding might be, particularly where the effect sizes are small. Being able to call on a value for clinical significance as a reference against which to judge such effects is one way to address this question. 

Finally, it should be remembered that the conditions of this study would have resulted in sensations that were at the lower end of those capable of perception in the ocular environment. Although this will be appropriate for many practical situations involving discomfort due to contact lenses or dry eye, in accordance with Weber's law, larger JNDs would be expected in circumstances in which higher levels of discomfort are encountered. 

Conclusion

For ratings of comfort during contact lens wear using 100-point numerical rating scales, the interocular JND was always greater than 5 units, with the mean value being 7 to 8 units. These numbers indicate the lowest level at which changes in comfort measured with such scales are likely to be clinically significant. 

There appear to be distinct groups in the populations who are either capable of reporting a different comfort rating between their eyes, even when they do not perceive an interocular preference (hyperraters), or do not necessarily report a different comfort rating between their eyes, even when they do perceive an interocular preference (hyporaters). 

Footnotes

 Supported by the Brien Holden Vision Institute, the Australian Government Cooperative Research Centre Program, and in part by CIBA Vision.

Footnotes

 Disclosure: E.B. Papas, Ciba Vision (F); L. Keay, Ciba Vision (F); B. Golebiowski, Ciba Vision (F)

The authors thank Andreas Hartwig and Kathy Laarakkers for assistance with data collection.