To decide whether
R was significantly different from 1 (i.e., the proportion of one image format judged superior being different from the proportion of the other format judged superior), the comparison of the superior proportion between two image formats was performed by using the Generalized Estimating Equation (GEE) approach,
30 to account for the grading of the same images by multiple graders. For this analysis, gradings of “equivalent” were excluded because they do not contribute any statistical information on superiority (analysis executed through PROC GENMOD in SAS ver. 9.1; SAS Inc., Cary, NC). The dependent variable was the preference of image format (coded as 1 for the primary digital image superior, and 0 for alternative image format superior), and the gradings from the same image sets were identified by an image ID number. The binomial distribution was specified; and an independent working correlation structure was used. In this analysis, larger deviations of
R from 1 do not necessarily yield smaller
P values, as the correlation among gradings by multiple graders may differ for different image parameters, leading to different variance estimates and resulting
P values. The Bonferroni correction for multiple comparisons was used with 19 parameters for each set of the image format comparisons in
Tables 1 to
3, and statistical significance was therefore defined as
P ≤ 0.0026 (i.e., 0.05/19). The κ statistics, weighted κ, and percentage of agreement were calculated to assess the concurrence of the observers for the image assessments and to assess intraobserver agreement. For these calculations, the gradings were first compressed into a three-level scale (i.e., one-image format superior to the comparator format, formats equivalent, or the comparator format superior). In calculating the weighted κ statistic, Cicchetti-Allison weights
31 were calculated by using the formula:
Wij = 1 − [absolute value (
Ci −
Cj )/(
C 3 −
C 1)], where
Wij is the weight for the cell in column
i and row
j in a 3 × 3 contingency table, and C
i is the value of the index for column or row
i. The formula yields the following weights:
W 11 =
W 22 =
W 33 = 1,
W 12 =
W 23 = 0.5, and
W 13 = 0. In comparing monoscopic to stereoscopic viewing (see
Table 5), only four comparisons were made for each of the three image format comparisons, and the level of statistical significance was defined as
P ≤ 0.0125 (i.e., 0.05/4) per the Bonferroni correction.