June 2015
Volume 56, Issue 7
Free
ARVO Annual Meeting Abstract  |   June 2015
Bayesian Test-Retest Repeatability Tells You Want You Want To Know In Equivalency Of Imaging Metrics
Author Affiliations & Notes
  • Natalie Hutchings
    School of Optometry & Vision Science, University of Waterloo, Waterloo, ON, Canada
  • Dylan Simpson
    Dept. of Political Science, Wilfrid Laurier University, Waterloo, ON, Canada
  • Trefford L Simpson
    School of Optometry & Vision Science, University of Waterloo, Waterloo, ON, Canada
  • Footnotes
    Commercial Relationships Natalie Hutchings, None; Dylan Simpson, None; Trefford Simpson, None
  • Footnotes
    Support None
Investigative Ophthalmology & Visual Science June 2015, Vol.56, 5248. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Natalie Hutchings, Dylan Simpson, Trefford L Simpson; Bayesian Test-Retest Repeatability Tells You Want You Want To Know In Equivalency Of Imaging Metrics. Invest. Ophthalmol. Vis. Sci. 2015;56(7 ):5248.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract
 
Purpose
 

To investigate Bayesian approaches to assessing test-retest variabilty and compare the findings with standard method comparison statistics.

 
Methods
 

A dataset of objective measures of tarsal conjunctival roughness were used to evaluate the methods. Images of the tarsal conjunctiva (n=29) with various amounts of unevenness in their specular reflection ('roughness'), were manually processed (NIH ImageJ) twice by a technician naive to the experiment’s purpose. Three of the 13 metrics of roughness obtained1 were evaluated (Ra, Rq & Rt) using a Bayesian approach. Prior distributions of the mean, SD and normality of the difference between the 1st and 2nd evaluation were estimated as wide normal, wide uniform and wide exponential respectively. Posterior distributions were derived from which a joint distribution of effect sizes was generated. The highest density interval (HDI) of the posterior distribution of effect size was compared to a region of practical equivalence (ROPE; Cohen’s d=0.5 & 0.8). The interpretation was compared to standard metrics in method comparison of CCC (R; package epiR) and, mean difference and limits of agreement (LOA).

 
Results
 

Bayes: The mean difference (modal SD) for each parameter were Ra +1.1(1.9), Rq +0.5(2.0) and Rt -0.6(1.7). For a medium effect size (d=0.5) 37%, 87% and 90% of the posterior distribution HDI lay within the ROPE for Ra, Rq and Rt, respectively. The corresponding data for a large effect size (d=0.8) were 86%, 99% and 99%. CCC & LOA: The mean difference (SD) were Ra +1.2(2.0), Rq +0.6(2.5) and Rt -3.32(15.9). Comparative outcomes for Rt are shown in Figure 1.

 
Conclusions
 

In test-retest repeatability, the intention is to characterise the difference between repeat measures. Point estimates of mean difference and LOA may give misleading results when there are departures from concordance. Since the Bayesian approach directly determines the parameter of interest and its varaiblity, acceptance of 'no difference' and gives more insight into equivalency.<br /> 1Chinga G et al. J Microsc. 2007;227:254-65.  

 
FIgure 1: (Left) )Posterior distributions of the differences, SD of the differences and effect size. The highest density interval (HDI) indicates probable values for each parameter. The region of practical equivalence (ROPE) is shown for Cohen's d=0.8. (Right) Shows evaluation 1 vs. evaluation 2 (top) and the corresponding mean/difference plot (bottom).
 
FIgure 1: (Left) )Posterior distributions of the differences, SD of the differences and effect size. The highest density interval (HDI) indicates probable values for each parameter. The region of practical equivalence (ROPE) is shown for Cohen's d=0.8. (Right) Shows evaluation 1 vs. evaluation 2 (top) and the corresponding mean/difference plot (bottom).

 
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×