March 2012
Volume 53, Issue 14
Free
ARVO Annual Meeting Abstract  |   March 2012
Ordinal Measurement Error Model for Assessing Agreement Among Raters for Glaucoma Progression
Author Affiliations & Notes
  • Yun Ling
    UPMC Eye Center, Eye & Ear Institute, Ophthalmology and Visual Science Research Center, Dept. Ophthalmology, U. Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
    Dept. Biostatistics, U. Pittsburgh Graduate School of Public Health, Pittsburgh, Pennsylvania
  • Richard A. Bilonick
    UPMC Eye Center, Eye & Ear Institute, Ophthalmology and Visual Science Research Center, Dept. Ophthalmology, U. Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
    Dept. Biostatistics, U. Pittsburgh Graduate School of Public Health, Pittsburgh, Pennsylvania
  • Hiroshi Ishikawa
    UPMC Eye Center, Eye & Ear Institute, Ophthalmology and Visual Science Research Center, Dept. Ophthalmology, U. Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
    Dept. Bioengineering, Swanson School of Engineering, U. Pittsburgh, Pittsburgh, Pennsylvania
  • Gadi Wollstein
    UPMC Eye Center, Eye & Ear Institute, Ophthalmology and Visual Science Research Center, Dept. Ophthalmology, U. Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
  • Joel S. Schuman
    UPMC Eye Center, Eye & Ear Institute, Ophthalmology and Visual Science Research Center, Dept. Ophthalmology, U. Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
    Dept. Bioengineering, Swanson School of Engineering, U. Pittsburgh, Pittsburgh, Pennsylvania
  • Footnotes
    Commercial Relationships  Yun Ling, None; Richard A. Bilonick, None; Hiroshi Ishikawa, None; Gadi Wollstein, None; Joel S. Schuman, Carl Zeiss Meditec (P)
  • Footnotes
    Support  R01-EY013178, P30-EY008098; Eye and Ear Foundation (Pittsburgh, PA); Research to Prevent Blindness
Investigative Ophthalmology & Visual Science March 2012, Vol.53, 6375. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Yun Ling, Richard A. Bilonick, Hiroshi Ishikawa, Gadi Wollstein, Joel S. Schuman; Ordinal Measurement Error Model for Assessing Agreement Among Raters for Glaucoma Progression. Invest. Ophthalmol. Vis. Sci. 2012;53(14):6375.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract
 
Purpose:
 

An ordinal measurement error model (OMEM) that describes agreement in terms of a slope (polychoric correlation) and intercept (threshold) was used to describe the agreement among 5 raters for glaucoma progression. This is an improvement over using kappa because, in general, agreement cannot be described as a single parameter.

 
Methods:
 

The agreement among 5 raters (A, B, C, D, E) judging progression in eyes of 60 subjects was modeled using a common factor measurement error model for ordinal data. It assumes that subjects had an unobserved (latent) progression factor on a continuous scale. Each rater had a latent continuous scale for judging but only ordinal outcomes (no progression, progression) were observed. The model for agreement includes a scale (slope, -1 < β < 1) that also measures the polychoric correlation between the common factor and each observed rating and simultaneously describes the precision. Thresholds (intercepts, α) convert the unobserved rater's continuous judgment to ordinal outcomes. Since there are two rating categories, one threshold is needed per rater. The probability of making a judgment of progression increases with the amount of true progression μ:P(rating = progression) = P(z > α - β*μ)where the probabilities come from a unit standard normal distribution. R software with OpenMx was used to describe the structural equation model (SEM) for the OMEM (Figure 1) and estimate the parameters.

 
Results:
 

Full information maximum likelihood (FIML) estimates of the most important and relevant functions of the model parameters are shown in Table 1. Rater E has the steepest slope, indicating E is the fastest in transition from non-progressor to progressor. Also E happens to have the lowest intercept, indicating E tends to rate more subjects as progressors. The estimated correlation between both eyes was ρ = 0.29, 95% CI (-0.06, 0.64). Subjects A, B, and C were very similar in their ratings and different from D and E.

 
Conclusions:
 

SEM accounts for the correlations between both eyes of each subject, and, when FIML is used, missing ratings. Correctly describing agreement requires two parameters: a scale that describes the rate of transition and thresholds that control when the transitions occur.  

 
Keywords: clinical (human) or epidemiologic studies: biostatistics/epidemiology methodology 
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×