Beyond Bland-Altman Plots - Using Structural Equation Models to Determine Bias and Imprecision for Comparative Studies of Multiple Devices

Abstract

 

Purpose:

 

Bland-Altman plots (BA) are not appropriate for data from both eyes, can only be used for 2 devices at a time, and cannot handle correlated measurement errors. When agreement is poor, BA provides no insight into how devices differ without making arbitrary assumptions that cannot be confirmed with the given data. Structural equation models (SEMs) provide a more informative comparison by determining the relative biases (including differences in measurement scale or scale bias) and device imprecision (adjusted for scale bias) simultaneously for any number of devices. SEMs easily accommodate measurements made on both eyes and correlated errors. An example of SEM is comparison of retinal nerve fiber layer (RNFL) thickness measurements among imaging devices.

 

Methods:

 

Global RNFL thickness measurements were made on both eyes using 3 optical coherence tomography devices (Cirrus[C], RTVue[R], Topcon[T]) for 20 subjects. 12 measurements per subject were used to assess simultaneously the relative bias (systematic error) and imprecision (random error) using SEM.

 

Results:

 

Parameter estimates (Figure) indicated the true values for each eye were highly correlated as are replicate pairs. The estimated standard errors for these correlations were very large given the small number of subjects. With Cirrus as reference, the calibration equations were:R = 9.52 + 1.16C & T = 16.63 + 0.94Csolve C:C= -8.21 + 0.86R & C = -17.69 + 1.06TThe latter results also could be determined by choosing RTVue and then Topcon, respectively, as reference.

 

Conclusions:

 

The limitations and misuse of BA can be avoided by SEMs to completely describe the relative bias and imprecision. Simulations using SEMs can be run to determine the sample sizes needed to achieve the desired reliability of the parameter estimates and the power for testing specific hypotheses.  

 

Keywords: clinical (human) or epidemiologic studies: biostatistics/epidemiology methodology • imaging methods (CT, FA, ICG, MRI, OCT, RTA, SLO, ultrasound)