Uncertainty in the diagnosis of Pre-plus Disease in Retinopathy of Prematurity

Abstract

Purpose: To measure the impact of pre-plus categorization by examining the diagnostic accuracy of experts as well as calculating quantitative tortuosity values using novel computer analysis.

Methods: Wide-angle retinal images captured from 73 infants during ROP-screening were interpreted by 2 experts using a 3-level (plus/pre-plus/neither) scale. Image-based interpretations were integrated with the clinical ophthalmoscopic exam to define a reference standard diagnosis for each image. Each image was analyzed quantitatively using a computer-based algorithm to generate a vascular tortuosity index (TI). Diagnostic performance of each expert and the computer-generated tortuosity index were compared based on absolute agreement and receiver operating characteristic area under the curve (AUC), with 3-level vs. 2-level (plus/not plus) classification of plus disease.

Results: Absolute agreement between experts and the reference standard was 92% in the 3-level classification and improved to 96% in the 2-level classification. Diagnostic performance of experts measured by AUC was 0.94 in the 3-level classification, and 0.97 in the 2-level classification. Diagnostic performance of the computer-based tortuosity index measured by AUC was 0.91 in the 3-level classification, and 0.90 in the 2-level classification. Sensitivity and specificity of computer-generated plus-disease diagnosis were 89% sensitivity/81%-specificity in the 3-level classification, and 100%-sensitivity/66% specificity in the 2-level classification.

Conclusions: Classifications of ROP that included a pre-plus disease category decreased the diagnostic performance of both experts and computer based systems. This study suggests that uncertainty in the diagnosis of pre-plus disease hinders diagnostic performance. Further clarification is needed in the definition of pre-plus disease to improve the accuracy and consistency of diagnosing retinopathy of prematurity.