EVALUATION OF READER AGREEMENT IN ASSESSMENT OF RETINAL IMAGES WITH RETINOPATHY OF PREMATURITY USING SOFTWARE ASSITANCE

Abstract

Purpose: To evaluate changes in reader variability for staging retinopathy of prematurity (ROP) and plus disease when using computer assisted tools for image grading

Methods: NeoScan is a software tool designed to provide image enhancement and quantitative measurements of retinal images from neonates aimed at reducing the number of inadequate images for quality and at reducing inter-reader variability. Quality and reading variability issues are hurdles to implementation of tele-ophthalmology solutions for ROP screening.<br /> To evaluate changes in reader variability, we performed a reading study with a retina specialist, a certified ophthalmic medical technician (COMT), and two ophthalmology residents. This group read images for twenty cases from the PHOTO-ROP study without NeoScan and then read the same dataset using NeoScan after a period of two weeks to three months. The graders evaluated the cases for quality, ROP stage, and plus disease.<br /> NeoScan has two software components: image enhancement and retinal vasculature assessment. For image enhancement three methods were combined: illumination correction, contrast limit adaptive histogram equalization, and anisotropic diffusion. For quantitative measurement of the retinal vasculature we calculated tortuosity and width for automatically segmented vessels. Tortuosity was calculated using a curvature-based method and vessel caliber was calculated by dividing the vessel into small segments of 15 pixels and measuring the width at every pixel cross section.

Results: There was a statistically significant reduction (p<0.02) in the amount of inadequate images for three out of the four readers. The average number of inadequate images decreased from 43% to 32%. When using NeoScan the COMT doubled her plus disease agreement coefficient (.29-.49) with the clinical ground truth. Also, the agreement coefficient for plus disease between the retina specialist and the COMT increased significantly, from 0.15 to 0.77 (p<0.05).

Conclusions: The reduction of inadequate images supports the readers’ assessment that NeoScan’s most useful feature was the image enhancement. The increased agreement in plus disease assessment between the retina specialist and the COMT indicates that NeoScan has a potential for reducing grading variability among experienced graders. A larger study needs to be conducted to confirm this hypothesis.