The assessment of the reproducibility of manual vessel segmentation in fundus images

Abstract

Purpose: Automated or computer aided medical diagnosis may play an important role in future applications in ophthalmology. Our purpose was to determine the reproducibility of manual localization and segmentation of the retinal vascular tree on colour fundus images which could provide the basis for ground truth data in future studies.

Methods: High resolution colour fundus images of 5 healthy eyes were randomly chosen from the database of the Semmelweis University Diabetic Retinopathy Reading Center. Each image was exported and segmented manually by two experienced segmentation operators. The entire segmentation of the major and minor vessels was performed by hand using publicly available GNU Image Manipulation Program (GIMP, version 2.8.8. Based on the binary vessel images, vessel density distance map images were generated. The reproducibility of vessel density maps was calculated by the average vessel boundary distance between the edges of the segmented object in pixels, and calculating the F-score and Cohen's Kappa score between operators. The generated vessel density, vessel distance and vessel thickness values were compared by Mann-Whitney U test and their correlation coefficients.

Results: The average vessel boundary distance between operators was 4.39±0.92 pixels with an F-score of 0.83±0.02 and a Cohen’s Kappa score of 0.82±0.02. There was no statistically significant difference between vessel density, vessel distance and vessel thickness data, while their correlation coefficients were 0.88, 0.98 and 0.46, respectively.

Conclusions: Our results indicate that manual segmentation of retinal vessels on color fundus images is highly reproducible in the hands of experienced operators. It seems that vessel distance and vessel density are the most reliable measurement options while vessel thickness data should be used with caution. Our results give the possibility to create ground truth data for the further development of automated segmentation algorithms and also show that manual segmentation of the retinal vascular tree could be a reliable tool for clinical studies.

Keywords: 550 imaging/image analysis: clinical • 688 retina