Evaluating Software-Assisted Grading Performance In The Detection Of Retinal Vasculature Abnormalities

Abstract

 

Purpose

 

To evaluate the effectiveness of the comprehensive assessment of retinal vasculature (CARV) software as a screening aid-tool for readers in the detection of retinal vasculature abnormalities.

 

Methods

 

VisionQuest Biomedical developed the CARV tool, which integrates fully automatic detection algorithms for vessel abnormalities in order to assess the presence of retinal features associated to cardiovascular diseases. CARV is composed of two modules: a) Vessel network analysis, and b) Vessel abnormality detection.<br /> For the vessel network analysis, the vasculature of the retinal images is segmented and the arteries and veins are automatically classified using color information and morphological features. These processes remove a significant time burden from readers using a 2^nd reader system and enable a fully automatic 1^st reader system.<br /> The vessel abnormalities detected and measured are: tortuosity, the presence of artery-vein (AV) nicking at AV crossings, the AV ratio, arterial copper/silver wiring, and the presence of retinal emboli in arteries vessel sections. These algorithms are integrated in a graphical user interface (GUI) to assist the reader in the detection of vessel abnormalities and disease.<br /> We evaluated the impact of CARV by enrolling two newly certified readers and one optometrist to read the images without CARV and to perform a second read using CARV after a period of memory erase. A total of 120 fundus images (30% were CVD cases) were selected for this analysis and were given to each reader.

 

Results

 

We compared the sensitivity of the readers against the ground truth for each image as well as the reading time with and without CARV. Results are shown in Table 1. The software-aid resulted in a statistically significant reduction of reading time per image (30% less). Usage of CARV also improved the sensitivity by an average of 16%. We measured the inter-reader agreement using Gwet’s AC1 and observed improvement for the disease detection from 0.29 to 0.70.

 

Conclusions

 

Software-assistance using CARV demonstrated significant improvement in reader’s performance in terms of accuracy, consistency, and reading efficiency. This system can benefit tele-retinal screening for increasing grading throughput and consistency, and could also be used as a learning tool for retinal readers.  

 

Table 1: Effect of CARV on reading time and sensitivity.