June 2015
Volume 56, Issue 7
Free
ARVO Annual Meeting Abstract  |   June 2015
A Multi-vendor Dataset and Standardized Evaluation Framework for Retinal Cyst Segmentation
Author Affiliations & Notes
  • Christian Simader
    Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
  • Jing Wu
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
  • Ana-Maria Glodan
    Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
  • Sebastian M Waldstein
    Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
  • Bianca S. Gerendas
    Vienna Reading Center, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
  • Georg Langs
    Computational Image Analysis and Radiology Lab, Medical University of Vienna, Vienna, Austria
    Medical Imaging Cluster, Medical University of Vienna, Vienna, Austria
  • Ursula Schmidt-Erfurth
    OPTIMA Lab, Department of Ophthalmology, Medical University of Vienna, Vienna, Austria
    Medical Imaging Cluster, Medical University of Vienna, Vienna, Austria
  • Footnotes
    Commercial Relationships Christian Simader, Novartis (F); Jing Wu, Novartis (F); Ana-Maria Glodan, Novartis (F); Sebastian Waldstein, None; Bianca Gerendas, None; Georg Langs, None; Ursula Schmidt-Erfurth, None
  • Footnotes
    Support None
Investigative Ophthalmology & Visual Science June 2015, Vol.56, 5279. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Christian Simader, Jing Wu, Ana-Maria Glodan, Sebastian M Waldstein, Bianca S. Gerendas, Georg Langs, Ursula Schmidt-Erfurth, ; A Multi-vendor Dataset and Standardized Evaluation Framework for Retinal Cyst Segmentation. Invest. Ophthalmol. Vis. Sci. 2015;56(7 ):5279.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: Optical coherence tomography (OCT) has become the most frequently used imaging method for retinal diseases. In clinical practice physicians evaluate pathologic OCT morphologies, e.g. cysts, as a basis for diagnostic decisions. However, precise monitoring of disease progression and treatment success is difficult due to a lack of validated automated segmentation algorithms for these 3D OCT morphologies. We present a multi-vendor ground truth dataset for training and testing of automated cyst segmentation algorithms to be made publically available as part of a segmentation grand challenge.

Methods: The presented dataset contains 16 OCT scans from 2 major scanners (Cirrus, Spectralis), divided into 3 sets. The first set contains 8 scans (4+4) for algorithm training, annotated manually by 2 expert graders. The other 2 sets were annotated by 1 grader: 4 scans for algorithm testing and another 4 for future challenge testing. Annotations were performed in the B-scan plane using a proprietary system and stored as separate 3D ground truth data.<br /> Two accuracy measures are defined for objective comparison of algorithm results to ground truth. One examines the overlap between system results and ground truth based on the Sørensen-Dice index. The second, based on Hausdorf distance, examines the distance between the system cyst boundaries and ground truth. In addition algorithm performance is computed based on 2 criteria, clinical significance and size of cyst.

Results: Manual annotations for ground truth of the training set by 2 graders presented:<br /> A mean number of annotated cysts per scan of 147 and 549, with a mean grader difference of 8±11 and 22±32 for Spectralis & Cirrus scans respectively. Further analysis showed that disparity between cyst annotations only occurred in cases of high number of small cysts per scan.

Conclusions: First manual annotations for ground truth training data for cyst detection algorithms gave us similar results for both graders. The difference in cyst numbers between vendors directly reflects the lower number of B-scans in the Spectralis protocol. Annotations for the next 2 vendors are being completed and will double our database. The ability to objectively rank the currently expected 15 different detection algorithms shall allow a better understanding of positive and negative aspects of each method and improve performance of retinal disease progression and treatment success monitoring.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×