April 2014
Volume 55, Issue 13
Free
ARVO Annual Meeting Abstract  |   April 2014
Mechanical Turk based system for macular OCT segmentation
Author Affiliations & Notes
  • Aaron Y Lee
    Medical Retina, Moorfields Eye Hospital, London, United Kingdom
  • Adnan Tufail
    Medical Retina, Moorfields Eye Hospital, London, United Kingdom
  • Footnotes
    Commercial Relationships Aaron Lee, None; Adnan Tufail, Alcon (F), Allergan (F), Bayer (C), Heidelberg Engineering (C), Novartis (F), Oculogics, Pfizer (C), Roche (C), Thrombogenics (F)
  • Footnotes
    Support None
Investigative Ophthalmology & Visual Science April 2014, Vol.55, 4787. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Aaron Y Lee, Adnan Tufail; Mechanical Turk based system for macular OCT segmentation. Invest. Ophthalmol. Vis. Sci. 2014;55(13):4787.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract
 
Purpose
 

The current generation of automated OCT segmentation software struggles with certain disease features such as subretinal fibrosis in neovascular AMD where OCT plays a critical role in management. In addition, precise segmentation of macular features such as subretinal or intraretinal fluid volume could lead to sensitive endpoints for clinical trials. We sought to achieve highly reliable segmentation by designing a novel system for distributed OCT segmentation over a scalable, human based infrastructure and to show proof of concept results.

 
Methods
 

A total of eighteen individual slices of one OCT set of a macula taken using the Heidelberg Spectralis was distributed through Amazon Mechanical Turk. Each Human Intelligence Task reward was set to $0.01 and required the user to draw two lines to outline the ILM-retina interface and the RPE line of the retinal OCT image after being shown example images. In addition to the lines drawn, data was collected on the time spent drawing each line segment and time to completion of segmentation. Each image was submitted twice for segmentation, and inter-rater reliability was calculated. The interface was created using custom HTML5 and Javascript code and data analysis was performed using R.

 
Results
 

Representative images showing the segmentation are shown (Figure 1A). Average time to completion of both lines was 30.83 seconds (SD = 7.30 seconds). The total cost of the segmentation per macular OCT was $0.18. More than 9200 data points were collected over the 18 retinal OCT images. Pearson's correlation of inter-rater reliability was 0.995 (p < 0.0001) and coefficient of determination was 0.991. Bland-Altman plots were calculated to estimate inter-rater agreement (Figure 1B).

 
Conclusions
 

Mechanical Turk provides a cost-effective, scalable, high-availability infrastructure for manual segmentation of OCT images. The resulting images could be recombined for high resolution 3D analysis and segmentation of OCT features that are difficult for automated algorithms could be achieved using this platform.

 
 
(A) Example segmentations (green lines) of macular OCT images. Each image was segmented separately by a human Mechanical Turk worker. (B) Bland-Altman plot with mean difference (solid blue line) and 2 standard deviations (dashed blue lines).
 
(A) Example segmentations (green lines) of macular OCT images. Each image was segmented separately by a human Mechanical Turk worker. (B) Bland-Altman plot with mean difference (solid blue line) and 2 standard deviations (dashed blue lines).
 
Keywords: 550 imaging/image analysis: clinical • 585 macula/fovea • 552 imaging methods (CT, FA, ICG, MRI, OCT, RTA, SLO, ultrasound)  
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×