June 2022
Volume 63, Issue 7
Open Access
ARVO Annual Meeting Abstract  |   June 2022
Detecting Double Layer Sign (DLS) with OCT using Multi-Region Segmentation Visual Transformers (ViT)
Author Affiliations & Notes
  • Yuka Kihara
    Ophthalmology, University of Washington, Seattle, Washington, United States
  • Yingying Shi
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Mengxi Shen
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Liang Wang
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Rita Laiginhas
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Xiaoshuang Jiang
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Jeremy Liu
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Rosalyn Morin
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Giovanni Gregori
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Philip J Rosenfeld
    Ophthalmology, University of Miami Health System, Miami, Florida, United States
  • Hironobu Fujiyoshi
    Chubu University, Kasugai, Japan
  • Aaron Y Lee
    Ophthalmology, University of Washington, Seattle, Washington, United States
  • Footnotes
    Commercial Relationships   Yuka Kihara None; Yingying Shi None; Mengxi Shen None; Liang Wang None; Rita Laiginhas None; Xiaoshuang Jiang None; Jeremy Liu None; Rosalyn Morin None; Giovanni Gregori Carl Zeiss Meditec, Code F (Financial Support); Philip Rosenfeld Carl Zeiss Meditec, Code C (Consultant/Contractor), Carl Zeiss Meditec, Code F (Financial Support); Hironobu Fujiyoshi None; Aaron Lee Genentech, Verana Health, Johnson and Johnson, Gyroscope, Code C (Consultant/Contractor), US Food and Drug Administration, Code E (Employment), Santen, Carl Zeiss Meditec, Novartis, Microsoft, NVIDIA, Code F (Financial Support), Topcon, Code R (Recipient)
  • Footnotes
    Support  NIH/NEI K23EY029246, NIH/NIA U19AG066567, NIH/NIA R01AG060942, Research to Prevent Blindness Unrestricted Core Grant, Latham Vision Grant, Karalis Johnson Retina Center, Carl Zeiss Meditec, Inc. (Dublin, CA), Salah Foundation (Ft. Lauderdale, FL), National Eye Institute Center Core Grant (P30EY014801)
Investigative Ophthalmology & Visual Science June 2022, Vol.63, 470 – A0007. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Yuka Kihara, Yingying Shi, Mengxi Shen, Liang Wang, Rita Laiginhas, Xiaoshuang Jiang, Jeremy Liu, Rosalyn Morin, Giovanni Gregori, Philip J Rosenfeld, Hironobu Fujiyoshi, Aaron Y Lee; Detecting Double Layer Sign (DLS) with OCT using Multi-Region Segmentation Visual Transformers (ViT). Invest. Ophthalmol. Vis. Sci. 2022;63(7):470 – A0007.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : For the task of segmenting the double layer sign (DLS), an important feature of type 1 macular neovascularization (MNV) in age-related macular degeneration, we applied a Vision Transformer (ViT)-based model, which is now state of the art in many computer vision tasks. The ViT is convolution-free transformer architecture that can capture global interactions between elements of a scene and make better use of long-range dependencies.

Methods : Eyes were imaged using swept-source OCT angiography (SS-OCT, PLEX Elite 9000, Carl Zeiss Meditec, Dublin, CA) 6x6mm scans. The scans consisted of 500 A-scans per B-scan; each B-scan repeated twice at each of 500 B-scan positions along the y-axis. The SS-OCTA structural B-scans were manually annotated for the presence of a DLS and drusen (Dr) and used for training. We built a multi-region segmentation ViT that labelled both DLSs and Dr on a single B-scan image. In order to extend ViT from image classification to semantic segmentation, we depended on the output embeddings corresponding to image patches and obtained class labels from these embeddings with a pointwise linear decoder. For comparison, a convolutional (CNN) model was trained on the same dataset.

Results : A total of 251 eyes (211 patients) were included; 188 eyes with DLS and 63 eyes with drusen only (Dr) as controls. Our ViT model had 12 layers, 768 token sizes, and 12 heads. Mean Intersections over Union (IoU) between predicted and annotated masks for DLSs and Dr were 59.7%, 62.4% for the ViT model, and 44.9%, 52.8% for the CNN model, respectively. The transformer-based model significantly outperformed the CNN-based model.

Conclusions : We present a network that can detect DLS from structural B-scans alone using a purely transformer-based model and have applied it to a dataset with coarse annotations. To our knowledge, this is the first application of ViT segmentations in ophthalmic imaging.

This abstract was presented at the 2022 ARVO Annual Meeting, held in Denver, CO, May 1-4, 2022, and virtually.

 

Framework of ViT model. 512x512 images were used as input. 16x16 image patches are projected to a sequence of embeddings and then encoded with a transformer and reshaped into a segmentation map.

Framework of ViT model. 512x512 images were used as input. 16x16 image patches are projected to a sequence of embeddings and then encoded with a transformer and reshaped into a segmentation map.

 

Ground Truth(left), prediction from ViT model(middle), and prediction from CNN model(right): a) prediction is more accurate on ViT model prediction, b) Drusen was partially mislabeled as DLS in CNN model, c) small lesion was missed in CNN model, d) CNN failed in PED case, e), f) failure case examples.

Ground Truth(left), prediction from ViT model(middle), and prediction from CNN model(right): a) prediction is more accurate on ViT model prediction, b) Drusen was partially mislabeled as DLS in CNN model, c) small lesion was missed in CNN model, d) CNN failed in PED case, e), f) failure case examples.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×