Student becomes teacher: faster deep learning (DL) lightweight models (LWM) for automated detection of abnormal OCT B-scans using student-teacher framework

Abstract

Purpose :
Significant barriers to OCT automated diagnosis include need for expert-labeled training data and long computing times required by state-of-the-art algorithms. We explore a student-teacher framework for training LWMs with fewer parameters leveraging unlabeled images to perform fast automated detection of abnormal B-scans.

Methods :
The dataset consisted of 76,396 expertly labeled B-scans from 598 patients (45,698 normal and 30,698 abnormal) and 478,588 unlabeled B-scans from 3,148 patients, acquired using a CIRRUS™ HD-OCT 5000 (ZEISS, Dublin, CA). B-scans were labeled abnormal if intraretinal/subretinal fluid; disruption of inner retinal layers, IS/OS, or vitreoretinal interface; or RPE atrophy/elevation were present. A ResNet50 “teacher” model and 27 “student” DL networks from 4 LWM families (SqueezeNet, SqueezeNext, MobileNet, and ShuffleNet) were trained to identify abnormal B-scans. The labeled dataset was split by patients into 80% training and 20% validation. Average inference time on a NVIDIA P100 and best validation accuracy over epochs were reported. The unlabeled B-scans were artificially labeled using the teacher network (Figure 2A) and then combined with the labeled B-scans to retrain the LWMs from random initialization.

Results : Figure 1 presents the accuracy versus inference-time tradeoff for all models. ResNet50 achieves 96.1% validation accuracy and the LWM range from 83.2% to 95.0%. The best performing LWM, a SqueezeNet model with residual connections (SRN.1), is 4.13 times faster than ResNet50 (0.109s vs. 0.452s). Figure 2 shows student-teacher training results, revealing that all models benefit from increasing training set by including unlabeled B-scans. SRN.1 again obtains the highest validation accuracy (96.3%), narrowly exceeding the teacher network.

Conclusions : We demonstrate the effectiveness of a student-teacher framework for training fast LWMs for automated abnormal B-scan detection leveraging unlabeled, routinely-available data.

This is a 2021 ARVO Annual Meeting abstract.

 

Average inference times for an OCT cube versus baseline validation accuracy over training epochs for student and teacher models using the labeled data set.

 

A) Student-teacher framework B) Average inference time for an OCT cube versus best validation accuracy over training epochs for LWMs using a student-teacher framework with unlabeled data. Dotted line shows ResNet50 validation accuracy.