Abstract
Purpose :
To study the effect of the pretraining dataset on the transferred model performance. It has been established that pretraining a deep learning system (DLS) on narrow field (NF) funds images then transfer the model to widefield (WF) fundus images enhances the performance on WF fundus diagnosis for diabetic retinopathy (DR). We studied the effect of the pretraining dataset size on the transferred model.
Methods :
We collected smaller (32k) and larger (120k) NF images datasets using VISUSCOUT® 100 (ZEISS, Jena, Germany) handheld fundus camera. The images were annotated for referable DR (more than mild DR). A DLS (Resnet-50) for each dataset was developed to detect referable DR from the images. For the transfer learning, we collected WF fundus images from 361 subjects (77% normal, 23% referable DR) using tabletop CLARUSTM 500 (ZEISS, Dublin, CA). The images were annotated for referable DR. We split the WF dataset into train/validation/test (80/10/10) sets and evaluated the final model on the test set. The two NF models were fine-tuned on the same training split of WF data and compared together and with directly evaluating WF images on NF models without fine-tuning. Augmentation techniques were used for robustness. We compared accuracy, sensitivity (Sen), specificity (Spc), and area under the curve of the receiver operating characteristics curve (AUC) for evaluation of the performance. Fig. 1 overviews the performed experiments
Results :
The pretrained model on the larger NF dataset produced marginally better results on the WF dataset after transfer compared with the pretrained model on the smaller NF dataset on the considered metrics. We observed large confidence intervals for Sen and Spc because of the dataset size limitations. The best model achieved 86.5% AUC, 84.4% accuracy, 90% Sen (95%CI [71%, 100%]), and 83% Spc (95%CI [70%, 95%])
Conclusions :
Increasing the NF pretraining dataset size potentially enhances the fine-tuning on the WF dataset. However, this enhancement is limited by the size of the WF dataset. This suggests that increasing the NF pretraining dataset size improves the prediction on WF dataset up to a certain limit where collecting more WF data is required for further improvements.
This is a 2021 ARVO Annual Meeting abstract.