Abstract
Purpose :
To compare and evaluate the utility of five online automated deep learning software to develop retinal image diagnostic classifier.
Methods :
We used five distinct public automated deep learning platforms to automatically develop corresponding deep learning models for the diagnosis of common retinal diseases. Namely, we trained deep learning models on Medicimind, Google Cloud AutoML Vision, Microsoft Custom Vision, Clarifai and Apple Create ML. The models were trained using the retinal fundus images from Messidor dataset and retinal optical coherence tomography (OCT) images (Waterloo). All the images had been previously labeled. The platforms were then used to train a deep learning classification model through the online interface without the use of coding. The AUC, precision, and recall were calculated for each platform.
Results :
First we evaluated the classification of referable diabetic retinopathy versus non-referable diabetic retinopathy in fundus images. The trained automated deep learning models reached an AUC, precision and recall, respectively: Google cloud AutoML Vision (0.880, 0.827, 0.827), Microsoft (0.628, 0.579, 0.579), Medicimind (0.826), Clarifai (0.729, 0.775, 0.747) and Apple Create ML (0.630, 0.595, 0.600).
Secondly we evaluated the classification of age-related macular degeneration (AMD), diabetic retinopathy, central serous retinopathy (CSR), macular hole and normal retina on OCT. The trained automated deep learning models reached an AUC, precision and recall, respectively: Google cloud AutoML Vision (0.988, 0.964, 0.964), Microsoft (0.945, 0.931, 0.931), Medicimind 0.938, Clarifai (0.962, 0.903, 0.678) and Apple Create ML (0.780, 0.548, 0.548).
The Medicimind only provides the AUC value.
Conclusions :
We showed that physicians with no coding experience can use automated deep learning platforms to develop algorithms. Most of the developed deep learning models used in our research demonstrated comparable performance among retinal fundus photos (Google cloud, Medicimind, Clarifai). And for OCT images classification, the automated deep learning models demonstrated comparable performance (Google cloud, Medicimind, Clarifai, and Microsoft).
This is a 2020 ARVO Annual Meeting abstract.