Glance: A Visual Analytics Approach for Opening the Black Box to Explain Deep Learning Predictions of Glaucomatous Visual Field Damage from Optical Coherence Tomography Scans

Abstract

Purpose : To design and evaluate a visual analytics approach for explaining Deep Learning (DL) predictions of visual field (VF) mean deviation (MD) based on Optical Coherence Tomography (OCT ) retinal nerve fiber layer (RNFL) and macula scans.

Methods : We designed a tool (Glance) to provide visual explanations for a DL model that predicts 24-2 and 10-2 VF MD from RNFL and macula OCT scans. Glance’s visual interface was designed through participatory design sessions with ophthalmologists, vision researchers and OCT manufacturers. Three variations of the tool were created: 1. The TSNIT standard RNFL plot and previous actual VF MDs; 2. The TSNIT plot with a DL-derived saliency map of regions utilized by the model to make its predictions; 3. A graphical presentation of previous actual and predicted VF MDs; 4. A TSNIT plot, a saliency map and presentation of actual and predicted past VF MDs. Three clinicians completed a standardized review of 18 eyes using the 4 variations and answered questions related to trust and reliability for each visualization tool.

Results : The visualization tool helped in 4 tasks: (1) assessing reliability of a prediction, (2) understanding why the model made a prediction, (3) alerting to features that are relevant and (4) guiding future scheduling of VF. Glance was helpful for interpreting a prediction and its reliability in 20% of early-to-moderate glaucoma cases and in 4% of advanced cases. Moreover, it helped in identifying predictions that were likely to be inaccurate. Clinicians’ self-reported confidence in their management plan based on the DL predictions was about 30% higher when presented with visualizations that included past predictions and saliency maps compared to having no visual explanations. Clinicians were much less likely to have trust in the predictions if past predictions were inaccurate. The DL model predicted 24-2 VF with mean absolute error (95% CI) of 2.16 dB (2.00 - 2.97). The clinicians’ perceived utility of the tool varied by severity of disease and accuracy of past predictions.

Conclusions : Visualization approaches can assist clinicians to determine interpretability and enhance trust in DL predictions, particularly if information on the accuracy of past predictions is provided. Clinical utility of the visualizations for disease management is dependent on disease severity.

This is a 2020 ARVO Annual Meeting abstract.