Analyzing clinical variables of choroidal melanocytic tumors to determine how they affect decisions made from an artificial intelligence classifier

Abstract

Purpose : The “black box” nature of many artificial intelligence (AI) models has limited their adoption in real-world ophthalmologic practices. We have developed an AI model for labeling colour fundus images as having a choroidal melanocytic tumor (CMT) or not. To understand the results of this AI model, we performed a retrospective cohort study of CMT patients to determine whether it is making any systematic errors. The purpose of this study is to determine if there are known clinical features that lead to a false-negative classification from the model. This study increases AI interpretability, which is needed to explicitly understand why the model is making systematic errors and provide a basis of improvement for development.

Methods : 388 fundus images from 194 patients with (n=194) and without (n=194) CMT were collected through routine clinical assessment and used to train an AI model. Patient demographic (sex, age, study eye), CMT characteristics (location, level of pigmentation, drusen, full visibility in fundus image), and risk factors detected via diagnostic imaging for choroidal melanoma (lesion thickness >2mm, lesion diameter >6mm, presence of subretinal fluid, presence of orange pigment, and ultrasonographic hollowness) were extracted from the clinical charts associated with each image. Logistic regression models were used to test for associations between the AI false-negatives and these characteristics.

Results : The AI model returned 150 true-positive and 44 false-negative for CMT eyes. From the multivariate logistic regression, of the CMT characteristics, decreased thickness (p = 0.02) had a significant relationship with false-negative classifications. None of the demographics or presence of imaging choroidal melanoma risk factors were shown to have any statistically significant relationship with a false-negative classification.

Conclusions : The results from this study demonstrate that the false-negatives for CMT eyes from our AI model are not associated with the presence of imaging risk factors for choroidal melanoma, but are influenced by the decreased thickness of the lesion. This helps validate the model and aids in its interpretability by analyzing which characteristics of the lesion have an effect on fundus image classification by an AI model. We will aim to use these results to improve results from AI classifications of fundus images.

This abstract was presented at the 2024 ARVO Annual Meeting, held in Seattle, WA, May 5-9, 2024.