Abstract
Purpose :
Flavoprotein fluorescence (FPF) imaging has emerged as a technology to better understand retinal metabolism. There is no disseminated, standardized method of grading these images. The purpose of this study was to apply a novel Likert-based grading scale to an existing image database to test its ability to select for high quality images, reduce variability in scores, and improve intergrader reliability.
Methods :
3664 images and their associated flavoprotein fluorescence (FPF) and heterogeneity score (CW) were automatically generated by Ocumet ® Image Analysis software at the time of patient presentation. The images had been analyzed by a combination of graders on a three point Likert scale without a rubric. Three independent, blinded graders were trained on appropriate use of a novel, five point novel grading rubric. A database was populated with the images and graders completed the images. Intergrader reliability was assessed by Fleiss Kappa metric.
Results :
Fleiss Kappa metric for the novel grading scale was 0.584, indicating moderate agreement. Of the mismatches occurring in the old grading scale roughly 33.3% were between adequate/inadequate images (highest/lowest category). Of all mismatches occurring in the new grading scale only 11.7% occurred between Grade A and B images and Grade C,D, F images (highest/lowest categories). With only 2.4% occurring in Grade A images. When applied to Diabetes Mellitus and Age-related Macular Degeneration the new model reduced the variability in FPF and CW images classified as grade A or B. The greatest variance in FPF and CW occurred in images classified as Grade D or F. The total number of usable images decreased from 62.3% to 11.7%.
Conclusions :
The Likert-based grading scale enhanced selection of high-quality images, reduced score variability, and improved inter-grader reliability. It has promise for future applications in similar settings.
This is a 2021 ARVO Annual Meeting abstract.