June 2013
Volume 54, Issue 15
Free
ARVO Annual Meeting Abstract  |   June 2013
The Uncertainty Classifier: Use of a Novel Statistical Classifier with Fuzzy Classification Boundaries to Evaluate Performance of Automated Electrooculogram (EOG) testing
Author Affiliations & Notes
  • Matthew Lee
    Faculty of Medicine, Nursing and Health Sciences, Monash University, Clayton, VIC, Australia
  • Marc Sarossy
    Ocular Diagnostic Clinic, Royal Victorian Eye and Ear Hospital, East Melbourne, VIC, Australia
  • Maria Triglia
    Victorian Institute of Forensic Mental Health, Clifton Hill, VIC, Australia
  • Footnotes
    Commercial Relationships Matthew Lee, None; Marc Sarossy, None; Maria Triglia, None
  • Footnotes
    Support None
Investigative Ophthalmology & Visual Science June 2013, Vol.54, 6138. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Matthew Lee, Marc Sarossy, Maria Triglia; The Uncertainty Classifier: Use of a Novel Statistical Classifier with Fuzzy Classification Boundaries to Evaluate Performance of Automated Electrooculogram (EOG) testing. Invest. Ophthalmol. Vis. Sci. 2013;54(15):6138.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: Traditional analyses of classifier performance used to compare clinical test methods such as specificity, sensitivity or ROC curves are often inadequate because the sharp classification boundary does not reflect the smooth gradient of certainty in a zone between possible classifier outcomes. In this study we develop a novel approach to analysing test performance by including a clinically determined loss function to allow the boundary between test outcomes to become fuzzy.

Methods: An aggregate loss function (a monotonic sigmoidal curve), the “loss function”, was developed by the authors in conjunction with other electrophysiologists for the significance of the Arden Ratio (AR) measurement of the EOG. 54 eyes of 27 patients were analysed with an automated method of EOG calculation previously described and compared to results from traditional manual calculation. In brief, an approximation to a square wave was automatically fitted to saccadic deflections by nonlinear least squares (nls) and robust regression (rr) techniques. The ARs (ratio of the maximum amplitude in the light divided by the minimum amplitude in the dark) determined from both methods were compared to manually measured results. The automated methods were compared to the manual method by traditional classifier measures and our Uncertainty Classifier.

Results: At an AR cutoff of 200%, sensitivities were 89% nls and 96% rr, while specificities were 92% nls and 92% rr. These differences were not statistically significant. The area under the ROC curve (AUC) was 0.957 nls and 0.982 rr. This difference was significant at the p=0.091 level using DeLong’s test. Correlation between the ARs determined by Pearson’s correlation coefficient was 0.945 (nls vs manual) and 0.948 (robust vs manual). This was not statistically significant. Using our uncertainty classifier, the correlation between the diagnostic confidence level was 0.927 (nls vs manual) and 0.943 (rr vs manual) which was significant at the p=0.08 level using Zou’s technique.

Conclusions: The Uncertainty classifier encapsulates the clinical weight applied to a test result especially where there is a quantifiable amount of uncertainty. This statistical technique has utility in comparing tests and may find a role in transforming inputs before they are used in machine learning systems.

Keywords: 510 electroretinography: non-clinical • 468 clinical research methodology • 459 clinical (human) or epidemiologic studies: biostatistics/epidemiology methodology  
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×