June 2023
Volume 64, Issue 8
Open Access
ARVO Annual Meeting Abstract  |   June 2023
Dry Eye Redness: Ora Auto Redness & Auto Horizontality Algorithms Evaluated Against Expert Human Graders
Author Affiliations & Notes
  • Benjamin Liu
    Research & Development, Ora Inc, Andover, Massachusetts, United States
  • Rachel Zilinskas
    Statistics and Data Corporation, Arizona, United States
  • Mark B Abelson
    Ora Inc, Andover, Massachusetts, United States
  • Footnotes
    Commercial Relationships   Benjamin Liu Ora, Code C (Consultant/Contractor); Rachel Zilinskas Statistics & Data Corporation, Code E (Employment); Mark Abelson Ora Inc, Code E (Employment), Ora Inc, Code I (Personal Financial Interest)
  • Footnotes
    Support  None
Investigative Ophthalmology & Visual Science June 2023, Vol.64, 3988. doi:
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Benjamin Liu, Rachel Zilinskas, Mark B Abelson; Dry Eye Redness: Ora Auto Redness & Auto Horizontality Algorithms Evaluated Against Expert Human Graders. Invest. Ophthalmol. Vis. Sci. 2023;64(8):3988.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose : To assess the outputs from Ora’s Auto-Redness and Auto-Horizontality algorithms against the scores of expert human graders for a set of 10,008 images of human Dry Eye Disease (DED).

Methods : 10,008 images were graded by 3 experts on a scale from (0-4) using the Ora Calibra® Conjunctival Redness Scale, and run through Ora’s Redness and Horizontality algorithms, which output continuous values from (0-255) and (0-100) for Auto-Redness and Auto-Horizontality respectively.1 The outputs of the two algorithms had several transformations applied which included Normalization of the Redness score and Z-scores for both Redness and Horizontality. These transformations were used as the inputs for a linear regression model that was trained using the simple mean of the 3 expert grader scores as the target for the model.

Results : The best performing linear model had a Mean Absolute Error (MAE) of 0.416 pts on the Ora Calibra® Conjunctival Redness Scale, relative to the mean of the expert grader scores. When fitting the linear regression model, the use of Auto-Horizontality as an input, significantly improved the predictive capability of the linear model’s output. Although the linear model had an average Mean Absolute Error < 0.5 pts., when compared with the simple mean score of the 3 expert graders, 65.5% of images were predicted with a MAE <0.5, 29.8% with a MAE >=0.5 and < 1.0, and 4.7% with a MAE >=1.0.

Conclusions : The output of the linear model, or the “Automated Composite Score”, is competitive with the average of expert grader scores and has been shown to reduce the required sample size for a study by approximately 80%.2 The Automated Composite Score from the linear model appears to be a viable alternative to using human graders to assess DED redness as a clinical trial endpoint.

References:
1. Rodriguez J, Johnston P, Ousler G, Smith LM, Abelson M. Automated grading system for evaluation of ocular redness associated with dry eye. Clin Ophthalmol. 2013;7:1197-1204. https://doi.org/10.2147/OPTH.S39703
2. Zalinskas et al ARVO 2023

This abstract was presented at the 2023 ARVO Annual Meeting, held in New Orleans, LA, April 23-27, 2023.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×