Abstract
Purpose :
P-values are widely used in medical studies to indicate statistical significance but recent literature has highlighted the potential pitfalls in their interpretation. The purpose of this literature review was to determine the robustness of statistically significant dichotomous outcomes published in ophthalmology randomized controlled trials (RCTs) through the Fragility Index (FI). The FI is a novel measure calculated by iteratively changing a patient from a non-event to an event in the treatment group, keeping overall trial group numbers stable, until statistical significance is lost.
Methods :
A literature search (MEDLINE) was performed for all RCTs published in top ophthalmology journals, and ophthalmology-related RCTs published in high-impact journals in the past 10 years. Two reviewers independently screened 1811 identified articles for inclusion if they (1) were an ophthalmology-related trial, (2) had a 1:1 prospective study design, and (3) reported a statistically significant dichotomous outcome in the abstract. All relevant data, including outcome, p-value, number of patients in each group, number of events in each group, number of patients lost to follow-up, and trial characteristics were extracted. The FI of each RCT was calculated and correlations between FI and trial parameters were determined. Lower FI values represent more fragile results.
Results :
The 156 included articles had a median total sample size of 89 patients/eyes (range: 13-2593), and a median total number of events of 27.5 (range: 4-2217). The median Fragility Index of the included trials was 2 (range: 0-48), which means that if 2 non-events were switched to events in the treatment group, the result would lose its statistical significance. A quarter of all trials had a FI of 1 or less, and 75% of trials had a FI of 6 or less. The FI was less than the number of patients/eyes lost to follow up in 52.6% of trials. A positive correlation was found between FI and total trial sample size and total number of events. FI and reported p-value for included outcomes were negatively correlated.
Conclusions :
In ophthalmology trials, statistically significant dichotomous results are often fragile, meaning that a difference of only a few events can change the statistical significance of the result. An application of the Fragility Index in RCTs may aid in the interpretation of results.
This is an abstract that was submitted for the 2017 ARVO Annual Meeting, held in Baltimore, MD, May 7-11, 2017.