Abstract
Purpose :
The idea that clinical practice guidelines should be based on evidence generated from reliable systematic reviews (SRs) is beyond dispute. Since 2014, the American Academy of Ophthalmology (AAO) has partnered with the Cochrane Eyes and Vision United States Satellite (CEV@US) to identify reliable [Box] SRs to inform their Preferred Practice Patterns (PPPs). We examined the reliability of SRs relevant to the 2019 PPP updates on retina/vitreous conditions and characterized where the SRs are published.
Methods :
CEV@US maintains a database of SRs in eyes and vision research (last updated in July 2018), which includes full-text reports that are self-described as SRs (or meet the definition of an SR as defined by the Institute of Medicine) and assessed interventions for conditions described in one of the seven AAO Retina/Vitreous PPPs. The SRs were assessed for reliability by two research assistants (RAs) working independently, or by an RA with verification by a senior CEV@US faculty member. We tabulated journals and their published SRs by reliability.
Results :
Of the 3,777 eyes and vision SRs we identified, 330 examined the effectiveness and safety of interventions relating to a retina/vitreous conditions and could be assessed for reliability. Half of the reviews were published in 2014 or after (range: 1993 – 2018). The SRs were published in the Cochrane Library (45/330, 14%), as an agency report (23/330, 7%), or in one of 111 unique journals (262/330, 79%). We classified 136/330 (41%) SRs as “reliable” and 194/330 (59%) as “not reliable.” All SRs published by Cochrane were classified as reliable, as were more than half (57%) of SRs published as agency reports, and 30% of SRs published in other journals were reliable (Table).
Conclusions :
Trustworthy clinical practice guidelines such as the AAO PPPs require reliable SRs as evidence to support recommendations. Because less than half of published SRs in retina/vitreous were classified as reliable, it is important that peer reviewers and journal editors promote best practices in SR methods and reporting to improve the proportion of reliable SRs in eyes and vision.
We thank current and former CEV@US faculty, methodologists, and research assistants for their assistance with data acquisition, assessment, and adjudication.
This abstract was presented at the 2019 ARVO Annual Meeting, held in Vancouver, Canada, April 28 - May 2, 2019.