Background: The number of systematic reviews is increasing rapidly. Several methodologies exist for systematic reviews. Cochrane Reviews follow distinct methods to ensure they provide the most reliable and robust evidence, ideally based on rigorous evaluations of randomized controlled trials and other high-quality studies. We aimed to examine the difference in citation patterns of Cochrane Reviews and other systematic reviews.
Methods: We conducted a bibliometric analysis of systematic reviews indexed in PubMed from 1993 to 2022. We collected data on citations from The Lens from 1993 to 2023, thus having at least one-year follow-up on citations. The reviews were linked through their PubMed identifier. Comparisons between the Cochrane Reviews and other systematic reviews included total citations per review, reviews with zero citations, and the time window within which they receive citations.
Results: We included 10,086 Cochrane Reviews and 231,074 other systematic reviews. Other systematic reviews received significantly more citations than Cochrane Reviews from 1993 to 2007. From 1993 - 1997, the median difference was 80 citations [95% CI 79.6-80.4]. From 2008 and forward, the overall number of citations was similar between Cochrane Reviews and other systematic reviews (2018-2022: median difference 5 [95% CI 4.9-5.1] in favor of Cochrane Reviews; p = 0.83). Systematic reviews with zero citations were rare in both groups, but it was observed more often among other systematic reviews than Cochrane Reviews. Over the last 30 years, the time window in which all reviews received citations narrowed.
Conclusion: In recent years, Cochrane Reviews and other systematic reviews had similar citation patterns, but other systematic reviews received more citations from 1993 to 2007. Other systematic reviews were more often never cited than Cochrane Reviews, and potentially wasted. The time window in which systematic reviews received citations has been progressively decreasing, possibly indicating a trend toward quicker recognition and uptake of these reviews within the academic community. Cochrane reviews aim to provide robust evidence, but this is not reflected in the citation metrics compared to other systematic reviews.
Keywords: Bibliometrics; Citations; Cochrane systematic reviews; Evidence-based practice; Research waste.