Study overlap across 501 Cochrane reviews: quantifying non-independence for meta-research
Abstract
Objective. To quantify the degree of primary-study overlap across Cochrane systematic reviews and assess whether non-independence threatens the validity of methodological meta-research using these reviews.
Design. Cross-sectional bibliometric analysis.
Data source. 501 Cochrane reviews from the Pairwise70 dataset, containing 10,006 unique primary studies.
Main outcome measures. Corrected Covered Area (CCA) index, pairwise Jaccard similarity coefficients, and study-frequency distributions across all 125,250 review pairs.
Results. Overlap was minimal: CCA was 0.0001 (95% CI 0.00005–0.00018). Only 444 of 10,006 unique studies (4.4%) appeared in more than one review. Of 604 overlapping review pairs, the most overlapping pair shared 43 studies (Jaccard 0.37), while 72% of pairs shared only one study. The median number of reviews per overlapping study was 2 (IQR 2–2).
Conclusions. The Pairwise70 Cochrane review collection exhibits negligible study overlap, supporting its use as a source of largely independent meta-analyses for large-scale methodological benchmarking.
References
Page MJ, Shamseer L, Altman DG, et al. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2016;13(5):e1002028. doi:10.1371/journal.pmed.1002028. PMID: 27218655.
Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi:10.1371/journal.pmed.0020124.
Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21(11):1539–1558. doi:10.1002/sim.1186.
Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews. BMJ. 2017;358:j4008. doi:10.1136/bmj.j4008. PMID: 28935701.
Pieper D, Antoine SL, Mathes T, et al. Systematic review finds overlapping reviews were not mentioned in every other overview. J Clin Epidemiol. 2014;67(4):368–375. doi:10.1016/j.jclinepi.2013.11.007.
Published
Issue
Section
License
Copyright (c) 2026 Mahmood Ahmad
This work is licensed under a Creative Commons Attribution 4.0 International License.
Articles in Synthesis are published under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence. Authors retain copyright.
