Effects of double data extraction on errors in evidence synthesis: a crossover, multicenter, investigator-blinded, randomized controlled trial

Lijun Tang; Ruoxi Wang; Suhail A R Doi; Luis Furuya-Kanamori; Lifeng Lin; Zongshi Qin; Fangbiao Tao; Chang Xu

doi:10.1093/postmj/qgae195

Effects of double data extraction on errors in evidence synthesis: a crossover, multicenter, investigator-blinded, randomized controlled trial

Postgrad Med J. 2025 Jan 7:qgae195. doi: 10.1093/postmj/qgae195. Online ahead of print.

Authors

Lijun Tang^{1

2

3}, Ruoxi Wang^{1

2

3}, Suhail A R Doi⁴, Luis Furuya-Kanamori⁵, Lifeng Lin⁶, Zongshi Qin⁷, Fangbiao Tao^{1

2

3}, Chang Xu⁸

Affiliations

¹ MOE Key Laboratory of Population Health Across Life Cycle, Anhui Medical University, No. 81 of Meishan Road, Hefei, Anhui, 230032, China.
² Anhui Provincial Key Laboratory of Population Health and Aristogenics, Anhui Medical University, No. 81 of Meishan Road, Anhui, 230032, China.
³ Department of Maternal, Child and Adolescent Health, School of Public Health, Anhui Medical University, No. 81 of Meishan Road, Hefei, Anhui, 230032, China.
⁴ Department of Population Medicine, College of Medicine, QU Health, Qatar University, Al Jamiaa Street, P. O. Box 2713, Doha, Qatar.
⁵ Faculty of Medicine, UQ Centre for Clinical Research, University of Queensland, 11 Wyndham Street, Herston Queensland, Brisbane, 4006, Australia.
⁶ Department of Epidemiology and Biostatistics, University of Arizona, 235 E 42nd St, Tucson, AZ, New York, NY 10017, United States.
⁷ Peking University Clinical Research Center, Peking University, 49 Huayuan NRd, Beijing, 100191, China.
⁸ Proof of Concept Center, Eastern Hepatobiliary Surgery Hospital, Third Affiliated Hospital, Second Military Medical University, Naval Medical University, No. 255, Yangpu District, Shanghai, 200433, China.

PMID: 39764599
DOI: 10.1093/postmj/qgae195

Abstract

Objectives: The objective was to investigate the role of double extraction in reducing data errors in evidence synthesis for pharmaceutical and non-pharmaceutical interventions.

Design: Crossover randomized controlled trial (RCT).

Setting: University and hospital with teaching programs in evidence-based medicine.

Participants: One hundred eligible participants were randomly assigned in a 1:1 ratio to perform data extraction tasks for either 10 RCTs of pharmaceutical interventions or 10 RCTs of non-pharmaceutical interventions, followed by a crossover pattern and a further cross-checking process (double extraction). Only data on binary adverse outcomes were extracted.

Intervention: Double data extraction versus single extraction.

Primary and secondary outcome measures: The primary outcome was the error rate before and after the cross-checking process. The secondary outcome was the absolute difference in error rates. Error rates were assessed at both the study level and the cell level.

Results: Error rates in the pharmaceutical and non-pharmaceutical groups were 64.65% and 59.90%, respectively, with an absolute difference of 4.75% and an odds ratio (OR) of 1.29 [95% confidence interval (CI): 1.06-1.57, P = .01] when measured at the study level. After cross-checking, error rates decreased to 44.88% and 39.54%, with the difference between the two groups remaining at 5.34%, and an OR of 1.27 (95%CI: 1.1-1.46; P < .01). Similar differences were observed when measured at the cell level.

Conclusion: Although double extraction reduced data errors, the error rate remained high after the process. Evidence synthesis research may consider triple data extraction to further minimize potential data errors.

Trial registration number: Chinese Clinical Trial Registry Center (Identifier: ChiCTR2200062206).

Keywords: cross-checking; data extraction errors; evidence synthesis; randomized controlled trial.

Abstract

Grants and funding