The fragility of statistically significant findings in randomised controlled anaesthesiology trials: systematic review of the medical literature

Abstract

The fragility index (FI), the number of events the statistical significance a result depends on, and the number of patients lost to follow-up are important parameters for interpreting randomised clinical trial results. We evaluated these two parameters in randomised controlled trials in anaesthesiology. For this, we performed a systematic search of the medical literature, seeking articles reporting on anaesthesiology trials with a statistically significant difference in the primary outcome and published in the top five general medicine journals, or the top 15 anaesthesiology journals. We restricted the analysis to trials reporting clinically important primary outcome measures. The search identified 139 articles, 35 published in general medicine journals and 104 in anaesthesiology journals. The median (inter-quartile range) sample size was 150 (70-300) patients. The FI was 4 (2-17) and 3 (2-7), and the number of patients lost to follow-up was 0 (0-18) and 0 (0-6) patients in trials published in general medicine and anaesthesiology journals, respectively. The number of patients lost to follow-up exceeded the FI in 41 and 27% in trials in general medicine journals and anaesthesiology journals, respectively. The FI positively correlated with sample size and number of primary outcome events, and negatively correlated with the reported P-values. The results of this systematic review suggest that statistically significant differences in randomised controlled anaesthesiology trials are regularly fragile, implying that the primary outcome status of patients lost to follow-up could possibly have changed the reported effect.

Keywords: anaesthesiology; lost to follow-up; randomised controlled trials; research methodology; statistical significance.