The influence of the dynamic context of the pandemic on the predictive performance of mortality predictions over time in older patients hospitalised for COVID-19

Bas F M van Raaij; Anum Zahra; Ewout W Steyerberg; Anne A H de Hond; Rosalinde A L Smits; Veerle M G T H van der Klei; Harmke A Polinder-Bos; Julia Minnema; Brent Appelman; Annemieke Smorenberg; Stella Trompet; Geeske Peeters; Maarten van Smeden; Karel G M Moons; Jacobijn Gussekloo; Simon P Mooijaart; Raymond Noordam; COOP consortium and COVID-OLD study

doi:10.1016/j.jclinepi.2024.111652

The influence of the dynamic context of the pandemic on the predictive performance of mortality predictions over time in older patients hospitalised for COVID-19

J Clin Epidemiol. 2024 Dec 26:111652. doi: 10.1016/j.jclinepi.2024.111652. Online ahead of print.

Authors

Bas F M van Raaij¹, Anum Zahra², Ewout W Steyerberg³, Anne A H de Hond², Rosalinde A L Smits⁴, Veerle M G T H van der Klei⁴, Harmke A Polinder-Bos⁵, Julia Minnema⁵, Brent Appelman⁶, Annemieke Smorenberg⁷, Stella Trompet⁴, Geeske Peeters⁸, Maarten van Smeden², Karel G M Moons², Jacobijn Gussekloo⁹, Simon P Mooijaart⁴, Raymond Noordam¹⁰; COOP consortium and COVID-OLD study

Collaborators

COOP consortium and COVID-OLD study:
Simon P Mooijaart, Jacobijn Gussekloo, Harmke A Polinder-Bos, Karel G M Moons, Maarten van Smeden, Geeske Peeters, René J F Melis, Petra J M Elders, Jan Festen, Harmke A Polinder-Bos, Carolien M J van der Linden, Steffy W M Jansen, Hanna C Willems, Jessica M van der Bol

Affiliations

¹ Department of Internal Medicine, Section of Geriatrics and Gerontology, Leiden University Medical Center, Leiden, The Netherlands; LUMC Center for Medicine for Older People, Leiden University Medical Center, Leiden, The Netherlands. Electronic address: b.f.m.van_raaij@lumc.nl.
² Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands.
³ Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands; Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Netherlands.
⁴ Department of Internal Medicine, Section of Geriatrics and Gerontology, Leiden University Medical Center, Leiden, The Netherlands; LUMC Center for Medicine for Older People, Leiden University Medical Center, Leiden, The Netherlands.
⁵ Division of Geriatric Medicine, Department of Internal Medicine, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands.
⁶ Center for Experimental and Molecular Medicine, Amsterdam University Medical Center, Amsterdam, the Netherlands.
⁷ Department of Internal Medicine, Section of Geriatric Medicine, Amsterdam UMC, Amsterdam, the Netherlands.
⁸ Department of Internal Medicine, Radboud University Medical Center, Nijmegen, The Netherlands.
⁹ Department of Internal Medicine, Section of Geriatrics and Gerontology, Leiden University Medical Center, Leiden, The Netherlands; LUMC Center for Medicine for Older People, Leiden University Medical Center, Leiden, The Netherlands; Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, The Netherlands.
¹⁰ Department of Internal Medicine, Section of Geriatrics and Gerontology, Leiden University Medical Center, Leiden, The Netherlands.

PMID: 39732182
DOI: 10.1016/j.jclinepi.2024.111652

Abstract

Objective: During the COVID-19 pandemic, dynamic factors such as governmental policies, improved treatment and prevention options and viral mutations changed the incidence of outcomes and possibly changed the relation between predictors and outcomes. The aim of the present study was to assess whether the dynamic context of the pandemic influenced the predictive performance of mortality predictions over time in older patients hospitalised for COVID-19.

Study design and setting: The COVID-OLD study, a multicentre cohort study in the Netherlands, included COVID-19 patients aged 70 years and older hospitalised during the first (early 2020), second (late 2020), third (late 2021) or fourth wave (early 2022). We developed a prediction model for in-hospital mortality that included variables commonly collected at the emergency department with least absolute shrinkage and selection operator (LASSO) regression on patients admitted in the first pandemic wave and temporally validated this model in patients admitted in the second, third or fourth wave.

Results: In total, 3067 patients (median age 79 years, 60% men) were included. The final model included demographics, frailty and indicators of disease severity that were generally available within three hours after admission. The model differentiated between death and alive after hospitalization for COVID-19 with an AUC of 0.80 (95% CI: 0.76-0.84) in the internal validation cohort. In terms of discrimination and calibration, predictive performance of the model decreased over time with an AUC of 0.76 (0.73-0.79) and calibration slope of 0.81 (0.68-0.96) in the second wave, an AUC of 0.77 (0.72-0.82) and calibration slope of 0.85 (0.65-1.10) in the third wave and an AUC of 0.59 (0.48-0.70) and calibration slope of 0.35 (-0.05, 0.72) in the fourth wave.

Conclusion: Compared to the moderate model performance in the first wave, we observed a slight decrease in terms of discrimination and calibration in the second and third wave with a much larger decrease in the fourth wave. This highlights the importance of ongoing data collection, monitoring of model performance and model updates during a pandemic.

Keywords: COVID-19; prediction model; temporal validation.