Objective: To assess the validity of SAPS II (new Simplified Acute Physiology Score) in a cohort of patients admitted to a large sample of Italian intensive care units (ICU).
Design and setting: The ability of the SAPS II scoring system to predict the probability of hospital mortality was assessed with calibration and discrimination measures obtained using published coefficients. A new logistic regression equation was then developed and further formal calibration and discrimination measures were estimated for the customized model.
Patients: From the 2202 consecutive patients recruited during a 1-month period in 99 ICUs, a total of 1393 patients were included in this validation study.
Results: When the parameters based on the standard model were applied, the expected probability of mortality did not fit those actually observed in the cohort (p < 0.001), although it showed satisfactory discrimination (area under the receiver operating characteristic curve = 0.80). Such lack of fit yields an overall under prediction of mortality (observed/expected ratio = 1.14) that reflects a uniform pattern across a preselected set of subgroups. Customization allowed new mortality estimates to be calculated, with satisfactory calibration (p = 0.82) and a more uniform pattern across subgroups.
Conclusions: SAPS II maintained its validity in an independent sample of patients recruited in a large network of Italian ICUs only after appropriate adaptation (first-level customization). Whether the determinants of this relatively poor performance are related to differences in unmeasured case-mix, methods of application, or quality of care delivered is a matter for discussion that cannot be solved with the data presently available. However, these findings suggest that caution is warranted before implementing the standard SAPS II scoring system parameters outside formal research projects.