Inter-Rater Reliability of Delirium Screening of Infants in the Cardiac ICU: A Prospective, Observational Study

Pediatr Crit Care Med. 2023 Mar 1;24(3):e147-e155. doi: 10.1097/PCC.0000000000003182. Epub 2023 Jan 20.

Abstract

Objectives: To determine the inter-rater reliability (IRR) of the Cornell Assessment for Pediatric Delirium (CAP-D) in infants admitted to a cardiac ICU (CVICU) and to explore the impact of younger age and mechanical ventilation on IRR.

Design: Prospective cross-sectional study of delirium screening performed by bedside CVICU nurses. We collected data from September 2020 to April 2021. We evaluated IRR with intraclass correlation coefficient (ICC) one-way random effects and Fleiss kappa for multiple raters.

Setting: Eighteen-bed academic pediatric CVICU.

Participants: Subjects: Infants 1 day to 1 year old admitted to the CVICU, stratified in two age groups (≤ 9 wk and 9 wk to < 1 yr). Exclusion criteria were patients' immediate postoperative day, State Behavioral Scale score less than or equal to -2, or at risk for hemodynamic instability with assessment. Raters: CVICU nurses working in the unit during study days.

Interventions: None.

Measurements and main results: Groups of four raters performed 91 assessments, a total of 364 CAP-D screens. Forty-five of 91 (49%) were in patients less than or equal to 9 weeks old and 43 of 91 (47%) in mechanically ventilated patients. Sixty-eight of 81 nurses (81%) participated. In infants less than or equal to 9 weeks old, ICC was 0.59 (95% CI 0.44-0.71), poor to moderate reliability, significantly lower than the ICC in infants greater than 9 weeks and 0.72 (95% CI 0.61-0.82), moderate to good reliability. In mechanically ventilated infants, ICC was 0.5 (95% CI 0.34-0.65), poor to moderate reliability, significantly lower than the ICC in nonmechanically ventilated infants and 0.69 (95% CI 0.57-0.8), moderate to good reliability. Fleiss kappa for all infants was 0.47 (95% CI 0.34-0.6), slight to fair agreement. Use of anchor points did not improve reliability.

Conclusions: In the youngest, most vulnerable infants admitted to the CVICU, further evaluation of the CAP-D tool is needed.

Publication types

  • Observational Study

MeSH terms

  • Child
  • Cross-Sectional Studies
  • Delirium* / diagnosis
  • Humans
  • Infant
  • Intensive Care Units, Pediatric
  • Prospective Studies
  • Reproducibility of Results