- Research article
- Open Access
Development and validation of a predictive score for ICU delirium in critically ill patients
BMC Anesthesiology volume 21, Article number: 37 (2021)
The incidence of delirium in intensive care unit (ICU) patients is high and associated with a poor prognosis. We validated the risk factors of delirium to identify relevant early and predictive clinical indicators and developed an optimized model.
In the derivation cohort, 223 patients were assigned to two groups (with or without delirium) based on the CAM-ICU results. Multivariate logistic regression analysis was conducted to identify independent risk predictors, and the accuracy of the predictors was then validated in a prospective cohort of 81 patients.
A total of 304 patients were included: 223 in the derivation group and 81 in the validation group, 64(21.1%)developed delirium. The model consisted of six predictors assessed at ICU admission: history of hypertension (RR = 4.367; P = 0.020), hypoxaemia (RR = 3.382; P = 0.018), use of benzodiazepines (RR = 5.503; P = 0.013), deep sedation (RR = 3.339; P = 0.048), sepsis (RR = 3.480; P = 0.018) and mechanical ventilation (RR = 3.547; P = 0.037). The mathematical model predicted ICU delirium with an accuracy of 0.862 (P < 0.001) in the derivation cohort and 0.739 (P < 0.001) in the validation cohort. No significant difference was found between the predicted and observed cases of ICU delirium in the validation cohort (P > 0.05).
Patients’ risk of delirium can be predicted at admission using the early prediction score, allowing the implementation of early preventive interventions aimed to reduce the incidence and severity of ICU delirium.
Delirium is a disturbance of consciousness characterized by an acute onset and a fluctuating course of impaired cognitive functioning. ICU delirium is a vital issue as its incidence rang from about 20 to 80% and highly depends on the population studied and the diagnostic method used [1, 2]. Some studies showed that the incidence of postoperative delirium is approximately 31–50% ; furthermore, the higher incidence rate is found in the elderly patients and ICU patients undergoing mechanical ventilation . Delirium are prone to prolonged hospital stay and mechanical ventilation, increased amount of sedative drugs and costs [2, 5].
The delirium risk factors reported by the 2018 PADIS (Pain Agitation Delirium Immobility Sleep) guidelines include pre-existing dementia, history of hypertension or alcoholism, serious illness at admission, coma, and benzodiazepine use . Bart Van Rompaey et al.  conducted a multicentre study involving 523 patients, found that neither age, age over 65 years, nor gender showed a relation to the onset of delirium. In addition, a history of drinking with more than three units per day was related to the development of delirium in ICU patients, which was in line with the finding by Spies et al.  that drinking predisposed individuals to perceptual disturbances. Although delirium monitoring is advocated in numerous evidence-based guidelines as part of routine clinical care, it is still not widely and consistently performed at the bedside in different patient care settings. Currently, several scoring systems have been proposed, such as the PRE-DELIRIC model, which was developed in 2012 by van den Boogaard et al.  and based on 10 readily available risk factors. However, the usefulness of the model is limited by the fact that it requires predictors obtained within 24 h of ICU admission. Another study  showed that patients who suffered from nervous system disease had a higher incidence of delirium by predicting delirium in critically ill patients using 11 related factors, and determined that prophylaxis with Dexmedetomidine Hydrochloride in delirium ICU patients was beneficial. Moreover, the PREDICt model is designed to detect POD in surgical ICU patients, but it’s difficult to exactly available for the general intensive care patients . Correlational research  suggested that screening could increase the rate of its diagnosis to 64%, and the early essential intervention could reduce both the incidence of delirium and its duration and complications. This highlights the importance of evaluating the performance of such models which based on identified risk factors for delirium in ICU patients.
Accordingly, the aim of this study was to develop and validate a simple delirium predictive model. We further evaluate the predictive value of the model regarding the development of delirium.
Design and sample
Prospective observational single study carried out in mixed intensive care units for adult patients. The study received institutional approval by Research Ethical Committees, and informed consent of the patients’ families was required. All consecutive patients who were aged ≥18 years, and admitted to stay in ICU for more than one day between September 2015 and September 2016 were included in the study. The assessment of mental illness was based on the American Psychiatric Diagnostic and Classification Manual, Fourth Edition, and cranial CT examination was used to determine organic encephalopathy. The exclusion criteria were  neurological disease or organic encephalopathy ; a history of dementia, depression or schizophrenia ; a history of preoperative cognitive impairment ; a sustained coma during complete ICU stay ; death at the time of screening.
Possible risk factors related delirium avaliable after patients were enrolled in the study were: age, APACHE-II scores (Acute physiology and chronic health evaluation) in the 24 h after ICU admission, mechanical ventilation, metabolic acidosis, history of hypertension, history of diabetes, history of heart disease, history of peptic ulcer toxaemia, history of pulmonary dysfunction, the application of Benzodiazepines, alcohol abuse, nicotine, acidosis, the types of surgery and the duration, arterial blood gas analysis after the outcome -delirium, sepsis, hypoxaemia (see Table 1 data supplement for exact definitions).
Several tools are available to assess delirium in intensive care patients, of which the CAM-ICU has the highest sensitivity and specificity , the inter-rate reliability of the delirium screenings by the intensive care nurses was 0.92–0.96 . All consecutive patients were screened by well-trained ICU nurses twice a day (at 9:00 AM-11:00 AM and 03:00 PM-05:00 PM) using the CAM-ICU. We determined a priori that an inter-rater reliability of 0.85 Cohen’s kappa indicated reliable data. Each patients were assessed for delirium lasting five days or until discharge from ICU. We defined patients as having delirium when they had at least one positive CAM-ICU screening during their intensive care stay.
With an anticipated delirium incidence of 20 to 40% and an attrition rate of 10%, we aimed to enroll 1.962*0.4*0.6/(0.15*0.4)2 /0.9 = 286 patients. The first aim was to develop the model, and the second was to validate the model.
Cohen’s Kappa test was used to detect the inter-rater reliability of delirium screening by trained intensive care nurses. Measurement data were tested for a normal distribution using the homogeneity of variance test and Kolmogorov-Smirnov Test. Continuous variables with a normal distribution were presented as the mean ± standard deviation (SD). Analysis was carried out using Student’s t-test. Categorical variables were recorded, percentages were calculated, and the χ2 test was used for these analyses. Fisher’s exact was used to test for T < 1 or n < 40. Each score’s discriminatory power was assessed by calculating the area under the receiver operating characteristic (ROC) curve (AUROC). The Hosmer-Lemeshow test was used to determine whether there were differences between the predicted and observed cases of ICU delirium in the validation cohort. Comparisons in which P < 0.05 were considered statistically significant.
Development of prediction model
A total of 445 consecutive patients were screened,141 of whom were excluded (Fig. 1). The most frequent reasons for exclusion were followed by a length of stay in the ICU < 24 h 50, mental nervous system disease 43, sustained coma 25, and died at time of screening 23. The cohort consisted of 304 patients, 223 of whom were allocated to the derivation cohort (September 2015 to January 2016) while the remaining 81 patients were allocated to the validation cohort (February 2016 to April 2016). We recorded each patients’s data in forms (Table 1-3), and we used data from first 223 critically ill patients to construct the predictive model, which consist of 6 risk factors (Table 4).
Validation of prediction model
We performed a univariate analysis using 16 variables from Table 1 to identify factors associated with ICU delirium: history of hypertension, history of diabetes, hypoxaemia, hypotension, deep sedation, benzodiazepines, metabolic acidosis, sepsis (Table 2).
Multivariate regression analysis demonstrated that history of hypertension, hypoxaemia, use of benzodiazepines, deep sedation, need for mechanical ventilation and infection were independent risk factors for delirium in critically ill patients (Table 4). The developed logistic regression model was based on these 6 independent predictors: logit(P) = − 3.563 + 1.474 * for history of hypertension + 1.219 * for hypoxaemia + 1.505 * for use of benzodiazepines + 1.206 * for deep sedation + 1.266 * for need for mechanical ventilation + 1.247 * for sepsis. The closer P was to one, the greater the likelihood of delirium, and the closer P was to zero, the smaller the likelihood of delirium.
The six risk factors of the β coefficient were divided by the smallest β coefficient in Table 5, multiplied by 2, and rounded to the nearest integer to obtain the weighted score of each predictor. The AUROC of delirium in the ICU was 0.862 ± 0.03 [P < 0.001, 95% CI (0.803–0.921)], which were determined in the risk score of the derivation cohort, (Fig. 2).
The associations of risk factors with the ICU delirium incidence in the derivation cohort were positive such that the higher the risk score, the higher the incidence. The occurrence of ICU delirium was 4.42% for a score of 0–4, 27.85% for a score of 5–9 and 61.29% for a score of 10–14 (Fig. 3). The formula for the association was as follows: expected incidence of delirium = − 0.089 + 0.059 * risk score, which was calculated as the risk score and the expected incidence of delirium in ICU patients in the validation cohort. The AUROC was 0.739 ± 0.06 [P = 0.002, 95% CI (0.620 ~ 0.857)] in the validation cohort (Fig. 4).
According to the predictive model, a Hosmer-Lemeshow test was used to confirm the formula for the risk score. The findings confirmed that there were no significant differences between the predicted and observed cases of delirium in the validation cohort (χ2 = 0.000, P > 0.05) (Table 5).
We found a 21.1% incidence of delirium in critically ill patients, which is consistent with Von Rueden’s finding  of a 24% incidence of delirium after trauma but is lower than the rate after liver transplantation reported by Beckmann . The APACHE II score system is used to evaluate the severity and prognosis of critically ill patients. Our study showed that the average APACHE II score was 15, which indicated a relatively mild state and lower incidence than the score of 24 reported by Pandharipande  and the score of 29 reported by Peterson . In addition, another reason for the lower incidence of delirium in our study was our heterogenous population, which included surgical and medical patients. The reason for the high prevalence of delirium remains poorly understood; however, recent studies have suggested that cognitive disorders and simultaneously slow brain wave activity are caused by extensive brain oxidative disturbances. Numerous studies have demonstrated that the occurrence of delirium increased significantly when patients had severe sepsis [19, 20]. Our review also showed that sepsis was independently associated with the development of delirium. Nerve neurons are particularly vulnerable to hypoxia and are susceptible to hypoxic encephalopathy. In turn, Hypoxia may further destroy nerve cells, causing repeated or acute changes in mental states. Therefore, oxygenation plays a vital role in critically ill patients, and consequently, a balance between oxygen supply and demand may reduce the incidence of delirium . In addition, hypertension is often associated with neurological damage, such as memory decline and cognitive impairment . Chronic hypertension could result in white matter damage and may contribute to dementia . Therefore, patients in the ICU with hypertension may have increased risks of hypoxia, hypoperfusion of neurons, and than increasing the occurence of delirium.
In accordance with previous studies [23, 24], the use of benzodiazepine was reported to be an independent risk factor for the development of delirium in critically ill patients. Benzodiazepine can block the γ-aminobutyric acid pump, affecting reuptake, which thus results in increased levels of the inhibitory neurotransmitter . A meta-analysis of 1235 candidate predictors  showed that non-benzodiazepine sedation was associated with shorter hospitalization (decreased by 1.6 days), reduced mechanical ventilation (decreased by 1.9 days) and lower mortality than benzodiazepine sedation. Therefore, non-benzodiazepine sedatives, such as propofol and dexmedetomidine, are preferred for intensive care patients. Moreover, some results indicated that the generation level of sedation may influence the incidence of delirium . Kollef, M.H . and Treggiari, M . also demonstrated that compared with light sedation, deep sedation increased the number of ventilation days, length of ICU stay and occurrence of post-ICU syndrome. Interestingly, other studies revealed no obvious mental disorders with daily interruptions of sedation programmes . The discomfort and communication barriers caused by tracheal intubation, alarm sounds from the machine, confined limb movements and sleep wake cycle disorders can all cause these patients to be prone to delirium .
Our model reliably predicted the development of delirium for the entire ICU length of stay on the basis of 6 risk factors readily available within 24 h of ICU admission. These findings confirm that the model has additional value in daily practice. Several limitations of this study need to be addressed. First, in this performance study, the CAM-ICU was measured at two time points on one day. Considering the fluctuating nature of delirium, all patients were screened more often if needed. Second, delirium was classified into subtypes of hypoactive, hyperactive, and mixed delirium . The incidence of delirium was lower in this study, which may be because hypoactive delirium is not easily identified by medical personnel. This finding suggested that future clinical work should aim to further strengthen training programmes for attending nurses and physicians. Third, disease type and environmental factors were not defined as risk factors. Finally, one study showed that C-reactive protein (CRP) measured on ICU stay was associated with the development of delirium , as many aetiological factors associated with delirium will produce inflammation, which elevates CRP levels. Accordingly, we did not include CRP in the model.
In conclusion, we can now easily identify patients who have a high risk of developing delirium following ICU admission using only six predictors. This will facilitate targeted initiation of preventive measures. Therefore, the model should be used daily in intensive care practice.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Jackson P, Khan A. Delirium in critically ill patients. Crit Care Clin. 2015;31:589–603.
Kotfis K, Marra A, Ely EW. ICU delirium - a diagnostic and therapeutic challenge in the intensive care unit. Anaesthesiol Intensive Ther. 2018;50:160–7.
Card E, Pandharipande P, Tomes C, Lee C, Wood J, Nelson D, Graves A, Shintani A, Ely EW, Hughes C. Emergence from general anaesthesia and evolution of delirium signs in the post-anaesthesia care unit. Br J Anaesth. 2015;115:411–7.
Salluh JI, Soares M, Teles JM, Ceraso D, Raimondi N, Nava VS, Blasquez P, Ugarte S, Ibanez-Guzman C, Centeno JV, Laca M, Grecco G, Jimenez E, Arias-Rivera S, Duenas C, Rocha MG. Delirium epidemiology in critical care (DECCA): an international study. Crit Care. 2010;14:R210.
Shehabi Y, Riker RR, Bokesch PM, Wisemandle W, Shintani A, Ely EW. Delirium duration and mortality in lightly sedated, mechanically ventilated intensive care patients. Crit Care Med. 2010;38:2311–8.
Devlin JW, Skrobik Y, Gelinas C, Needham DM, Slooter A, Pandharipande PP, Watson PL, Weinhouse GL, Nunnally ME, Rochwerg B, Balas MC, van den Boogaard M, Bosma KJ, Brummel NE, Chanques G, Denehy L, Drouot X, Fraser GL, Harris JE, Joffe AM, Kho ME, Kress JP, Lanphere JA, Mckinley S, Neufeld KJ, Pisani MA, Payen JF, Pun BT, Puntillo KA, Riker RR, Robinson B, Shehabi Y, Szumita PM, Winkelman C, Centofanti JE, Price C, Nikayin S, Misak CJ, Flood PD, Kiedrowski K, Alhazzani W: Clinical Practice Guidelines for the Prevention and Management of Pain, Agitation/Sedation, Delirium, Immobility, and Sleep Disruption in Adult Patients in the ICU. Crit Care Med 2018; 46: e825-e873.
Van Rompaey B, Elseviers MM, Schuurmans MJ, Shortridge-Baggett LM, Truijen S, Bossaert L. Risk factors for delirium in intensive care patients: a prospective cohort study. Crit Care. 2009;13:R77.
Spies CD, Otter HE, Huske B, Sinha P, Neumann T, Rettig J, Lenzenhuber E, Kox WJ, Sellers EM. Alcohol withdrawal severity is decreased by symptom-orientated adjusted bolus therapy in the ICU. Intensive Care Med. 2003;29:2230–8.
van den Boogaard M, Pickkers P, Slooter AJ, Kuiper MA, Spronk PE, van der Voort PH, van der Hoeven JG, Donders R, van Achterberg T, Schoonhoven L. Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ. 2012;344:e420.
Chen Y, Du H, Wei BH, Chang XN, Dong CM. Development and validation of risk-stratification delirium prediction model for critically ill patients: a prospective, observational, single-center study. Medicine (Baltimore). 2017;96:e7543.
Cai S, Lv M, Latour JM, Lin Y, Pan W, Zheng J, Cheng L, Li J, Zhang Y. Incidence and risk factors of PostopeRativE delirium in intensive care unit patients: a study protocol for the PREDICt study. J Adv Nurs. 2019.
van Eijk MM, van Marum RJ, Klijn IA, de Wit N, Kesecioglu J, Slooter AJ. Comparison of delirium assessment tools in a mixed intensive care unit. Crit Care Med. 2009;37:1881–5.
Gusmao-Flores D, Salluh JI, Chalhub RA, Quarantini LC. The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies. Crit Care. 2012;16:R115.
Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, Truman B, Speroff T, Gautam S, Margolin R, Hart RP, Dittus R. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA. 2001;286:2703–10.
Von Rueden KT, Wallizer B, Thurman P, McQuillan K, Andrews T, Merenda J, Son H. Delirium in trauma patients: prevalence and predictors. Crit Care Nurse. 2017;37:40–8.
Beckmann S, Schubert M, Burkhalter H, Dutkowski P, De Geest S. Postoperative delirium after liver transplantation is associated with increased length of stay and lower survival in a prospective cohort. Prog Transplant. 2017;27:23–30.
Pandharipande P, Cotton BA, Shintani A, Thompson J, Costabile S, Truman PB, Dittus R, Ely EW. Motoric subtypes of delirium in mechanically ventilated surgical and trauma intensive care unit patients. Intensive Care Med. 2007;33:1726–31.
Peterson JF, Pun BT, Dittus RS, Thomason JW, Jackson JC, Shintani AK, Ely EW. Delirium and its motoric subtypes: a study of 614 critically ill patients. J Am Geriatr Soc. 2006;54:479–84.
Aldemir M, Ozen S, Kara IH, Sir A, Bac B. Predisposing factors for delirium in the surgical intensive care unit. Crit Care. 2001;5:265–70.
Lin SM, Liu CY, Wang CH, Lin HC, Huang CD, Huang PY, Fang YF, Shieh MH, Kuo HP. The impact of delirium on the survival of mechanically ventilated patients. Crit Care Med. 2004;32:2254–9.
Gard PR. Non-adherence to antihypertensive medication and impaired cognition: which comes first? Int J Pharm Pract. 2010;18:252–9.
Gasecki D, Kwarciany M, Nyka W, Narkiewicz K. Hypertension, brain damage and cognitive decline. Curr Hypertens Rep. 2013;15:547–58.
Pandharipande P, Shintani A, Peterson J, Pun BT, Wilkinson GR, Dittus RS, Bernard GR, Ely EW. Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104:21–6.
Marra A, Pandharipande PP, Patel MB. Intensive care unit delirium and intensive care unit-related posttraumatic stress disorder. Surg Clin North Am. 2017;97:1215–35.
Kazmierski J, Kowman M, Banach M, Pawelczyk T, Okonski P, Iwaszkiewicz A, Zaslonka J, Sobow T, Kloszewska I. Preoperative predictors of delirium after cardiac surgery: a preliminary study. Gen Hosp Psychiatry. 2006;28:536–8.
Fraser GL, Devlin JW, Worby CP, Alhazzani W, Barr J, Dasta JF, Kress JP, Davidson JE, Spencer FA. Benzodiazepine versus nonbenzodiazepine-based sedation for mechanically ventilated, critically ill adults: a systematic review and meta-analysis of randomized trials. Crit Care Med. 2013;41:S30–8.
Woien H, Balsliemke S, Stubhaug A. The incidence of delirium in Norwegian intensive care units; deep sedation makes assessment difficult. Acta Anaesthesiol Scand. 2013;57:294–302.
Kollef MH, Levy NT, Ahrens TS, Schaiff R, Prentice D, Sherman G. The use of continuous i.v. sedation is associated with prolongation of mechanical ventilation. Chest. 1998;114:541–8.
Treggiari MM, Romand JA, Yanez ND, Deem SA, Goldberg J, Hudson L, Heidegger CP, Weiss NS. Randomized trial of light versus deep sedation on mental health after critical illness. Crit Care Med. 2009;37:2527–34.
Kress JP, Pohlman AS, O'Connor MF, Hall JB. Daily interruption of sedative infusions in critically ill patients undergoing mechanical ventilation. N Engl J Med. 2000;342:1471–7.
Lat I, McMillian W, Taylor S, Janzen JM, Papadopoulos S, Korth L, Ehtisham A, Nold J, Agarwal S, Azocar R, Burke P. The impact of delirium on clinical outcomes in mechanically ventilated surgical and trauma patients. Crit Care Med. 2009;37:1898–905.
Grover S, Sharma A, Aggarwal M, Mattoo SK, Chakrabarti S, Malhotra S, Avasthi A, Kulhara P, Basu D. Comparison of symptoms of delirium across various motoric subtypes. Psychiatry Clin Neurosci. 2014;68:283–91.
Zhang Z, Pan L, Deng H, Ni H, Xu X. Prediction of delirium in critically ill patients with elevated C-reactive protein. J Crit Care. 2014;29:88–92.
Conflict of interest
All authors have no conflicts of interest regarding this paper.
This study was supported by This study was supported by the Anhui Provincial Special Project of Central Government Guiding Local Science and Technology Development of China (201907d07050001) the Science and Technology Program of Anhui Province (1604f0804043) the “Summit” Training Program of Yijishan Hospital of Wannan Medicine College (GF2019J03).
the Science Foundation for Excellent Scholars of Universities and Colleges of Anhui Province (gxbjZD19).
Ethics approval and consent to participate
I confirm that all methods were carried out in accordance with relevant guidelines and regulations in this study.
I confirm that all experimental protocols were approved by Institutional Review Board of Wannan Medical College Yijishan Hospital.
- Must include a sentence confirming that informed consent was obtained from all subjects or, if subjects are under 18, from a parent and/or legal guardian.
The informed consent from patients’ families was obtained before enrollment in the study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zhang, H., Yuan, J., Chen, Q. et al. Development and validation of a predictive score for ICU delirium in critically ill patients. BMC Anesthesiol 21, 37 (2021). https://doi.org/10.1186/s12871-021-01259-z
- Intensive care unit
- Risk factors