Establishment and validation of a predictive model for tracheotomy in critically ill patients and analysis of the impact of different tracheotomy timing on patient prognosis

Abstract

Background

In critically ill patients receiving invasive mechanical ventilation (IMV), it is unable to determine early which patients require tracheotomy and whether early tracheotomy is beneficial.

Methods

Clinical data of patients who were first admitted to the ICU and underwent invasive ventilation for more than 24 h in the Medical Information Marketplace in Intensive Care (MIMIC)-IV database were retrospectively collected. Patients were categorized into successful extubation and tracheotomy groups according to whether they were subsequently successfully extubated or underwent tracheotomy. The patients were randomly divided into model training set and validation set in a ratio of 7:3. Constructing predictive models and evaluating and validating the models. The tracheotomized patients were divided into the early tracheotomy group (< = 7 days) and the late tracheotomy group (> 7 days), and the prognosis of the two groups was analyzed.

Results

A total of 7 key variables were screened: Glasgow coma scale (GCS) score, pneumonia, traumatic intracerebral hemorrhage, hemorrhagic stroke, left and right pupil responses to light, and parenteral nutrition. The area under the receiver operator characteristic (ROC) curve of the prediction model constructed through these seven variables was 0.897 (95% CI: 0.876–0.919), and 0.896 (95% CI: 0.866–0.926) for the training and validation sets, respectively. Patients in the early tracheotomy group had a shorter length of hospital stay, IMV duration, and sedation duration compared to the late tracheotomy group (p < 0.05), but there was no statistically significant difference in survival outcomes between the two groups.

Conclusion

The prediction model constructed and validated based on the MIMIC-IV database can accurately predict the outcome of tracheotomy in critically ill patients. Meanwhile, early tracheotomy in critically ill patients does not improve survival outcomes but has potential advantages in shortening the duration of hospitalization, IMV, and sedation.

Introduction

Invasive mechanical ventilation (IMV) is an important respiratory support technique. In the United States, 270–314 of every 100,000 people require IMV, and this number is increasing every year [1, 2]. In the intensive care unit (ICU), approximately 30–40% of patients require mechanical ventilation (IMV) [3]. Due to their condition, patients in the ICU use MV much more frequently and for a longer period than patients in the general ward. In patients requiring prolonged MV, further tracheotomy is often necessary [4]. Tracheotomy enhances patient oral care, decreases airway resistance, increases patient comfort, and decreases sedation medication use [5]. It has been suggested that early implementation of tracheotomy in mechanically ventilated patients can reduce the length of hospitalization, duration of invasive ventilation, and the incidence of ventilator-associated pneumonia [6,7,8]. Despite the many advantages of tracheotomy, its use as an invasive operation is generally limited to patients who are expected to require prolonged endotracheal intubation, given the risks and complications of the procedure. However, predicting the duration of invasive ventilation in clinical work is difficult. Moreover, for critically ill patients, prolonged endotracheal intubation not only aggravates airway injury but is more likely to induce lung infections and reduce the survival rate of patients [9].

Therefore, early identification of patients requiring tracheotomy in mechanically ventilated patients is of particular importance. This study retrospectively analyzed patients with successful extubation and tracheotomy in the Medical Information Marketplace in Intensive Care (MIMIC)-IV database to find risk factors for tracheotomy in critically ill patients. Constructing a prediction model to identify the high-risk group of tracheotomy at an early stage to guide clinical tracheotomy decision-making. Meanwhile, we analyze the prognostic differences of tracheotomy in different periods for tracheotomized patients and explore whether early tracheotomy is beneficial for critically ill patients.

Materials and methods

Research source

The data utilized in this research was sourced from the Medical Information Marketplace in Intensive Care IV database (MIMIC-IV, version 2.2) [10]. The MIMIC-IV database is made available largely through the work of researchers at the Massachusetts Institute of Technology Laboratory for Computational Physiology and collaborating research groups. The database contains information on more than 50,000 patients admitted to the emergency department or ICU within Beth Israel Deaconess Medical Center from 2008 to 2019. The database provides case information on patients throughout their hospitalization, including demographic data, vital signs, laboratory results, surgeries, medications, monitoring records, imaging reports, etc. All the patients in this database have been de-identified, so there are no ethical issues. Access to the database by researcher Xinghua Chen was granted following approval by the review boards of both the Massachusetts Institute of Technology and Beth Israel Deaconess Medical Center (Certificate No. 58951192).

Subjects of the study

We collected clinical data from the MIMIC-IV database on patients who were admitted to the ICU for the first time and received more than 24 h of IMV after ICU admission. Patients were categorized into the successful extubation group and tracheotomy group according to whether they were subsequently successfully extubated or underwent tracheotomy. Successful extubation group was defined as extubation of the tracheal tube as planned and not reintubated during subsequent treatment, and survival within 7 days of extubation [11]. The tracheotomy group was defined as unsuccessful extubation of the tracheal tube with subsequent percutaneous tracheotomy or open tracheotomy. Exclusion of patients with tracheotomy before admission. Patients were randomized in a 7:3 ratio into training and validation sets. It is noted that only patients older than 18 years of age were included in the MIMIC-IV database. In addition, tracheotomized patients were categorized into an early tracheotomy group (< = 7 days) and a late tracheotomy group (> 7 days) based on the interval between the start of IMV and the implementation of tracheotomy. The flow chart of the study is shown in Fig. 1.

Fig. 1

Flow chart

Information collection

Retrospective collection of clinical data on critically ill patients. Including: (1) Demographic data: age, sex, body mass index (BMI); (2) Comorbidities: hypertension, diabetes, coronary artery disease (CAD), congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), traumatic intracerebral hemorrhage, hemorrhagic stroke, ischemic stroke, epilepsy, pneumonia, sepsis; (3) Various scores within 24 h of admission to the ICU: Acute Physiology Score-III (APS-III), Oxford Acute Severity of Illness Score (OASIS), Sequential Organ Failure Assessment (SOFA) score, Glasgow coma scale (GCS) score; (4) Laboratory tests within 24 h of ICU admission: red blood cell (RBC) count, white blood cell (WBC) count, platelet (PLT) count, hemoglobin (Hb), serum creatinine (SCr); (5) Vital signs within 24 h of ICU admission: temperature (T), heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), respiratory rate (RR), arterial oxygen saturation (SaO₂), left pupil reaction to light, right pupil reaction to light; (6) Blood gas analysis within 24 h before extubation or tracheotomy: pH, lactic acids, partial pressure of oxygen (PO₂), partial pressure of carbon dioxide (PCO₂), oxygenation index (OI); (7) Treatment received during IMV: vasoactive drug therapy (VAT), continuous renal replacement therapy (CRRT); parenteral nutrition (PN).

In-hospital mortality, 90-day mortality, 1-year mortality, length of hospitalization, length of ICU stay, duration of IMV, and duration of sedative medication use were collected from tracheotomized patients. Patients ' comorbidity information was screened according to the international statistical classification of diseases and related health problems (ICD-9/10) in the patient's discharge diagnosis. All variables were missing within 20% of each other.

Statistical analysis

Structured Query Language (SQL) and PostgreSQL tools were used to extract the medical records from the database, and STATA 15.1 and R 4.3.1 software were used to organize and analyze the data. Variables with less than 20% missing were filled by multiple interpolations using the MICE package of R software. Data were tested for normality using the Shapiro–Wilk test. Continuous variables following a normal distribution were presented as mean (standard deviation) and compared using the t-test. Non-normally distributed continuous variables were expressed as median (interquartile range) and compared using the Mann–Whitney U test. Categorical variables were presented as frequencies (proportions), with inter-group comparisons conducted using a chi-square test. Univariate logistic regression was used to find potential risk factors for tracheotomy in critically ill patients. Variables with p-values less than 0.2 in univariate logistic regression analyses were Lasso regression analyses and cross-validated to screen for variables with the most predictive value. Screened variables were included in a multifactorial logistic regression analysis to identify independent risk factors and construct predictive models. The predictive model was visualized by constructing a nomogram using R software, and dynamic web page nomograms were produced for clinical use. The Hosmer–Lemeshow test was used to evaluate the goodness of fit of the predictive model. The accuracy of the model in predicting tracheotomy outcomes was analyzed by plotting the receiver operator characteristic (ROC) curve and comparing the area under the ROC curve (AUC). Evaluating model accuracy by plotting calibration curves based on Bootstrap 1000 times self-service resampling. Decision curve analysis (DCA) was used to assess the clinical validity of the predictive model.

For patients with early and late tracheotomy, propensity score matching (PSM) was used to balance the difference between groups. A 1:1 nearest-neighbor matching technique was utilized to pair individuals with similar characteristics, with a caliper value of 0.1. The effectiveness of PSM was assessed by standardized mean difference (SMD), with SMD ≤  ± 0.1 indicating that the difference between the two groups was essentially equalized. We also performed a Kaplan–Meier analysis to assess 90-day and 1-year survival in patients with different timing of tracheotomy and assessed differences between groups by log-rank test. All tests were two-sided, and p < 0.05 was considered a statistically significant difference.

Result

Comparison of baseline information for the training set

A total of 5357 patients were enrolled in the study and randomized in a 7:3 ratio into a training set (n = 3749) and a validation set (n = 1608). A total of 3749 patients were enrolled in the training set, of whom 3475 were successfully extubated and 274 were tracheotomized. The validation set had a total of 1608 patients enrolled in the study, of which 1483 were successfully extubated and 125 were tracheotomized. Baseline characteristics of patients in the training set are shown in Table 1. The results showed that there were statistically significant differences between the successful extubation group and the tracheotomy group in terms of BMI, CAD, CHF, COPD, traumatic intracerebral hemorrhage, hemorrhagic stroke, pneumonia, APS-III, OASIS, GCS score, WBC count, PLT count, Hb, SCr, SBP, RR, left and right pupil responses to light, pH, PCO₂, VAT and PN (all p < 0.05). The baseline characteristics of the total population, training set, and validation set of patients are shown in Supplementary Table 1. The results showed that the differences in most variables were not statistically significant (p > 0.05), except for differences between the training and validation sets on RBC count and WBC count.

Table 1 Baseline characteristics of patients in the training set

Screening of predictors

Univariate logistic regression analysis was performed for the training group (Supplementary Table 2). To miss as few valuable variables as possible, we included variables with p less than 0.2 in the univariate logistic regression analysis in the Lasso regression analyses and cross-validated them (Fig. 2). To prevent overfitting of the model and to make the predictive model as concise as possible, we select the final candidate variables with a penalty term (λ) = λmin + 1 standard errors. Seven variables predicting tracheotomy in critically ill patients were finally screened: GCS score, pneumonia, traumatic intracerebral hemorrhage, hemorrhagic stroke, left and right pupil responses to light, and PN. These seven variables were included in a multifactorial logistic regression model, and the results showed that they were all independent risk factors for tracheotomy in critically ill patients(all p < 0.05, Table 2).

Fig. 2

Lasso regression screening variables: Figure A shows the path of the Lasso coefficients for the 26 risk factors; Figure B shows the cross-validation curves of the Lasso regression showing the course of the optimal penalty term (λ). The dashed line on the left shows λ (λmin) at the smallest deviation, and the dashed line on the right shows λmin + 1 standard error

Table 2 Multifactor logistic regression analysis of variables screened in the training set

Construction of nomogram prediction model

Based on multifactorial logistic regression analysis, GCS score, pneumonia, traumatic intracerebral hemorrhage, hemorrhagic stroke, and left and right pupil responses to light and PN, were selected to establish a prediction model for tracheotomy in critically ill patients in the present study and plotted nomogram (Fig. 3). Each important variable in the graph is assigned a weighted score from 0 to 100, and a total score is calculated by summing the scores for each risk factor in the nomogram to accurately predict the risk of tracheotomy in critically ill patients. The higher the total score, the higher the risk of tracheotomy. To facilitate clinical applications, the DynNom package of R software was used to create online dynamic nomograms (https://chentracheotomy.shinyapps.io/dynnomapp/).

Fig. 3

Nomogram: Each variable is scored from 0 to 100. A vertical line is drawn on the axis as a score according to the particular state the variable is in. The scores for each variable are summed to give a total score, based on which to assess the patient's risk of tracheotomy

Evaluation and validation of predictive models

The Hosmer–Lemeshow test results for the prediction model in the training and validation sets are χ² = 11.134 (p = 0.194 > 0.05) and χ² = 4.293 (p = 0.830 > 0.05), which demonstrate that the model has good goodness-of-fit in both datasets. The evaluation and validation results of the prediction model are shown in Fig. 4. The AUC of the training set was 0.897 (95% CI: 0.876–0.919), and the sensitivity and specificity of the prediction model were 88.6% and 77.0%, respectively, when the critical value was the maximum value of the Youden index. The AUC of the validation set was 0.896 (95% CI: 0.866–0.926), and the sensitivity and specificity of the prediction model were 87.7% and 75.2%, respectively, when the critical value was the maximum value of the Youden index. It indicates that the prediction model showed excellent prediction ability in both groups of patients. The calibration curves plotted are all close to the reference line, indicating good agreement between predicted and observed outcomes. The results of the DCA showed that in the training and validation sets when the threshold probability of tracheotomy in critically ill patients was in the range of 0.02 to 0.78 and 0.03 to 0.81, the level of net benefit of applying the prediction model was significantly higher than that of the "no-intervention" and "full intervention" program. This suggests that the prediction model has good clinical applicability.

Fig. 4

Evaluation and validation of predictive models: Figures A and D show the ROC curves for the training and validation sets, Figures B and D show the calibration curves for the training and validation sets, and Figures C and F show the DCA curves

Comparison of baseline data for early and late tracheotomy before and after PSM

Before PSM there were statistically significant differences in sex, CAD, pneumonia, sepsis, APS-III, OASIS, SOFA score, GCS score, SaO₂, PO₂, OI, VAT, CRRT, and PN between the two groups of patients with tracheotomy (all p < 0.05, Table 3). After PSM there were no statistically significant differences in baseline characteristics between the two groups of patients (all p > 0.05, Supplementary Table 3). The difference in SMD between the two groups for most of the variables after PSM was less than ± 0.1, indicating that PSM had effectively reduced the differences between groups (Supplementary Fig. 1).

Table 3 Baseline characteristics of patients in the early and late tracheotomy groups before PSM

Comparison of outcomes in patients with early and late tracheotomy

In the post-PSM cohort, the differences in in-hospital mortality (10% vs 8.8%), 90-day mortality (21.2% vs 20%), and 1-year mortality (33.8% vs 31.2%) between the early and late tracheotomy groups were not statistically significant (all p > 0.05, Table 4). The length of hospitalization (18.2 [14.1, 26.6] vs 24.6 [18.1, 33.6] days), ICU stay (12.3 [8.9, 16.1] vs 19.1 [13.4, 24.5] days), IMV (10 [6.7, 14.2] vs 16.6 [12.4, 21.9] days), and sedation (8.0 [6.0, 11.5] vs 14.0 [11.0, 18.0] days) in the early tracheotomy group were shorter than those in the late tracheotomy group, and the difference was statistically significant (all p < 0.05). However, the difference in IMV duration after tracheotomy between the two groups was not statistically significant (p > 0.05). The Kaplan–Meier curves for the two groups of patients after PSM are shown in Fig. 5. There were no statistically significant differences between groups in 90-day survival and 1-year survival rates after the log-rank test between the two groups of patients (both p > 0.05).

Table 4 Outcomes of early and late tracheotomy in the cohort after PSM

Fig. 5

Kaplan–Meier’s survival analysis of the 90-day (A) and 1-year (B)

Discussion

In this study, using a large sample of critically ill patients from the MIMIC-IV database, we successfully developed a prediction model with seven simple indicators: GCS score, pneumonia, traumatic intracerebral hemorrhage, hemorrhagic stroke, left and right pupil responses to light and PN. The predictive model showed good predictive efficacy and clinical validity in both the training and validation sets, which is a significant guide for clinicians in making tracheotomy decisions.

We found that patients with pneumonia were more likely to develop tracheotomy. A large retrospective cohort study showed that pneumonia was the most important cause of prolonged mechanical ventilation in critically ill patients [12]. Pneumonia, in addition to direct damage to the lung parenchyma caused by pathogens, also causes inflammation and edema of the lung tissue, affecting alveolar gas exchange and leading to respiratory failure [13, 14]. The study by Peñuelas et al. also found that patients with pneumonia were more likely to present with extubation failure and prolonged mechanical ventilation [15]. Similarly, the creation of an artificial airway removes the protective effect of the epiglottis and airway mucosa, making it easier for external or oral bacteria to enter the lower airways and exacerbate lung infections. Many studies have found that the longer the duration of mechanical ventilation, the higher the risk of ventilator-associated pneumonia [16,17,18].

We also found that patients with neurologic compromise were at higher risk of tracheotomy. This is mainly due to the fact that damage to the central nervous system can affect the regulation of the respiratory center, including respiratory rate and depth, airway tone, as well as normal neural reflexes, including the cough reflex, swallowing reflex, apnea, and respiratory facilitation [19]. Meanwhile, severe neurologic impairment is often accompanied by feeding disorders, so we found that patients who received PN were more likely to have a subsequent tracheotomy. In ICU patients, the long-term inability to feed themselves orally is more likely to lead to malnutrition [20]. Chronic malnutrition reduces respiratory muscle mass and compromises respiratory effectiveness [21, 22]. A study by Chuang CY et al. also noted that underweight patients were at a higher risk of extubation failure compared to normal-weight and overweight patients [23]. In addition, the adverse effects of neurologic impairment in critically ill patients are not limited to prolonged ventilation. A prospective cohort study of acute stroke patients in a neurointensive care unit found that higher National Institutes of Health Stroke Scale (NIHSS) scores on admission were associated with a worse prognosis for patients at 90 and 180 days [24]. Perhaps our predictive model has more promising applications in neurointensive care units.

In addition, our team compared the prognosis of critically ill patients who underwent tracheotomy at different times. After utilizing propensity score matching to reduce the effect of confounding factors, we found that early and late tracheotomy groups did not show statistically significant differences in in-hospital mortality, 90-day mortality, and 1-year mortality. There was a statistically significant shortening of the early tracheotomy group for the late tracheotomy group in terms of length of hospitalization, ICU stay, IMV time, and sedation time. Many previous studies have produced similar results [6, 25, 26]. This is probably related to the fact that tracheotomy increases patient comfort and reduces the need for sedation [5]. Shorter sedation times allow patients to regain voluntary respiration, swallowing, and communication functions more quickly and help patients discontinue invasive ventilation. It also allows for a higher level of consciousness so that patients can get out of bed early, which helps them recover early. In addition, sustained sedation has been associated with many side effects, including hypotension, respiratory depression, and impaired cognition [27, 28]. Prolonged mechanical ventilation and sedation are associated with an increased incidence of ICU-acquired weakness, which is more likely to be avoided by early tracheotomy [29].

There are some limitations to this study: first, we were unable to include some potentially valuable variables, such as data on respiratory mechanics and ventilator parameter settings, due to the high level of missingness of some variables; and second, in the case of patients with pneumonia, we were unable to determine whether it was community-acquired or hospital-acquired pneumonia. Finally, this study is a retrospective study and the conclusions drawn need to be tested and refined in prospective cohort studies.

Conclusions

Based on a large intensive care database, MIMIC-IV, we developed and validated a prediction model for tracheotomy in critically ill patients. Based on this prediction model, the risk of performing tracheotomy in critically ill patients can be effectively predicted at an early stage and guided to make tracheotomy decisions. However, for critically ill patients, although early tracheotomy has many advantages, such as shorter hospitalization time, invasive ventilation time, and sedation time, it does not show advantages in reducing patient mortality.