The revised-risk analysis index as a predictor of major morbidity and mortality in older patients after abdominal surgery: a retrospective cohort study

Abstract

Background

The revised-Risk Analysis Index (RAI-rev) can accurately predict postoperative mortality risk. However, the association of RAI-rev with composite outcome of major morbidity and mortality (MMM) among older surgical patients is largely unknown. This study investigated the association between RAI-rev and postoperative MMM in older patients undergoing abdominal surgery. It also assessed the predictive value of RAI-rev combined with other preoperative risk factors.

Methods

This retrospective cohort study reviewed the medical records of all patients aged 65 and older who underwent abdominal surgery between January 2018 and December 2019. The primary outcome was the postoperative MMM during hospitalization, and its association with preoperative RAI-rev scores was assessed using multivariable logistic regression analysis. The prediction of postoperative outcomes was used the receiver-operating characteristic curve analysis.

Results

A total of 2225 older patients were analyzed, and 258 (11.6%) developed postoperative MMM. After adjusting for confounders, each unit increase in RAI-rev scores resulted in a 2.3% increase in the MMM risk and a 3.0% increase in the odds of life-threatening complications and mortality (both P < 0.05). The area under the curves (AUCs) of RAI-rev scores in predicting MMM and life-threatening complications and mortality was 0.604 (95% CI: 0.567 to 0.640) and 0.633 (95% CI: 0.592 to 0.675), respectively (both P < 0.001); when the RAI-rev was combined with age, gender, American Society of Anesthesiologists (ASA) classification, operative stress, and urgency status of surgery (emergency or elective), the AUCs were 0.694 (95% CI: 0.659 to 0.729) and 0.739 (95% CI: 0.702 to 0.777), respectively (both P < 0.001).

Conclusions

Higher RAI-rev scores were independently associated with increased risk of MMM. When combined with age, gender, ASA classification, operative stress, and urgency status of surgery, RAI-rev had improved performance in predicting the risk of MMM, particularly the life-threatening complications and mortality.

Background

With a significant increase in life expectancy, the proportion of older adults (≥65 years) has increased dramatically over the past decades [1]. This has led to an increasing number of older people undergoing surgical procedures, which presents a huge challenge for clinicians. Despite improvement in perioperative medical care and management, older patients still have higher odds of postoperative complications and mortality than younger patients [2, 3]. Therefore, it is imperative for clinicians to accurately stratify the perioperative risks among older patients and provide tailored clinical care to improve postoperative outcomes. Frailty, as a geriatric syndrome characterized by a combination of reduced physiologic reserve and multisystem deficit accumulation distinct from normal aging processes [4], has emerged as a key predictor for adverse outcomes in older surgical patients [2, 3, 5, 6].

Various screening tools have been proposed to measure frailty in perioperative settings, such as the Risk Analysis Index (RAI) developed by Hall and colleagues [7]. The RAI is an instrument based on the deficit accumulation model of frailty and comprises multiple frailty domains, including comorbidities, cognitive ability, social, nutrition, and functional status [7,8,9]. It represents a more comprehensive measure than other similar frailty instruments (e.g., modified frailty index [mFI]) [9, 10]. Initially, the validation of RAI was limited to the veteran surgical cohorts, with women in the minority [7, 11, 12]; several authors then validated it with external cohorts [9, 13, 14]. Among them, Arya et al. modified the original RAI and proposed the revised-Risk Analysis Index (RAI-rev); furthermore, they found that the RAI-rev had improved discrimination and calibration for mortality over the original RAI in perioperative settings [9]. So far, there has been limited data on the association of RAI-rev with postoperative complications and the predictive value of RAI-rev in predicting the composite outcome of major morbidity and mortality (MMM) in older surgical patients.

The current study aimed to determine the association between the RAI-rev scores and the risk of postoperative MMM in older patients undergoing abdominal surgery. Additionally, we sought to explore the predictive value of RAI-rev, as well as that of the combination of RAI-rev with other baseline risk factors (age, gender, American Society of Anesthesiologists [ASA] physical status classification, type of surgery categorized by operative stress, and urgency status of surgery [emergency or elective]), in predicting the occurrence of MMM. We hypothesized that a higher RAI-rev score was associated with an increased odds of MMM and could accurately predict the occurrence of MMM when combined with other preoperative risk factors.

Methods

The study protocol was approved by the Biomedical Research Ethics Committee of Peking University Third Hospital, Beijing, China (2022 [158–02]). Due to the retrospective design and that no patient follow-up was performed, the Biomedical Research Ethics Committee of Peking University Third Hospital agreed to waive the written informed consent from the patients. The investigators who performed the data collection were blinded to the objective of the study and received strict training sessions.

Patient selection

This retrospective cohort study reviewed the electronic medical records of older patients (≥65 years of age) who underwent abdominal surgery (including urologic and general surgical procedures) from January 2018 to December 2019, in Peking University Third Hospital. Patients with incomplete or missing perioperative data were excluded. All personal information on patients was kept confidential.

Measurement of RAI-rev score

RAI-rev score was calculated by evaluating 11 variables derived from the Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects (VASQIP/ACS-NSQIP) datasets, i.e., age, sex, cancer, poor appetite, unintentional weight loss, renal failure, congestive heart failure, shortness of breath, residence other than independent living, cognitive decline, and functional status [7,8,9]. Total score ranges from 0 to 81, with higher scores indicating more severe frailty. Details on the weight of each item are listed in Supplemental Digital Content (SDC) 1. If a patient experienced more than one surgical procedure during the hospital stay and had multiple preoperative RAI-rev scores, only the first round of the surgery and the corresponding preoperative RAI-rev score were analyzed.

Covariates

Baseline characteristics not covered by the RAI-rev were gathered, including body mass index (BMI), smoking and drinking status, major comorbidities, ASA physical status classification, and main laboratory test results. Intraoperative factors were also extracted, including type of surgery categorized by operative stress [15], urgency status of surgery (emergency or elective), anesthetic methods, duration of surgery, estimated blood loss, and intraoperative blood transfusion. The operative stress levels of surgical procedures were stratified using the Operative Stress Score (OSS), i.e., OSS1, very low stress; OSS 2, low stress; OSS 3, moderate stress; OSS 4, high stress; and OSS 5, very high stress [15].

Postoperative outcomes

The primary outcome was the occurrence of MMM during hospitalization, i.e., grade III or greater complications according to the Clavien-Dindo (CD) scoring system (SDC 2) [16]. For patients with multiple complications, we included the most severe complication for analysis. The diagnostic criteria for major complications are listed in SDC 3. The secondary outcome was the development of life-threatening complications and mortality, i.e., CD IV or greater complications.

Statistical analysis

The baseline and perioperative variables were compared between patients with MMM and those without. Continuous variables were analyzed with the independent samples t-test or Mann-Whitney U test; the Kolmogorov-Smirnov test was performed to check for normality. Categorical variables were analyzed using χ² tests, continuity-corrected χ² tests, or Fisher’s exact tests. Time-to-event outcomes in four different RAI-rev subgroups (stratified by RAI-rev scores: 20–29, 30–39, 40–49, and ≥ 50) were compared by using Kaplan–Meier curves (Log-Rank test). The hazard ratios were estimated with univariate Cox proportional hazard regression models.

Perioperative variables that might be associated with the development of MMM were screened using univariate logistic regression analyses and tested for multicollinearity. Independent variables with P values < 0.10 in univariate logistic regression analyses and those considered clinically significant were entered into a multivariable logistic regression model to identify the adjusted association of RAI-rev scores with the MMM risk. Similarly, another multivariable logistic regression model was constructed to investigate the adjusted relationship between RAI-rev scores and life-threatening complications and mortality. The 11 variables included in the RAI-rev were not separately enrolled in either univariate or multivariable analyses. The Hosmer-Lemeshow test was used to confirm the goodness of fit of the multivariable logistic regression models.

The predictive performances of RAI-rev scores alone and the combination of age, gender, RAI-rev scores, ASA classification, operative stress, and urgency status of surgery were assessed using the receiver-operating characteristic (ROC) curve analysis. The area under the curve (AUC) was used to test the discriminative power (ability to classify correctly) of these risk factors for outcomes. Differences between the AUCs were compared using the DeLong’ test. The relevant predictive parameters, including sensitivity, specificity, and positive and negative predictive values (PPV and NPV), were calculated for different thresholds of RAI-rev scores. For all analyses, two-tailed P values < 0.05 were considered significantly statistical. All statistical analyses were performed with the SPSS version 26.0 (IBM Corp., Armonk, NY, USA) and MedCalc version 19.05 (Ostend, Belgium).

Although the sample size was not estimated in advance, 258 cases of MMM and 16 independent variables included in the corresponding multivariable logistic regression model, as well as 178 cases of life-threatening complications and mortality and 15 independent variables included in the corresponding multivariable model, meet the requirement of the “ten events per variable” rule [17]. Therefore, the sample size (2225) of our study was sufficient and could guarantee the reliability and validity of the regression estimates.

Results

Patient characteristics

From January 2018 to December 2019, 4195 patients who were ≥ 65 years of age and experienced abdominal surgery were screened. Of these, 1970 patients with missing data on RAI-rev components or other baselined factors (no assessment of preoperative functional status or unintentional weight loss, ambiguous medical histories, or no necessary preoperative laboratory test results) were excluded, leaving 2225 patients for analysis (Fig. 1).

Fig. 1

Flowchart of the study

The study cohort had a mean age of 73.9 years; 61.4% (1366/2225) were men. The median RAI-rev value of our patients was 38 [IQR: 34 to 42], with most patients having RAI-rev scores between 30 and 39 (Table 1 and Fig. 2). Two hundred fifty-eight patients (11.6%) developed postoperative MMM during hospitalization, of whom 80 (3.6%), 152 (6.8%), and 26 (1.2%) experienced CD grade III, IV complications, and death, respectively (detailed in SDC 3). The median [IQR] RAI-rev score in the patients with MMM was significantly higher than that in those without MMM (41 [37 to 45] vs. 38 [34 to 42], P < 0.001). Other baselines and perioperative data are presented in Table 1 and SDC 4.

Table 1 Baseline and perioperative characteristics

Fig. 2

Distribution of the revised-Risk Analysis Index scores in the study patients

Association between RAI-rev scores and MMM

There was a significant difference in the occurrence of MMM between the four RAI-rev subgroups (Log-Rank test: P = 0.004; Fig. 3A). In the univariate Cox proportional hazard regression analysis, higher RAI-rev scores were associated with a higher rate of MMM (HR: 1.343 per 10-unit increase in scores, 95% CI: 1.141 to 1.580, P < 0.001).

Fig. 3

The occurrence of MMM (A) and life-threatening complications and mortality (B) between four different RAI-rev subgroups (stratified by RAI-rev scores: 20–29, 30–39, 40–49, and ≥ 50). Abbreviations: MMM Major morbidity and mortality

The univariate logistic regression analysis also showed that a higher RAI-rev score was associated with an increased risk of MMM, i.e., with every unit increase in the RAI-rev value, the odds of MMM increased by 5.3% (unadjusted OR: 1.053; 95% CI: 1.034 to 1.072; P < 0.001). After testing the multicollinearity, 15 other potential risk factors for MMM (P < 0.10) were identified by univariate logistic regression analyses (SDC 5 and Table 2). After correcting for the above confounding factors, the rising RAI-rev score remained an independent predictor for an increased risk of MMM, i.e., each unit increase in RAI-rev scores resulted in a 2.3% increase in the odds of MMM (adjusted OR: 1.023; 95% CI: 1.003 to 1.044; P = 0.026; Table 2).

Table 2 Predictors of postoperative MMM

Association between RAI-rev scores and life-threatening complications and mortality

A significant difference was noted in the rates of life-threatening complications and mortality between the four RAI-rev subgroups (Log-Rank test: P < 0.001; Fig. 3B). Based on the univariate Cox proportional hazard regression analysis result, higher RAI-rev scores were correlated with a higher rate of life-threatening complications and mortality (HR: 1.619 per 10-unit increase in scores, 95% CI: 1.328 to 1.974, P < 0.001).

The univariate logistic regression analysis revealed that with each unit increase in the RAI-rev score, the rate of postoperative life-threatening complications and mortality increased by 6.7% (unadjusted OR: 1.067; 95% CI: 1.044 to 1.091; P < 0.001). After testing the multicollinearity, 15 variables with P < 0.10 that were screened by univariate analyses were included in a multivariable model (see SDC 6 and Table 3). After adjustment for confounding factors, rising RAI-rev scores were independently associated with stepwise increased risk of life-threatening complications and mortality, i.e., every one unit increase in RAI-rev score predicted a 3.0% increase in the odds of this serious adverse outcome (adjusted OR: 1.030; 95% CI: 1.005 to 1.055; P = 0.017; Table 3).

Table 3 Predictors of postoperative life-threatening complications and mortality

Receiver-operating characteristic analysis for MMM

The AUC of RAI-rev scores in predicting MMM was 0.604 (95% CI: 0.567 to 0.640; P < 0.001; Fig. 4A). The sensitivity, specificity, PPV, and NPV for different threshold values of RAI-rev scores were summarised in Table 4. The AUC of the combined model (age, gender, RAI-rev scores, ASA classification, operative stress, and urgency status of surgery) was 0.694 (95% CI: 0.659 to 0.729; P < 0.001; Fig. 4A). The combined model had markedly better discrimination than the RAI-rev alone (DeLong’s test: Z = 4.794, P < 0.0001).

Fig. 4

Receiver-operating characteristic curves. A The discriminative power of RAI-rev alone (AUC: 0.604; 95% CI: 0.567 to 0.640; P < 0.001) and RAI-rev combined with age, gender, ASA classification, operative stress, and urgency status of surgery (AUC: 0.694; 95% CI: 0.659 to 0.729; P < 0.001) in predicting the MMM; the combined model had better discrimination than the RAI-rev alone (DeLong’ test: Z = 4.794, P < 0.0001). B The discriminative power of RAI-rev alone (AUC: 0.633; 95% CI: 0.592 to 0.675; P < 0.001) and RAI-rev combined with age, gender, ASA classification, operative stress, and urgency status of surgery (AUC: 0.739; 95% CI: 0.702 to 0.777; P < 0.001) in predicting the life-threatening complications and mortality; the combined model had better discrimination than the RAI-rev alone (DeLong’ test: Z = 5.028, P < 0.0001)

Table 4 Different thresholds of RAI-rev scores

Receiver-operating characteristic analysis for life-threatening complications and mortality

The AUC of RAI-rev scores in predicting life-threatening complications and mortality was 0.633 (95% CI: 0.592 to 0.675; P < 0.001; Fig. 4B). The sensitivity, specificity, PPV, and NPV for different threshold values of RAI-rev scores were detailed in Table 4. The combination of age, gender, RAI-rev scores, ASA classification, operative stress, and urgency status of surgery had improved discriminative power (AUC: 0.739; 95% CI: 0.702 to 0.777; P < 0.001; Fig. 4B) than the RAI-rev alone (DeLong’s test: Z = 5.028, P < 0.0001).

Discussion

This retrospective cohort study determined that rising RAI-rev scores were independently associated with stepwise increased risk of MMM in older patients after abdominal surgery. The AUC of the RAI-rev scores was between 0.60 and 0.65 when predicting postoperative MMM or life-threatening complications and death. Compared with the RAI-rev alone, the combination of RAI-rev scores with other baseline risk factors (i.e., age, gender, ASA classification, operative stress, and urgency status of surgery) had significantly improved predictive value for major postoperative complications. Especially for the life-threatening complications and mortality, the combined model showed a moderate predictive value with an AUC of more than 0.70, which is clinically useful in the decision-making process.

It was revealed that postoperative deaths accounted for 7.7% of all deaths worldwide, making it the third leading cause of death [18]. Undoubtedly, major postoperative complications lead to a cascade of perioperative adverse events, including death; furthermore, the occurrence of major complications is associated with poor long-term survival outcomes [19, 20]. The prediction of MMM is the critical first step for clinicians to address the burden of postoperative mortality. In the current study, postoperative MMM occurred in 11.6% of our patients. In previous studies of patients undergoing abdominal surgery, the reported incidence of CD grade III or greater complications ranged from 9.7 to 13.2% [20,21,22]; the incidence of postoperative MMM in our study population was within this range.

Like the original RAI scoring system, the RAI-rev comprises more comprehensive frailty domains than mFI. The mFI is another well-known deficit accumulation model of frailty and includes merely the domains of comorbidity and functional status [10]. Furthermore, unlike mFI, the RAI-rev is a weighted model with each item having different weights derived from a valid model [9]. When compared with the original RAI, the RAI-rev performed better discrimination and calibration in predicting postoperative mortality [9]. Although the RAI-rev scoring system offered higher weight to male sex, Arya et al. revealed that it also performed robust validity in the female population, confirming its general applicability in clinical settings [9]. Compared with previous studies [8, 9], a larger proportion of patients in our study had high RAI-rev scores. This discrepancy might be attributed to the differences in target patients and clinical settings. To our knowledge, this study is the first to investigate the association of RAI-rev scores with the postoperative MMM, as well as the predictive power of RAI-rev scores for the MMM in older surgical patients.

Our results showed that higher RAI-rev scores were independently associated with an increased risk of major complications, including life-threatening complications and death, in older patients after abdominal surgery. This finding reinforces the available evidence that preoperative frailty is an important predictor of adverse postoperative outcomes [2, 3, 5, 6]. Our results may help perioperative clinicians identify frail patients, predict the postoperative outcomes, and help patients make better informed decisions before surgery. Given the higher risk of major morbidity and mortality in the frail older population, patients and clinicians should adequately evaluate the tradeoff between survival and other potential adverse outcomes (e.g., morbidity, dependent functional status, poor quality of life after surgery) during the preoperative process of shared decision-making. Once frailty is identified, it is essential to determine whether surgical intervention can get the patients to their goals of care. For frail patients, avoiding major morbidity, loss of functional independence, and poor quality of life may sometimes make more sense than longevity. Sensible decision making may, in turn, reduce their mortality. Additionally, our findings can help guide more effective allocation of perioperative care resources and treatment to high-risk patients, thereby improving the safety and quality of surgery among the older population.

Our results demonstrated that the RAI-rev scores lacked good discrimination for the MMM or CD IV or greater complications in older patients undergoing abdominal surgery (AUCs: 0.60–0.65). Previous findings from the studies that had used the frailty tools alone to predict postoperative complications were equally disappointing [14, 23, 24]. This may be attributed to the fact that the etiology of postoperative complications is multifactorial and difficult to predict; the patient-level factors alone could not well explain the variation in complication risk. Thus, additional baseline characteristics or surgical-related factors should also be considered when predicting the risk of postoperative complications. Despite its poor discriminative ability, RAI-rev displayed high NPV at all thresholds; of course, this was also related to the low incidence of postoperative MMM. The high NPV indicated that the RAI-rev possesses a superior ability to exclude patients at low risk of major complications. For patients classified as ‘non-frail’, unnecessary medical modification or intervention (such as planned admission to the ICU after surgery) may be avoided, which can help efficiently allocate perioperative medical resources and reduce hospital costs.

As expected, the combination of RAI-rev with other commonly–used baseline factors (age, gender, ASA classification, operative stress, and urgency status of surgery) showed significantly improved performance to discriminate the risk of major complications, particularly life-threatening complications and mortality (with an AUC above 0.70). A prediction model with AUC exceeding 0.70 may be considered to be useful in clinical decision-making [25]. ASA physical status classification is a traditional preoperative risk stratification tool based on the subjective estimate, reflecting a patient’s physiologic reserve and tolerance to surgical trauma stressors. The operative stress, represented by the OSS, categorizes the surgical procedures based on different degrees of physiologic stress [15]. Given that the OSS system lacks an assessment of the urgency status of surgery, we added the latter into the combined model. Emergency surgery constitutes an important predictor of poor postoperative outcomes due to acute disease processes and inadequate medical optimization before surgery [26]. The above risk factors were all identified as credible predictors for major complications in our multivariable analyses and can be easily acquired in routine clinical practice. Despite its limited ability to predict the MMM risk, the combination of RAI-rev scores with the above risk factors might help clinicians assess the expected risk of life-threatening complications and mortality in older surgical patients. Further studies are required to explore an excellent combined model to predict major postoperative complications.

Besides the retrospective nature, this study had some other limitations. First, our study did not include gynecological patients who underwent abdominal surgery due to the concern about the influence of the special sex distribution of those patients on the RAI-rev score calculation and the final results. However, this might lead to selection bias. Second, the primary endpoint was limited to in-hospital MMM; the occurrence of postdischarge MMM was not observed, which might underestimate the rate of adverse outcomes. Finally, as single-center research, our results may not be extrapolated to patients from other institutions. Despite these, our study for the first time explored the association of RAI-rev scores with postoperative complications and the predictive value of RAI-rev for major complications.

Conclusion

In conclusion, this study demonstrated that higher RAI-rev scores were associated with an increased risk of postoperative MMM in older patients undergoing abdominal surgery. When combined with age, gender, ASA physical status classification, operative stress, and urgency status of surgery, RAI-rev had better performance in predicting postoperative MMM, particularly the life-threatening complications and mortality. Our findings enable clinicians to better identify high-risk older patients and thus optimize perioperative care and management.