Predictors of the accuracy of pulse-contour cardiac index and suggestion of a calibration-index: a prospective evaluation and validation study

Background Cardiac Index (CI) is a key-parameter of hemodynamic monitoring. Indicator-dilution is considered as gold standard and can be obtained by pulmonary arterial catheter or transpulmonary thermodilution (TPTD; CItd). Furthermore, CI can be estimated by Pulse-Contour-Analysis (PCA) using arterial wave-form analysis (CIpc). Obviously, adjustment of CIpc to CItd initially improves the accuracy of CIpc. Despite uncertainty after which time accuracy of CIpc might be inappropriate, recalibration by TPTD is suggested after a maximum of 8 h. We hypothesized that accuracy of CIpc might not only depend on time to last TPTD, but also on changes of the arterial wave curve detectable by PCA itself. Therefore, we tried to prospectively characterize predictors of accuracy and precision of CIpc (primary outcome). In addition to “time to last TPTD” we evaluated potential predictors detectable solely by pulse-contour-analysis. Finally, the study aimed to develop a pulse-contour-derived “calibration-index” suggesting recalibration and to validate these results in an independent collective. Methods In 28 intensive-care-patients with PiCCO-monitoring (Pulsion Medical-Systems, Germany) 56 datasets were recorded. CIpc-values at baseline and after intervals of 1 h, 2 h, 4 h, 6 h and 8 h were compared to CItd derived from immediately subsequent TPTD. Results from this evaluation-collective were validated in an independent validation-collective (49 patients, 67 datasets). Results Mean bias values CItd-CIpc after different intervals ranged between -0.248 and 0.112 L/min/m2. Percentage-error after different intervals to last TPTD ranged between 18.6% (evaluation, 2 h-interval) and 40.3% (validation, 6 h-interval). In the merged data, percentage-error was below 30% after 1 h, 2 h, 4 h and 8 h, and exceeded 30% only after 6 h. “Time to last calibration” was neither associated to accuracy nor to precision of CIpc in any uni- or multivariate analysis. By contrast, the height of CIpc and particularly changes in CIpc compared to last thermodilution-derived CItd(base) univariately and independently predicted the bias CItd-CIpc in both collectives. Relative changes of CIpc compared to CItd(base) exceeding thresholds derived from the evaluation-collective (-11.6% < CIpc-CItd(base)/CItd(base) < 7.4%) were confirmed as significant predictors of a bias |CItd-CIpc| ≥ 20% in the validation-collective. Conclusion Recalibration triggered by changes of CIpc compared to CItd(base) derived from last calibration should be preferred to fixed intervals.

There is consensus that indicator dilution techniques provide best accuracy. However, due to rapid and unpredictable changes in hemodynamics in critically ill, the usefulness of intermittent CI-determinations has been questioned, and continuous CI-monitoring might enhance sensitivity of CI-monitoring [19].
However, some data suggest more frequent recalibration with intervals as short as one hour [20,21]. Nevertheless, there are a number of reasons to limit the frequency of TPTDs: TPTD requires a certain amount of time of qualified personal.
Furthermore, calibration with a limited number of TPTDs carries a certain risk of imprecision that might sum up in case of repeated measurements. At least three TPTDs are required to provide acceptable precision ≤10% and detection of changes in CI ≥15% that are generally considered as clinically relevant [25][26][27].
However, repeated TPTDs with at least 45 ml per triplicate measurement might result in a substantial fluid load with impact on hemodynamics itself.
With the data available being not fully consistent and in part retrospective, there is a lack of studies prospectively evaluating the impact of pre-defined periods without calibration and other possible predictors of imprecision, all systematically determined within the same patient.
We hypothesized that accuracy of CIpc might not only depend on time to last TPTD, but also on changes of the arterial wave curve detectable by PCA itself.
Therefore, the aims of our study were to prospectively investigate accuracy and precision of CIpc after pre-defined intervals of 1 h, 2 h, 4 h, 6 h and 8 h after the last TPTD, to evaluate the impact of "time to the last TPTDcalibration" and other factors on the agreement of CIpc and CItd, to derive a "calibration index" solely from PCAparameters comprising a formula predicting the disagreement of CIpc and CItd and an alarmingfunction suggesting recalibration when predicted disagreement exceeds user-defined limits (e.g. >15% or >0.5 L/min/m 2 ), and to validate these results in an independent second collective.

Methods
The study was approved by the institutional review board (Ethikkommission der Fakultät für Medizin der Technischen Universität München; Ismaninger Straße 22; 81675 München). The need of informed consent was waived. In 28 consecutive patients (evaluation collective) with PiCCO-monitoring treated in a general intensive care unit (ICU) or a toxicology ICU, 56 data-sets each including a total of 6 triplicate TPTDs at baseline and after intervals of 1 h, 2 h, 4 h, 6 h and 8 h after the last TPTD were recorded within 21 hours. Since follow-up TPTDs recalibrated CItd, these measurements also provided the baseline CItd for the next interval. The sequence of intervals was randomized.
Results derived from this evaluation-collective were validated in an independent validation-collective of 49 patients with 67 datasets. Due to practical reasons (e.g. transport, external intervention) 21/615 (3%) of measurements could not be performed within ±10 min of the scheduled time and could not be included in the final analysis. A total of 123 datasets with 594 measurements were finally analyzed.
CIpc and CItd were determined using the PiCCO-System (Pulsion Medical Systems, Munich, Germany) as described before [21,28]. Briefly, a 5-French thermistortipped arterial line (Pulsiocath, Pulsion Medical Systems) placed in the femoral artery and a hemodynamic monitor (PiCCO-Plus; PiCCO-2, Pulsion Medical Systems) were used for analysis of pulse-contour and a thermodilution curve after injection of a cold indicator-bolus (15 mL saline 0.9%) through a central-venous catheter (CVC).
CIpc recorded immediately before recalibration with triplicate TPTD was compared to CItd derived from the new TPTD.
Primary endpoint: Analysis of parameters independently associated with the bias CItd-CIpc. These parameters included "time to last calibration" as well as factors continuously provided by pulse contour analysis and their changes compared to baseline.
Secondary endpoints: Analysis of parameters associated with bias CItd-CIpc exceeding pre-defined thresholds (20%, 15% of CItd and 0.5 L/min/m 2 ) and development of a "calibration-index" suggesting recalibration based on parameters derived from pulse-contour-analysis and/or last thermodilution.

Statistics
To describe accuracy and precision of CIpc compared to CItd after different intervals, we performed analyses according to Bland-Altman [29]. To avoid analysis of repeated measurements and different numbers of measurements, Bland-Altman-analyses included only one dataset per patient (first series) and were performed separately for each interval. Percentage-error was calculated as described previously [30].
All other analyses were performed including all datasets except as indicated. For appropriate consideration of multiple measurements per patient in these analyses, uni-and multivariable regression models were fitted in a "Generalized Linear Mixed Model" (GLMM) framework. ROC-analyses were performed to assess discriminative ability of predictor variables regarding pre-defined thresholds of the bias CItd-CIpc (exceeding 20%, 15% of CItd or 0.5 L/min/m 2 ). Percentages were calculated based on the measurements with valid data. In the course of GLMManalysis, standard-errors of regression coefficients were reported. In order to consider repeated measurements per individual, partial correlation-coefficients (r part ) were calculated for bivariate correlation.
Predictors of bias exceeding critical thresholds derived from the evaluation-collective were analysed in the validation-collective based on ROC-and percentageerror-analysis.
All statistical analyses were performed by statistician co-author TS using IBM SPSS Statistics 21 (SPSS Inc., Chicago, IL, USA).

Patients characteristics and interventions
Patients characteristics are demonstrated in Table 1.

Bias values exceeding pre-defined thresholds
Despite low mean bias values, "CItd-CIpc" exceeded critical thresholds in a relevant number of single comparisons (Table 2). In the merged data, bias values exceeding ±20%, ±15% and ±0.5 L/min/m 2 were observed in 85/594 (14.3%), 138/594 (23.2%) and 166/594 (27.9%) of measurements. Mean bias values in general were low and ranged between -0.248 (evaluation-collective, after 6 h-interval) and 0.112 L/min/m 2 (validation-collective after 2 hinterval). In both collectives mean bias values were not dependent on time to last calibration (Table 3, Figure 1). Notched boxplots further support that bias CItd-CIpc did not increase over time and did not differ after various times to last calibration: Only if the notches of two boxplots do not overlap, the two medians differ [31,32].
Bland-Altman-diagrams (one data set per patient; Figure 2) with lower and upper limits of agreement and bias-values demonstrate comparable accuracy and precision for the different intervals to last TPTD.
Univariate analysis of potential predictors of bias CItd-CIpc including "time to last TPTD" Similarly to the data based on one data set per patient (Figures 1 and 2 and Table 3), the interval to last TPTD was not associated to the bias CItd-CIpc when including repeated data sets for correlation analysis. Comparison of bias CItd-CIpc to "time to last calibration" provided poor coefficients of partial correlation r part and p-values in evaluation-collective (r part = -0.09; p = 0.536), validation-collective (r part =0.083; p = 0.605) and merged data (r part = 0.076; p = 0.363). As demonstrated in Table 4 and Figure 3, bias CItd-CIpc was most strongly associated to the difference CIpc-CItd(base) (r part = -0.592 (evaluation-collective), r part = -0.630 (validation-collective) and r part = -0.606 (merged data); p < 0.001 for both collectives and merged data). The second strongest predictor of the bias CItd-CIpc was CIpc itself (r part = -0.367 (evaluation), r part = -0.573 (validation) and r part = -0.466 (merged data; p < 0.001 for both collectives and merged data; Table 4; Figure 4).
In addition to these associations of absolute bias CItd-CIpc to absolute changes in the above-mentioned predictors, relative bias (CItd-CIpc)/CItd was similarly associated to relative changes in the predictors (Table 4).

Multivariate analysis regarding prediction of absolute bias CItd-CIpc
In multivariate GLMM-analysis, absolute bias CItd-CIpc was independently associated to CIpc-CItd(base) (p < 0.001) and to CIpc itself (p < 0.001), but not the interval to the last TPTD. These findings were consistent for evaluation, validation and merged data. Similar results were obtained for "relative bias" CItd-CIpc/CItd (data not shown).

Multivariate ROC-analysis regarding critical thresholds of bias CItd-CIpc
Multivariate analysis demonstrated independent association of relative changes in CIpc-CItd(base) to relative bias (CItd-CIpc)/CItd exceeding ±15%, 20% and 0.5 L/min/m 2 . By contrast, "interval to last TPTD" was not independently  Figure 1 Notched boxplots demonstrate that bias CItd-CIpc did not increase over time and did not differ after various times to last calibration. Only if the notches of two boxplots do not overlap, this is 'strong evidence' that the two medians differ [31,32]. CItd: Thermodilutionderived Cardiac Index. CIpc: Pulse-contour-derived Cardiac Index. TPTD: Transpulmonary thermodilution. associated to relative bias (CItd-CIpc)/CItd exceeding these thresholds.
Predictive capabilities of relative changes in "CIpc-CItd(base)" regarding several thresholds could be further improved by also including changes in "Index of Left Ventricular Contractility" (dPmax) ( Figure 6) or changes in PP in a GLMM-derived multivariate model. E.g. a model derived from the evaluation-collective including changes in CIpc-CItd(base) and in dPmax slightly improved ROC-AUC regarding relative bias exceeding ≥15% in the evaluation-collective (AUC 0.883 vs. 0.857) as well as in the validation-collective (0.761 vs. 0.720) ( Figure 6).

Validation of predictors of substantial bias derived from the evaluation collective in the validation collective
Analysis of evaluation data demonstrated that a decrease in CIpc-CItd(base) of at least 11.62% or an increase of at least 7.43% significantly predicted a bias CItd-CIpc exceeding ±20% (see above).
To assess the potential practical use of these thresholds derived from the evaluation-collective in the validation-collective, we compared the 8 h percentage-error of patients in the validation-collective staying within and without these critical thresholds. In case of repeated inclusion, only the first 8 h-observation-period was analysed. 8 h-measurements in 26 validation-patients staying within the critical evaluation-thresholds of relative changes in CIpc compared to CItd(base) (-11.62% < (CIpc-CItd (base)/CItd(base) < 7.43%) provided a percentage-error of 22.6% compared to 44.0% for those 16 validation-patients outside of these evaluation-thresholds. Similar analysis of the validation-data after a 1 h-interval demonstrated a percentage-error of 9.7% in 29 patients staying within the critical thresholds of relative changes in CIpc compared to  CItd(base) compared to 36.7% for patients outside of these thresholds.

Parameter
Correlation to "absolute bias" (coefficient of partial correlation) Correlation to "relative bias" (coefficient of partial correlation)   parameters included in the GLMM-analysis (time to last calibration, changes in PP and in dPmax) failed sig0nificance.
Therefore, we prospectively evaluated the accuracy of CIpc after pre-defined calibration-free periods. The main results of this study can be summarized as follows: Mean bias of CIpc vs. CItd was acceptable after all time-periods. However, about 23% of CIpc-values deviated ≥15% from immediately subsequent CItd. This emphasizes the need for repeated recalibration.
Adaptation of recalibration to a fixed time-based scheme is not substantiated by our data, since "time to last calibration" was neither associated to accuracy nor precision of CIpc.
By contrast, relative and absolute changes in CIpc compared to the last TPTD-derived CItd(base) were independent predictors of relative and absolute bias CItd-CIpc.
Critical thresholds of changes in CIpc compared to CItd(base) derived from the evaluation-collective were confirmed as predictors of the bias CItd-CIpc in the validation-collective.
Multivariate analyses suggest that more complex mathematical models also including CIpc itself, changes in PP and dPmax might further improve prediction of disagreement between CIpc and CItd.
In our study CIpc provided appropriate accuracy irrespective of the interval to last TPTD with mean bias values between -0.21and 0.068 L/min/m 2 (merged data). These low mean bias values in all subgroups are in accordance with two studies evaluating PiCCO-derived CIpc with mean bias values of 0.03-0.16 L/min/m 2 [20] and 0.06-0.29 L/min/m 2 [24] after different intervals to last TPTD [20,24] and different dosages of noradrenalin [24].
Driven by the practical need to define "when" to re-calibrate, time-dependency of CIpc-accuracy is an obvious hypothesis. However, this is neither well substantiated by previous investigations nor by our study: Although percentage-error was within the critical threshold of 30% only within the 1st hour in Hamzaoui's study, time to last TPTD was not an independent predictor of precision in multivariate analysis [20].
In our merged data, percentage-error was below 30% within the first 4 hours and after 8 hours and exceeded 30% only after 6 hours. In addition to percentage-error comparison after different times to last TPTD, we performed univariate and multivariate analysis regarding agreement of single CIpc-values with CItd in two different collectives and in the merged data. Furthermore, we performed notched box-plot-analyses-favouring comparison of medians over means -for pre-defined intervals to last TPTD. None of these analyses provided evidence for time-dependency of the agreement of CIpc and CItd. This is also in accordance with a study by Gruenewald et al. who did not find any hints for an association of CIpc-precision with interval to last TPTD [24].
In general, assessment of CIpc is mainly based on the assumption that left-ventricular stroke-volume is proportional to the area under the systolic portion of the arterial pressure-curve (AUSPC). Depending on compliance and systemic vascular resistance, identical AUSPC-values result in different stroke-volumes. Therefore, most of the pulse-contour-technologies try to correct for these individual factors. These individual factors can be assumed to be composed of static (individual biometry: age, gender, height, weight etc.) and dynamic components (changes in compliance and resistance/impedance).
Early pulse-contour-approaches were mainly based on intermittent re-adjustment. More recent approaches also tried continuous correction based on more sophisticated waveform-analysis also including shape of the waveform, position of the dicrotic notch and analysis of the postsystolic area behind the dicrotic notch. This part represents passive emptying of the aorta due to the Windkessel-effect [10]. Additionally, pulse-contour-algorithms include empirical and biometric data to a different extent. This finally resulted in approaches of CIpc-assessment exclusively based on pulse-contour-analysis, empiric and biometric data, thus totally rejecting any calibration [14][15][16][17][18].
The algorithm used in recent PiCCO-devices is based on intermittent recalibration as well as continuous adjustment. With the exact algorithm being proprietary, it can be assumed that TPTD-derived calibration has impact on a "patient-specific calibration-factor" remaining constant until the next calibration ("cal"; Figure 7). Furthermore, calibration intermittently modifies continuous adjustment of Systemic Vascular Resistance (SVR) and compliance ("C(p)").
This might in part explain that parameters adjustable by the above-mentioned formula such as changes in heart rate and arterial pressure were not substantially associated to the deviation of CIpc in our study. By contrast, changes in CIpc compared to CItd(base) and CIpc itself were independently associated with the deviation of CIpc. Independent association of CIpc to the bias CItd-CIpc suggests a systematic deviation which offers potential of systematic correction for an improved algorithm.
CIpc-CItd(base) was -by far -the most important predictor of inaccuracy of CIpc. With these changes being easily and continuously detectable by pulse-contour itself, our data suggest this parameter as a main component of a "calibration-index" triggering recalibration. Prediction of CIpc-deviation exceeding pre-defined thresholds wasin part -improved by including changes in PP and/or dPmax.
However, even a calibration-index restricted to this single predictor "CIpc-CItd(base)" provided ROC-AUCs between 0.75 and 0.81 in the prediction of CIpc-deviation exceeding several critical thresholds.
In addition to our findings of an association of inaccuracy of CIpc to CIpc-CItd(base) and CIpc itself, a number of clinical and/or non-hemodynamic predictors might be associated with reduced accuracy of CIpc.
Data on the impact of (changes in) vasopressor-therapy are -in part-conflicting: In an animal model, Bein and coworkers demonstrated marked deteriorated accuracy and precision after haemorrhage and application of noradrenalin [23]. In a recent study in 73 ICU-patients, the authors demonstrated improved accuracy of CIpc in patients with high doses of noradrenalin (>0.1 μg/kg/min) compared to lower doses of noradrenalin or no noradrenalin [24]. This might be explained by noradrenalin -induced arterial stiffness stabilizing compliance and resistance.
Furthermore, changes in SVR have been suggested as predictors of inaccuracy of CIpc. Rodig et al. demonstrated marked impaired bias and precision of CIpc after marked changes in SVR >60% induced by phenylephrine [7]. Yamashita et al. showed reduced accuracy and precision of CIpc after SVR-decreases induced by prostaglandin [33]. However, in Hamzaoui's study neither univariate nor multivariate analysis demonstrated an association of changes in SVR to the agreement of CIpc and CItd [20].
Among several other parameters and interventions, increased IAP [22], haemorrhage [23] and volume resuscitation [22] have been associated with decreased accuracy of CIpc.

Practical implications
The present recommendation to recalibrate after "8 h or in case of instability or events probably associated with inaccuracy of CIpc" is difficult to perform: Even under study conditions TPTD-intervals frequently exceed 8 h [24].
Regarding an increasing number of factors and interventions (see Table 1) potentially associated to inaccuracy of CIpc, permanent screening for these factors is cumbersome and hardly feasible.
Furthermore, many of the above-mentioned events associated to CIpc-inaccuracy (vasopressors, hemorrhage, increased intra-abdominal pressure etc.) can be assumed to result in changes in arterial pulse-wave. Our data support that a calibration-index derived solely and continuously from pulse-contour-analysis might be a useful tool to improve the yield of relevant TPTD-measurements and to reduce "routine"-measurements passed down from devices incapable of combining intermittent and continuous monitoring. Summarizing different analyses of this study, re-calibration should be considered in case of changes of CIpc of more than 10% compared to the last CItd.

Limitations of the study
Despite inclusion of two independent collectives our data are derived from only two ICUs. Although our data suggest that the (in) accuracy of CIpc is predominantly associated to changes in CIpc compared to baseline CItd, we cannot definitely rule out a certain impact of time to last calibration due to the limited number of patients.
Regarding ethical considerations we did not extend the calibration-free observation-period above the maximum interval of 8 h suggested by the manufacturers.
At first glance the study design not including a predefined sequence of interventions (e.g. fluid-challenge, changes in vasoactive drugs) might be considered to be observational. However, based on clinical requirements most of the patients experienced substantial changes in treatment modalities during the 21 h observation-period including onset and termination of renal-replacementtherapy, changes in ventilator-settings and vasoactive drugs (Table 1).

Conclusion
At present recalibration of CIpc by TPTD is suggested after a maximum of 8 h, although there is an ongoing debate to which extent accuracy of CIpc depends on the "time to last calibration". By contrast, this study suggests that recalibration triggered by changes of the CIpc itself compared to the last calibration should be preferred to fixed intervals to last TPTD.

Key messages
1) At present recalibration of CIpc by TPTD is suggested after a maximum of 8 h, although there is an ongoing debate to which extent accuracy of CIpc depends on the time to last calibration. 2) None of several analyses of this study supports that accuracy and/or precision of CIpc depend on the time to last calibration by TPTD.
3) By contrast, our data suggest that recalibration triggered by changes of CIpc itself compared to CItd (base) derived from the previous TPTD should be preferred to fixed intervals. 4) A "calibration-index" derived solely and continuously from pulse-contour-analysis might be a useful tool to improve the yield of relevant TPTD-measurements and to reduce "routine"-measurements after rigid intervals. 5) In addition to CIpc-CItd(base), changes in pulse pressure and/or dPmax might further improve a continuously derived "calibration-index" suggesting recalibration. Competing interests Wolfgang Huber and Bernd Saugel collaborate with Pulsion Medical Systems AG as members of the Medical Advisory Board. All authors declare that they have no competing interests.
Authors' contributions WH performed conception and design of the study, participated in the analysis of the data, drafted the manuscript and finally approved the manuscript. JK performed the majority of measurements in the evaluation collective, participated in analysis of the data and in drafting the manuscript and finally approved the manuscript. SM performed the majority of measurements in the validation collective, participated in analysis of the data and in drafting the manuscript and finally approved the manuscript. TS participated in acquisition of the data, performed the statistical analyses, participated in drafting the manuscript and finally approved the manuscript. BS participated in conception and design of the study, participated in drafting the manuscript and finally approved the manuscript. FE performed measurements in the toxicology ICU, participated in analysis of the data and in drafting the manuscript and finally approved the manuscript. VP participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. CS participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. PT participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. UM participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. HE participated in concept and design of the study, participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. MT participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. JH participated in acquisition and analysis of the data, participated in drafting the manuscript and finally approved the manuscript. RS substantially contributed to conception and design of the study, participated in the analysis of the data, participated in drafting the manuscript and finally approved the manuscript. All authors read and approved the final manuscript.