- Open Access
Comparison of accuracy of two uncalibrated pulse contour cardiac output monitors in off-pump coronary artery bypass surgery patients using pulmonary artery catheter-thermodilution as a reference
BMC Anesthesiology volume 21, Article number: 189 (2021)
Cardiac output (CO) is a key measure of adequacy of organ and tissue perfusion, especially in critically ill or complex surgical patients. CO monitoring technology continues to evolve. Recently developed CO monitors rely on unique algorithms based on pulse contour analysis of an arterial blood pressure (ABP) waveform. The objective of this investigation was to compare the accuracy of two monitors using different methods of pulse contour analysis – the Retia Argos device and the Edwards Vigileo-FloTrac device – with pulmonary artery catheter (PAC)-thermodilution as a reference.
Fifty-eight patients undergoing off-pump coronary artery bypass surgery formed the study cohort. A total of 572 triplets of CO measurements from each device – Argos, Vigileo-FloTrac (third generation), and thermodilution – were available before and after interventions (e.g., vasopressors, fluids, and inotropes). Bland–Altman analysis accounting for repeated measurements per subject and concordance analysis were applied to assess the accuracy of the CO values and intervention-induced CO changes of each pulse contour device against thermodilution. Cluster bootstrapping was employed to statistically compare the root-mean-squared-errors (RMSE = √(μ2 + σ2), where μ and σ are the Bland–Altman bias and precision errors) and concordance rates of the two devices.
The RMSE (mean (95% confidence intervals)) for CO values was 1.16 (1.00–1.32) L/min for the Argos device and 1.54 (1.33–1.77) L/min for the Vigileo-FloTrac device; the concordance rate for intervention-induced CO changes was 87 (82–92)% for the Argos device and 72 (65–78)% for the Vigileo-FloTrac device; and the RMSE for the CO changes was 17 (15–19)% for the Argos device and 21 (19–23)% for the Vigileo-FloTrac device (p < 0.0167 for all comparisons).
In comparison with CO measured by the PAC, the Argos device proved to be more accurate than the Vigileo-FloTrac device in CO trending and absolute CO measurement in patients undergoing off-pump coronary artery bypass surgery.
Cardiac output (CO) monitoring is routinely used to assist in the hemodynamic management of patients undergoing major surgery and in the intensive care unit. Devices that do not require calibration (with a reference CO method), or un-calibrated CO monitors, are commonly used in clinical practice. Of these, the pulse contour-based devices permit continuous CO monitoring via mathematical analysis of a peripheral arterial blood pressure (ABP) waveform . Some of these devices detect pulse pressure as a marker of stroke volume and then multiply the pulse pressure with heart rate to compute the ratio of CO to the arterial compliance [2, 3]. Other devices fit an exponential to the ABP diastolic decay to determine the “Windkessel” time constant (τ = systemic vascular resistance times arterial compliance) and then divide mean ABP by the τ to compute the ratio of CO to the arterial compliance as well . For example, the Vigileo-FloTrac device (Edwards Lifesciences, Irvine, USA) essentially determines pulse pressure times heart rate and then computes the arterial compliance term using various ABP waveform statistics and patient demographic information . However, arterial wave reflection may cause the peripheral pulse pressure to change irrespective of stroke volume due to vasoconstriction or vasodilation and may also obscure exponential diastolic decays in peripheral ABP waveforms. While these devices have been extensively tested in human subjects, accuracy still remains a concern [1, 5,5,7]. Notably, none of the devices have emerged as being more accurate than another.
The Argos device (Retia Medical, Valhalla, USA) is a new un-calibrated pulse contour device that has recently received US Food & Drug Administration (FDA) clearance . The unique idea underlying this device is to apply a multi-beat analysis (MBA™) to model arterial wave reflection, as described previously [9,9,10,11,13] and shown in Fig. 1. Initially, the ABP response to a single heartbeat is estimated from a radial ABP waveform segment over multiple beats via mathematical modeling. The Windkessel time constant τ is then determined by fitting an exponential to the tail end of this response once the faster wave reflection vanishes. Finally, the arterial compliance term is determined from the ABP waveform and patient age, height, weight, and gender utilizing a proprietary formula.
Recently, Saugel and colleagues compared the Argos device with bolus thermodilution CO via a pulmonary artery catheter (PAC) in off-pump coronary artery bypass surgery patients . The objective of the present study was to perform a secondary analysis of these data to compare the accuracy of the Argos device with the Vigileo-FloTrac device using thermodilution as a reference.
Existing patient data
All patient data for the current study have been previously described in detail  and were collected by Saugel and colleagues at University Medical Center Hamburg-Eppendorf as part of studies testing the accuracy of the Argos device. Their study was approved by the Ethics Committee of the Hamburg Medical Association (Germany), and all enrolled patients provided written, informed consent. The inclusion criteria were adults undergoing off-pump coronary artery bypass surgery who had clinical indications for radial and pulmonary artery catheterization. Exclusion criteria included the presence of severe arrhythmias or missing informed consent. Bolus thermodilution CO via a PAC (average of four consecutive injections of 10 ml ice-cold saline) was measured before and after up to six clinical interventions (e.g., fluids, vasopressors, and inotropes) in each patient, while the ABP waveform via a radial artery catheter was recorded at a sampling rate of 100 Hz throughout the surgery. The radial ABP waveform segments during the time periods of the reference thermodilution CO measurements were visually screened to exclude unreliable data . A total of 572 pairs of ABP waveform segments and reference CO values from 58 patients (17% female, 70 ± 10 (mean ± SD) years in age, 1.74 ± 0.09 m in height, 81 ± 15 kg in weight, and 26.8 ± 4.2 kg/m2 in body mass index; see Table 1 in reference ) remained for analysis. The ABP waveform segments were entered off-line into the Argos device operating in 20-s mode, which outputs the average CO over the last 20-s every 5-s, by an independent contractor (see Acknowledgements section) . The resulting CO values were then averaged over the time intervals of the reference CO values by the contractor. The contractor performed this work while blinded to the reference CO values.
All ABP waveforms were also entered off-line into the Vigileo-FloTrac device (software version 3.02) operating in 20-s mode by the same independent contractor. The data feeding procedure was similar to a procedure that was reviewed and approved by the US FDA (see Argos device Operator’s Manual ). The resulting CO values were then similarly averaged by the contractor to correspond to the reference CO values. The contractor was again blinded to the reference CO values.
Secondary statistical data analysis
The total of 572 triplets of Argos, Vigileo-FloTrac, and thermodilution CO values as well as 509 corresponding triplets of intervention-induced changes in consecutive CO values (ΔCO = 100⋅(COafter—CObefore)/CObefore) from the 58 patients were provided after de-identification by Saugel and colleagues for secondary analysis. The provided data are included here – without alteration – as supplemental information (see Additional file 1). The local Institutional Review Board declared that this secondary analysis did not constitute human subjects research (Michigan State University Study ID: STUDY00002367).
Statistical analysis of the data was performed to compare the Argos and Vigileo-FloTrac devices in terms of the accuracy of their CO values (absolute measurement ability) and ΔCO values (trending ability) against the reference thermodilution CO values. The analysis was performed using the MATLAB software package (Natick, USA), and the code is also included here as supplemental information (see Additional file 2).
The accuracy of each of the Argos and Vigileo-FloTrac devices against reference thermodilution was individually assessed via Bland–Altman analysis of the CO and ΔCO values and standard concordance analysis of the ΔCO values (with a 15% exclusion zone). For the Bland–Altman analysis, repeated measurements per subject were taken into account via mixed effects modeling . To quantify device errors, the root-mean-squared-errors (RMSEs) of the CO and ΔCO values of each device were computed as √(μ2 + σ2), where μ and σ are the Bland–Altman bias and precision errors, respectively.
The relative accuracies of the Argos and Vigileo-FloTrac devices were assessed via statistical comparisons of their CO RMSEs, ΔCO RMSEs, and concordance rates. Non-parametric cluster bootstrapping was utilized to calculate confidence intervals and make the comparisons . Specifically, 10,000 random samples of patients with replacement of patients were taken from the collected data. The number of patients in each sample was 58, which is the number of patients in the clinical study, and all triplets of CO and ΔCO values from a patient were included in each sample to maintain the correlated data structure. For each sample, the CO RMSE, ΔCO RMSE, and concordance rate of each device were computed as described above, and the difference between each pair of the quantitative metrics of the two devices (Argos – Vigileo-FloTrac) was taken. Both 95% CIs of each quantitative metric of each device and X% CIs of each difference between the metrics of the two devices were then calculated from the corresponding distribution of 10,000 values via a standard percentile bootstrap. If the upper CI for the CO or ΔCO RMSE difference were less than 0 or the lower CI for the concordance rate difference were greater than 0, then the Argos device would be considered superior to the Vigileo-FloTrac device in terms of that particular metric. Since a total of three statistical comparisons were made, a Bonferroni correction was applied such that a two-sided p < 0.05/3 (i.e., X = 98.33) was considered statistically significant.
A total of 572 triplets of Argos, Vigileo-FloTrac, and thermodilution CO values as well as 509 corresponding triplets of intervention-induced changes in consecutive CO values (ΔCO = 100⋅(COafter—CObefore)/CObefore) from 58 patients were included in this study for secondary analysis. Figure 2A and C show Bland–Altman plots of the CO and ΔCO values of the Argos device versus reference thermodilution and of the Vigileo-FloTrac device versus the same reference, respectively. Figure 2B shows concordance plots of the ΔCO values of the Argos device versus reference thermodilution and of the Vigileo-FloTrac device versus this reference. Note that the Bland–Altman plot of the CO values and the concordance plot of the ΔCO values for the Argos device matched those reported by Saugel and colleagues (see Figs. 2 and 3 in reference ). The Table 1 shows statistical comparisons of the CO RMSE, ΔCO RMSE, and concordance rate of the Argos device versus the Vigileo-FloTrac device. This table also indicates the bias and precision error components of each RMSE.
The CO RMSE (mean (lower 95% CI-upper 95% CI)) was 1.16 (1.00–1.32) L/min for the Argos device and 1.54 (1.33–1.77) L/min for the Vigileo-FloTrac device (p < 0.0167). This difference was mainly due to the smaller bias error of the Argos device (see Fig. 2A and Table 1). The concordance rate was 87 (82–92)% for the Argos device and 72 (65–78)% for the Vigileo-FloTrac device (p < 0.0167). The ΔCO values of the Argos device also showed less scatter about the identity line in the concordance plots (see Fig. 2B). Consistent with these plots, the ΔCO RMSE was 17 (15–19)% for the Argos device and 21 (19–23)% for the Vigileo-FloTrac device (p < 0.0167). This difference was due to the smaller precision error (i.e., less scatter) of the Argos device (see Fig. 2C and Table 1).
Cardiac output monitoring allows improved hemodynamic management of critically ill and complex patients. The Retia Argos device is a new un-calibrated pulse contour device for CO monitoring that aims to improve accuracy by accounting for confounding arterial wave reflection in a radial ABP waveform with multi-beat analysis (see Fig. 1). In the present study, the Argos device was compared to the Edwards Vigileo-FloTrac device  using PAC-thermodilution CO as a reference via a secondary analysis of de-identified data from patients undergoing cardiac surgery. While the accuracy results of the Argos device against thermodilution (see left column of Fig. 2AB) were previously reported , the accuracy results of the Vigileo-FloTrac device against thermodilution are new (see right column of Fig. 2). These additional results importantly allowed for head-to-head comparisons of the two un-calibrated pulse contour devices on the exact same data (see entire Fig. 2 and Table 1). The aggregate results revealed that the Argos device was more accurate in absolute CO measurement and CO trending than the Vigileo-FloTrac device. Importantly, the testing of accuracy was performed using the standard clinical method (i.e., PAC-bolus thermodilution) as reference and before and after clinically relevant hemodynamic interventions (e.g., fluids, vasopressors, and inotropes).
The improvements in accuracy afforded by the Argos device could be seen visually in the Bland–Altman and concordance plots (see Fig. 2) and reached statistical significance (see Table 1). Most notably, the Argos device reduced the absolute CO RMSE by 25% and increased the concordance rate by 21% relative to the Vigileo-FloTrac device. To the best of our knowledge, few, if any, past studies have statistically shown that one un-calibrated pulse contour device is more accurate than another.
Standard Bland–Altman and concordance analyses were employed as a basis for the comparison of the two un-calibrated pulse contour devices . Bland–Altman analysis of the L/min CO values of each device was performed to assess the absolute CO measurement accuracy. Bland–Altman analysis of the intervention-induced CO changes of each device was conducted to evaluate the trending accuracy in terms of both direction and magnitude of the CO change, while concordance analysis was performed to assess the directional trending accuracy alone. The repeated measurements per patient were accounted for by applying (i) mixed effects modeling  to compute the Bland–Altman bias and precision errors of each device and (ii) cluster bootstrapping  to compare RMSEs (i.e., a single number representing the aggregate of the two errors) and concordance rates of the two devices. Note that the popular percentage error (i.e., twice the precision error divided by the grand mean of the device and reference CO values) was not employed, as it is not a robust metric of accuracy. For example, as a device increasingly overestimates the reference measurements, the percentage error will progressively decrease – instead of increase. However, for completeness, the percentage errors of the Argos and Vigileo-FloTrac devices were both about 50% in this study. Also note that Bland–Altman analysis of the CO changes was preferred to polar plot analysis, which is more commonly used to assess the magnitude and direction of the CO change , because the error in the CO change yielded by the Bland–Altman analysis is more intuitive than the “angle” outputted by the polar plot analysis. In addition, percent CO changes rather than L/min CO changes were assessed, as L/min CO changes are confounded by the arterial compliance scale factor (see, e.g., Fig. 1) and are thus not a pure indicator of trending ability.
Acceptable levels of accuracy have previously been proposed including percentage errors within 30% or 40% and concordance rates exceeding 90% [6, 19]. The Argos device, but not the Vigileo-FloTrac device, achieved a concordance rate close to the proposed level and may thus offer value in CO trending. However, even though percentage error is not a robust accuracy metric, it may still be concluded that neither of the two pulse contour devices is interchangeable with PAC-thermodilution for absolute CO measurement.
A limitation of this study was that the third generation Vigileo-FloTrac device was used rather than the latest fourth generation device. However, the third generation device has often been used in clinical practice  and continues to be the subject of recent publications [20,20,21,22,24]. Further, while the fourth generation device appears to be improved in terms of measuring fast CO changes induced by phenylephrine boluses , it yielded an average percentage error of 47% and average concordance rate of 70% against PAC-thermodilution CO measurements in cardiac surgery patients in the OR [26, 27]. These results are similar to the results of the third generation device reported herein. Nevertheless, future comparisons of the Argos device and the fourth generation device should be conducted using the same patient data to draw definitive conclusions. Other limitations of the study were that the available patient cohort was confined to a relatively homogenous cardiac surgery population and that the accuracy difference between the two pulse contour devices could not be assessed for clinical significance. Additional studies are warranted to determine the relative accuracy and relative clinical value of these and other devices in hyperdynamic and other patient cohorts.
In this study, the Retia Argos device with multi-beat analysis was more accurate than the Edwards Vigileo-FloTrac device in CO trending and absolute CO measurement in patients undergoing off-pump coronary artery bypass surgery. The accuracy attained in CO trending suggests the Argos device will potentially be more informative and useful if CO measurement is initiated when the arterial catheter is inserted rather than connecting the device to the catheter after a patient has already decompensated. A recent meta-analysis indicated that use of such un-calibrated pulse contour devices is associated with a decrease in postoperative morbidity . Enhancing the CO measurement accuracy of the devices would not only provide a truer picture of the hemodynamic status of patients but could also potentially help to further improve patient outcomes.
Arterial blood pressure
Food & Drug Administration
Intervention-induced cardiac output change in percent
Mean arterial pressure
Pulmonary artery catheter
Root-mean-squared-error (aggregate of the Bland–Altman bias error (μ) and precision error (σ))
Windkessel time constant, which equals the systemic vascular resistance times the arterial compliance (AC)
Grensemann J. Cardiac output monitoring by pulse contour analysis, the technical basics of less-invasive techniques. Front Med. 2018;6(5):64.
Pratt B, Roteliuk L, Hatib F, Frazier J, Wallen RD. Calculating arterial pressure-based cardiac output using a novel measurement and analysis method. Biomed Instrum Technol. 2007;41:403–11.
Rhodes A, Sunderland R. Arterial pulse power analysis: the LidCOTMplus system. In: Pinsky MR, Payen D, editors. Functional hemodynamic monitoring. Berlin: Springer; 2005. p. 183–92.
Godje O, Hoke K, Goetz AE, et al. Reliability of a new algorithm for continuous cardiac output determination by pulse-contour analysis during hemodynamic instability. Crit Care Med. 2002;30(1):52–8.
Michard F, Giglio MT, Brienza N. Perioperative goal-directed therapy with uncalibrated pulse contour methods: impact on fluid management and postoperative outcome. Br J Anaesth. 2017;119(1):22–30.
Peyton PJ, Chong SW. Minimally invasive measurement of cardiac output during surgery and critical care. Anesthesiology. 2010;113(5):1220–35.
Joosten A, Desebbe O, Suehiro K, et al. Accuracy and precision of non-invasive cardiac output monitoring devices in perioperative medicine: a systematic review and meta-analysis. Br J Anaesth. 2017;118(3):298–310.
Retia Medical. http://www.retiamedical.com. Accessed 24 Feb 2021.
Mukkamala R, Reisner AT, Hojman HM, Mark RG, Cohen RJ. Continuous cardiac output monitoring by peripheral blood pressure waveform analysis. IEEE Trans Biomed Eng. 2006;53(3):459–67.
Lu Z, Mukkamala R. Continuous cardiac output monitoring in humans by invasive and non-invasive peripheral blood pressure waveform analysis. J Appl Physiol. 2006;101(2):598–608.
Mukkamala R, Xu D. Continuous and less invasive monitoring of central hemodynamics by blood pressure waveform analysis. Am J Physiol. 2010;299(3):H584–99.
Reisner AT, Xu D, Ryan KL, Convertino VA, Rickards CA, Mukkamala R. Monitoring non-invasive cardiac output and stroke volume during experimental human hypovolemia and resuscitation. Br J Anaesth. 2011;106(1):23–30.
Zhang G, Mukkamala R. Continuous and minimally invasive cardiac output monitoring by long time interval analysis of a radial arterial blood pressure waveform: assessment using a large, public intensive care unit patient database. Br J Anaesth. 2012;109(3):339–44.
Saugel B, Heeschen J, Hapfelmeier A, Romagnoli S, Greiwe G. Cardiac output estimation using multi-beat analysis of the radial arterial blood pressure waveform: a method comparison study in patients having off-pump coronary artery bypass surgery using intermittent pulmonary artery thermodilution as the reference method. J Clin Monit Comput. 2020;34(4):649–54.
Products. Retia Medical. http://www.retiamedical.com/products/installation-and-use/. Accessed 24 Feb 2021.
Bland JM, Altman DG. Agreement between methods of measurement with multiple observations per individual. J Biopharm Stat. 2007;17(4):571–82.
Efron B, Tibshirani RJ. An Introduction to the bootstrap. Boca Raton: Chapman and Hall; 1993.
FloTrac System. https://www.edwards.com/gb/devices/hemodynamic-monitoring/FloTrac. Accessed 24 Feb 2021.
Critchley LA, Lee A, Ho AM. A critical review of the ability of continuous cardiac output monitors to measure trends in cardiac output. Anesth Analg. 2010;111(5):1180–92.
Hao GW, Liu Y, Ma GG, et al. Reliability of three-dimensional color flow Doppler and two-dimensional pulse wave Doppler transthoracic echocardiography for estimating cardiac output after cardiac surgery. Cardiovasc Ultrasound. 2019;17(1):5.
Ganter MT, Geisen M, Harnack S, Dzemali O, Hofer CK. Prediction of fluid responsiveness in mechanically ventilated cardiac surgical patients: the performance of seven different functional hemodynamic parameters. BMC Anesthesiol. 2018;18(1):55.
Geisen M, Ganter MT, Hartnack S, Dzemali O, Hofer CK, Zollinger A. Accuracy, precision, and trending of 4 pulse wave analysis techniques in the postoperative period. J Cardiothorac Vasc Anesth. 2018;32(2):715–22.
Jalil B, Thompson P, Cavallazzi R, et al. Comparing changes in carotid flow time and stroke volume induced by passive leg raising. Am J Med Sci. 2018;355(2):168–73.
Milam AJ, Ghoddoussi F, Lucaj J, et al. Comparing the mutual interchangeability of ECOM, FloTrac/Vigileo, 3D-TEE, and ITD-PAC cardiac output measuring systems in coronary artery bypass grafting. J Cardiothorac Vasc Anesth. 2021;35(2):514–29.
Suehiro K, Tanaka K, Mikawa M, et al. Improved performance of the fourth-generation FloTrac/Vigileo system for tracking cardiac output changes. J Cardiothorac Vasc Anesth. 2015;29(3):656–62.
Kusaka Y, Ohchi R, Minami T. Evaluation of the fourth-generation FloTrac/Vigileo system in comparison with the intermittent bolus thermodilution method in patients undergoing cardiac surgery. J Cardiothorac Vasc Anesth. 2019;33(4):953–60.
Lin SY, Chou AH, Tsai YF, et al. Evaluation of the use of the further version FloTrac system in cardiac output measurement before and after cardiopulmonary bypass. J Clin Monit Comput. 2018;32(5):807–15.
The authors thank Dr. Bernd Saugel and colleagues of University Medical Center Hamburg-Eppendorf (Germany) for providing the patient data for analysis and Dr. Ricardo Toledo-Crow of the Advanced Science Research Center, City University of New York (New York, USA) for feeding the arterial blood pressure waveforms into the Argos and Vigileo-FloTrac devices.
This work was supported in part by Retia Medical through consulting fees.
Ethics approval and consent to participate
This study is a secondary analysis of existing, de-identified patient data. The Institutional Review Board of Michigan State University (East Lansing, USA) declared that this secondary analysis did not constitute human subjects research (MSU Study ID: STUDY00002367).
Consent for publication
R. M. is Co-Founder, Chief Technology Officer, and Board Member of Retia Medical (Valhalla, NY, USA) and has equity and a paid consulting agreement with the company. B. A. K. is an unpaid member of the Clinical Advisory Board of Retia Medical. A. M. declares no conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
R. M. started this study when he was in the Department of Electrical and Computer Engineering, Michigan State University, East Lansing, MI, USA.
Patient data provided by Saugel and colleagues. L/min cardiac output (CO) via reference pulmonary artery catheter-thermodilution (CO_TD (Lpm) sheet), Argos (CO_Retia (Lpm) sheet), and Vigileo-FloTrac (CO_Edwards (Lmin) sheet) devices. %CO change between consecutive L/min CO measurements via reference pulmonary artery catheter-thermodilution (d_TD (%) sheet), Argos (d_Retia (%) sheet), and Vigileo-FloTrac (d_Edwards (%) sheet) devices. Quality of blood pressure (BP) waveforms corresponding to L/min CO with accompanying legend (Quality sheet). Row corresponds to patient, and column denotes each measurement from the patient. PID – patient identification number; NaN – unavailable measurement in second through seventh sheets
MATLAB code employed for secondary analysis of the patient data. Copy file content and paste into MATLAB as a script or at the prompt to reproduce data analysis results.
About this article
Cite this article
Mukkamala, R., Kohl, B.A. & Mahajan, A. Comparison of accuracy of two uncalibrated pulse contour cardiac output monitors in off-pump coronary artery bypass surgery patients using pulmonary artery catheter-thermodilution as a reference . BMC Anesthesiol 21, 189 (2021). https://doi.org/10.1186/s12871-021-01415-5
- Cardiac output monitoring
- Pulse contour analysis