Performance of the new SmartCardia wireless, wearable oximeter: a comparison with arterial SaO2 in healthy volunteers

Abstract

Background

Our goal was to evaluate the performance of a new wearable arm located pulse oximeter.

Methods

Twelve volunteers were monitored with three pulse oximeters and underwent desaturation to 70% SaO₂.

We compared the accuracy of SpO₂ reading from the SmartCardia system with SpO₂ using two well established devices (Masimo and Nellcor) as reference.

Oximetry was performed at different level of oxygen saturation varying from 70 to 100%. Bias, ARMS and precision were evaluated using Bland-Altman plots.

Results

The mean (SD) differences between SaO₂ compared to SpO₂ and the devices were as follows: SaO₂ versus Masimo 2,12 ± 1,01% (95% CI 1,45 to 2,79), SaO₂ versus Nellcor 0,78 ± 0,58% (95% CI − 0,29 to 1,65) and SaO₂ versus SmartCardia 0,42 ± 0,24% (95% CI − 0,64 to 1,46). The bias between SmartCardia, Masimo, Nellcor devices and SaO₂ was 0.16 (95% CI 0.05 to 0.33) and LoA (level of agreements) 2.96 (95% CI − 2,68 to 2,89) for SmartCardia, 2,02 (95% CI 1,49 to 2,54) and LoA − 6 to 11 for Masimo, and 0,76 (95% CI 0,5 to − 1) and LoA − 3,5 to 5,0 for Nellcor. ARMS for the 70–100% SaO₂ range was 1,4 for SmartCardia, 5,0 for Masimo and 2,31 for Nellcor.

Conclusions

The new wireless SmartCardia SpO₂ measurement system demonstrated in-line results, bias, ARMS and precision in healthy volunteers, when compared with the gold standard SaO2 and with two well established systems, Masimo and Nellcor.

Trial registration

The present trial was prospectively registered at UCSF record (registration number:10–00437), on March 8, 2021.

Introduction

Pulse oximetry is a procedure for measuring the level of oxygen in the blood expressed as oxygen saturation [1,2,3]. This parameter is widely used in the evaluation of various medical conditions that affect the function of the heart and lungs and is a standard monitoring modality during anesthesia delivery [4,5,6].

Conventional oximeters use either the reflective or the transmitive method to measure SpO2. The reflective method is known to be less accurate. For this reason, most commercially available patches do not provide SpO2 measurements in addition to cardiac monitoring. The transmissive method is the more commonly used of the two. Transmitive technology transmits red and infrared light through the finger to a photo detector. The pulse oximeter sensor reads the transmitted light beams to estimate oxygen saturation of the blood and pulse rate. This method positions the transmitter and receiver in the same plane and the sensors can be placed on areas of the anatomy other than the finger. Conventional pulse oximeters have various limitations including false alarm and failed measurements [7,8,9,10,11] which may be related to positioning of the device located on the finger. A new generation of reflective pulse oximetry device (SmartCardia oximeter) has been designed on a wireless basis [12, 13]. The present trial was designed to validate this technology and to demonstrate the potential of this device using the reflective method. We compared this new device to the Nellcor N-600 (Covidien, Boulder, CO, USA) and the Masimo Radical device (Masimo Corp., Irvine, CA, USA), both units considered the two most used and accurate FDA approved non-invasive devices for SpO₂ measurements. SaO2 was used as the gold standard.

Methods

Ethical considerations

Even if the device was developed in Switzerland the trial was conducted in a reference laboratory accepted by the FDA, the Hypoxia Research Laboratory at UCSF. The study was reviewed and approved by the UCSF Ethical Committee on Human Research. The Approval number is 10–00437, expiring on March 8th, 2021. The approval letter is on file at UCSF. The laboratory conforms to Good Clinical Practice Standards for the involvement of human subjects and handling of test data. Written informed consent was obtained from each participant.

System description

The SmartCardia 7 L device is a wireless patch with a low-cost disposable component and a re-usable electronic unit (Fig. 1). The patch acquires ECG and measures heart rate (HR) and SpO₂. The data are transmitted by Bluetooth to a mobile phone or router. The measured signals and parameters are also stored on the device. The device for SpO₂ recording is placed on the left arm of the subject (Fig. 1). The size of the patch is 55 × 130 mm. The patch-based device offers up to 14-days monitoring and data storage and 7-days real-time connectivity with the cloud through a smartphone. The ability to receive, store, and interpret a broad range of signals offers the opportunity to go far beyond monitoring individual parameters. If the patient’s vital measurements reach a pathological value, the system gives an alert on the cloud and the physician can see the real-time parameters and ECG signals.

Fig. 1

SmartCardia device especially dedicated for SpO2 measurement

Protocol

The protocol required brief stable arterial oxygen desaturation in healthy volunteers and sampling arterial blood when a stable level of hypoxia has been attained. The blood sample was analyzed for oxygen saturation with a gold standard bench CO-oximeter, currently a Radiometer ABL-90 multi-wavelength oximeter (Hemoximeter, Radiometer, Copenhagen (Denmark), serial 1393-090R0359N0002). This instrument contains factory certified calibration standards and quality control algorithms. Twenty to twenty-five arterial blood samples from each subject can be analyzed following a protocol aligned with current ISO and FDA guidance documents for pulse oximeter testing.

A radial arterial cannula was placed in either the left or right wrist of each subject for arterial blood sampling and blood pressure monitoring.

Our approach to obtaining stable, safe, and controlled hypoxia was breath-by-breath respiratory gas analysis. A computer program permits the inspired gas mixture to be adjusted to achieve a level of lung alveolar gas that will achieve the desired degree of hypoxia. This was obtained with the use of a nonerebreathing circuit with CO2 removal. Typically, saturation is determined once with air breathing and then at each of the following levels, e.g., 93, 90, 87, 85, 82, 80, 77, 75 and 70% saturation for about 30–60 s at each level. An arterial blood sample is obtained from the catheter at the end of each hypoxic plateau. The operator changes the inspired oxygen concentration at the end of each blood sampling to attain the next desired steady-state hypoxic condition. A run takes 10–15 min, and each run is terminated by a breath of 100% O₂ followed by room air. Two runs together enable obtaining a total of 20–25 blood samples, two samples at each different plateau. Saturation of each arterial blood sample is determined by direct oximetry using the Radiometer ABL-90 multi- wavelength oximeter.

Sites for affixing pulse oximeter probes was the fingers for the Masimo device, the ears for the Nellcor, and the arm for the SmartCardia. In order to avoid inaccurate readings due to head or fingers movements with these 2 devices, the arm of the subjects was fixed during the measurements period. In addition, the participants were asked not to move their heads, as much as possible for the duration of the measurements.

Statistics

Pulse oximeter data was taken as 5 s averages corresponding to the point of SaO₂ analysis. Individual data points may be missed or excluded for dropped signals or failure of the oximeter signal to achieve an appropriate plateau. Agreement in SpO₂ and arterial SaO2 were analyzed using Bland Altman analysis [14, 15]. Bland Altman curve gives a graphical representation of the agreement between SpO₂ and arterial SaO2 value. The average of the SpO₂ and SaO₂ is plotted on the x-axis while the difference between the two values is plotted on the y-axis. The more the graph points toward a zero difference with narrow dispersion of the ‘body’, the better the agreement.

Tables of mean, standard deviation, standard error, minimum, maximum, 95% CI, count and root mean square are provided for each oximeter’s bias, and all oximeters combined in the following ranges of SaO2: 70–100%, 60–70%, 70–80%, 80–90%, and 90–100%. LoA was obtained between methods of measurement with multiple observations per individual [16]. Root mean square error (ARMS) was calculated as it represents the best way to compare SaO2 measurements. It includes both values of SaO2 and its stability. The following formula was used for calculation:

$$\mathrm{ARMS}=\sqrt{\frac{\sum {\left({\mathrm{SpO}}_2-{\mathrm{SpO}}_2\right)}^2}{\mathrm{n}}}$$

Data were managed on MS excel spreadsheet and analysed using stata 9.0 software (Stata Corp, College Station, TX). On Bland Altman curves linear regression is shown for all subjects combined and the equation with R2 is shown on the plot. Mean bias is displayed as a solid horizontal line, and the upper and lower limits of agreement (mean bias ±1.96•SD*) are shown by dashed horizontal lines. For “pooled” plots, different markers are used for each pulse oximeter. Continuous variables were compared using ‘t’ test. A p value < 0.05 was considered as significant.

Results

The current study was performed on 4/13/2021 and 4/14/2021 at UCSF. Twelve volunteers with a mean age of 28 (range 21–29) (3 women and 9 men) were included in the trial (Table 1). All subjects enrolled in the study had normal Hb level (Hb ≥ 10 g/l) and all were healthy and non-smoking individuals.

Table 1 Mean ± SD SpO2 values at different plateaus with the 3 oximeters for the 12 subjects. Mean differences between the 3 oximeters are shown as delta

A total of 298 samples were obtained at the saturation plateaus during the study. Data were collected for each type of oximeter and probes. Mean (SD) differences measurements between SaO₂ and SpO₂ and between the devices tested were as follow: SaO₂ versus Masimo 2,12 ± 1,01% (95% confidence interval CI, 1,45 – 2,79), SaO2 versus Nellcor 0,78 ± 0,58% (CI, − 0,29 - 1,65) and SaO2 versus SmartCardia 0,42 ± 0,24% (95% CI, − 0,64 - 1,46). A statistically significant difference in precision was observed between the SmartCardia and the Masimo device (p < 0.001), between the SmartCardia and the Nellcor device (p = 0.03) and between the Nellcor and the Masimo (p < 0.001) (Table 1 and Fig. 2).

Fig. 2

Plots showing agreement between SaO2 (H) and SpO2 detected by the SmartCardia device (SC), the Masimo device (Ma) and the Nellcor (N). Results +/− SD are presented as the difference between the device and the gold standard

Bias, precision and ARMS are presented in Table 2. Plots assessing the agreement between arterial SaO₂ and SpO₂ detected by the SmartCardia, the Masimo and the Nellcor pulse oximeters are shown in Fig. 2. The bias between SmartCardia, Masimo, Nellcor and SaO₂ was as follows: 0,16 (95% CI 0.05 to 0,33) and the LoA were 2,96 (95% CI − 2,68 to 2,89) for SmartCardia, 2,02 (95% CI 1,49 to 2,54) and the LoA were − 6 to 11 for Masimo, 0,76 (95% CI 0,5 to − 1) and the LoA were − 3,5 to 5,0 for Nellcor. This bias is significantly lower for SmartCardia compared to Masimo (p < 0.001) and Nellcor (p < 0.05), and for Nellcor compared with Masimo (p < 0.05). ARMS for the 70–100% SaO₂ range was 1,4 for the SmartCardia device, 5,0 for the Masimo device and 2,31 for the Nellcor device (Table 2 and Fig. 3).

Table 2 Bias and ARMS at different oximetry ranges with the 3 oximeters compared with SaO2

Fig. 3

Bland-Altman curve showing agreement between a: SmartCardia, b: Masimo and c: Nellcor, SpO2 and arterial SpO2. Data is plotted as Hemoximeter data (SaO2) vs. pulse oximeter bias (SpO2 – SaO2). A different marker is used for each study subject. Linear regression is shown for all subjects combined, and the equation with R2 is shown on the plot. Mean bias is displayed as a solid horizontal line, and the upper and lower LoA (mean bias ±1.96•SD*) are shown by dashed horizontal lines

Discussion

Currently, arterial oxygenation is primarily measured using pulse oximeters that provide immediate and continuous non-invasive surveillance for SpO2. Several devices are available in clinics with some limitations, including false alarms and failed measurements when used in patients with low perfusion or during patient motion. The positioning of the fingertips makes measurements at times difficult and impractical for patients. Some trials showed varying sensitivity and accuracy in patients with hypotension and hypoxemia [14, 16], primarily due to the location of the device. We compared the performance of two well-established pulse oximeters to a next-generation pulse oximeter, the SmartCardia device. This wireless system is located on the patient’s arm and the values are transmitted electronically and stored in the cloud, both representing major benefits over other devices. Furthermore, today, as a result of the COVID-19 pandemic, the measurement of SpO2, even outside the hospital, has become increasingly important. We have shown that despite the use of the usually less accurate reflective method, the SmartCardia device provides very precise and precise results when compared to well-established devices in clinical use. Limitations due to the presence of wired connections are avoided, and patient hand movement is not limited by the presence of a finger mounted sensor. Furthermore, the oximeter can be worn during walking with permanent transmission of data to a control station or a phone providing reliable electronic wifi transmission. This allows access to permanent and uninterrupted information regarding patient oxygenation levels. To confirm the quality of the device, which is currently awaiting FDA approval, its performance was compared with the Masimo and Nellcor pulse oximeter devices. Both use the transductive method. In our study, the results are encouraging and equivalent or even slightly better based on bias, ARMS and precision measurements. We also showed that this system provides good SpO₂ assessments over the range of SaO₂ tested between 70 to 100% oxygen saturation.

Our trial involved young healthy volunteer subjects in a reference laboratory setting. This allowed the use of a more rigorous protocol while maintaining control over variables.

As mentioned before, this system is placed on the subject’s arm which is an advantage due to fewer disturbances related to the hand’s movements. Devices placed on the ear are also more prone to disturbances related to movements of the patient’s head. We found significant performance differences among the 3 devices tested. We showed that the Masimo device tended towards 2 to 3% higher SpO2 values than actual SaO₂. This may be clinically relevant if confirmed in ICU patients. As our trial was not performed in a clinical setting, we cannot conclude that one oximeter would be better than the others in daily practice. Further trials in real life situations, such as ICU environments, hypotensive or hypothermic conditions, etc. are needed to further draw definitive conclusions. It is our experience that even well simulated clinical scenarios often lead to ambiguous results. We observed a significant difference in the measurement of SpO2 between the Nellcor and the Masimo in favor of the Nellcor system. However, our data were collected in healthy volunteers. Furthermore, previous trials comparing the Masimo, the Nellcor and the Philips Intellivue have not shown significant differences between these three pulse oximeters when tested in ICU patients [16, 17]. Conversely, other trials conclude to a higher sensitivity and specificity of the Masimo device when compared to the Nellcor system [18]. These divergent results clearly show the need for further studies utilizing real life environments and patient’s conditions to further corroborate our results.

Study limitation

The major limitation is that we performed this trial in a laboratory. This may not fully replicate the characteristics of complex clinical settings exhibited by patient populations in the ICU or operating theater. In addition, the trial was conducted with a small number of patients. These results need confirmation in a larger cohort of subjects in a clinical setting.

Conclusion

The new wireless SmartCardia SpO₂ measurement system demonstrated in-line results, bias, ARMS and precision in healthy volunteers, when compared with the gold standard SaO2 and with two well established systems, Masimo and Nellcor.