- Open Access
Problems of subgroup analysis in randomized controlled trial
BMC Anesthesiology volume 20, Article number: 186 (2020)
- The original article was published in BMC Anesthesiology 2019 19:175
Multiple subgroup analyses of the same data increase the risk of generating false positive findings. All outcomes and planned subgroup analyses should thus be prespecified and described in the original trial registry. When outcome changes during an ongoing trial seem justifiable, publications must disclose and explain such changes.
To the Editor:
The publication by Licker et al.  in BMC Anesthesiology raises several concerns. Foremost, all of the reported patients were included in a previous publication by the same group of authors . The principal findings of both publications and of another very recent publication by the same group of authors  are basically identical, that is, glucose-insulin-potassium (GIK) infusion before start of cardiopulmonary bypass (CPB) improves LV function after open heart surgery. This creates the impression that three independent investigations [1,2,3] have documented a benefit of GIK-infusion during on-pump cardiac surgery. However, the publication by Licker et al.  reports findings in a post hoc defined subgroup of patients already included and analyzed in the first publication assessing the identical outcome, although the title states that this is a secondary analysis of the previously reported randomized controlled trial .
The authors claim that analysis of this subset of patients was pre-planned. This is an incorrect statement because it fundamentally contradicts the trial registration which reads: “We will investigate the impact of short term GIK on the extent of myocardial injuries as well as on the left-ventricular systolic and diastolic function in 2 high-risk groups of cardiac surgical patients: Patients with cardiac dysfunction undergoing aortocoronary bypass surgery and patients with severe aortic stenosis.” . Furthermore, the authors define secondary study endpoints as” other TEE parameters as well as hemodynamic parameters”. However, the trial registration  lists as secondary outcome measures intraoperative systolic and diastolic cardiac function using transesophageal echocardiography (TEE), intraoperative hypo- and hyperglycemia and hypo- and hyperkalemia, 48 h postoperative serious cardiovascular adverse events (myocardial infarction, cardiac arrhythmia, low cardiac output, stroke), and intraoperative and 48 h postoperative serum troponin and creatine kinase concentrations. As a general principle, whenever changes from the original trial registration occur, they must be fully explained and reported to allow appropriate interpretation of results .
The authors neither mention nor critically discuss the considerable implications of a discrepancy between pre-specified and reported primary outcome measures. Accurate reporting of a pre-specified primary outcome in a subsequent publication is a critical component of clinical research, and incorrect reporting constitutes a major scientific flaw [6, 7]. Outcome misreporting increases the likelihood that reported differences are chance findings or are exaggerated . Therefore, all outcomes and planned subgroup analyses should uniformly be prespecified and described in the original trial registry, and confidence intervals be provided for all outcomes to indicate the precision (uncertainty) of the estimate [9, 10]. The CONSORT (Consolidated Standards of Reporting Trials) statement even discourages, in general, multiple subgroup analyses of the same data because of the increased risk of generating false positive findings . Unfortunately, reported primary outcomes in published randomized controlled trials frequently differ from those specified in the respective clinical trial registry [11, 12]. This includes publications in journals publicly endorsing the CONSORT guidelines on best practice in trial reporting which emphasize the importance of reporting all pre-specified outcomes [7, 12].
In the original publication , the a priori power calculation based on the primary outcome postcardiotomy ventricular dysfunction (PCVD) required 88 patients per group for adequate statistical power. Of those 88 patients, the present publication  reports selective findings in just 54 and 38 patients, respectively. In the absence of an a priori power calculation, it cannot be ruled out that the study is statistically underpowered. A statistical significance of interaction test was listed as one of 10 critical criteria for assessment of the credibility of a subgroup effect .
The CONSORT flow diagram was designed for the reporting of randomized trials . Inclusion of such flow diagram in the publication  makes the reader believe that isolated coronary artery bypass graft (CABG) surgery, the combination of CABG surgery and aortic valve replacement (AVR), and poor quality or no TEE were exclusion criteria as per protocol. As this was not the case, presentation of a CONSORT flow diagram is inappropriate in this context. It is equally inappropriate and misleading to start the Discussion section with the words, “In this randomized controlled trial ( …)” when this publication does definitely not report findings of a randomized controlled trial.
Several of the statistically significant differences between groups in the absolute values of TEE variables reported in the publication  lie within the accuracy and precision of the applied methodology and are thus of questionable clinical relevance. Furthermore, presenting the same data as percent changes is unwarranted duplication of identical data and exaggerates the observed small changes in absolute values. Interpretation is exclusively based on p-values, a practice that is increasingly being discouraged [5, 6, 9]. Confidence intervals are not provided.
In conclusion, the present publication  is based on a previous study by the same authors . It substantially overlaps in study hypothesis, study population, methods, data and conclusions, does not adhere to the pre-specified outcomes as defined in the trial registry, and does not introduce novel scientific aspects. Editors and reviewers can limit such shortcomings by, (a) ensuring that all results of an investigation are reported in a single comprehensive publication, (b) routinely consulting the trial registry when assessing a manuscript, and (c) demanding that authors report and analyze pre-specified outcomes [9, 10]. When outcome changes during an ongoing trial seem justifiable, publications must disclose and explain such changes .
Availability of data and materials
Consolidated Standards of Reporting Trials
postcardiotomy ventricular dysfunction
coronary artery bypass graft; aortic valve replacement
Licker M, Diaper J, Sologashvili T, Ellenberger C. Glucose-insulin-potassium improves left ventricular performances after aortic valve replacement: a secondary analysis of a randomized controlled trial. BMC Anesthesiol. 2019;19(1):175.
Ellenberger C, Sologashvili T, Kreienbuhl L, Cikirikcioglu M, Diaper J, Licker M. Myocardial protection by glucose-insulin-potassium in moderate- to high-risk patients undergoing elective on-pump cardiac surgery: a randomized controlled trial. Anesth Analg. 2018;126(4):1133–41.
Licker M, Reynaud T, Garofano N, Sologashvili T, Diaper J, Ellenberger C. Pretreatment with glucose-insulin-potassium improves ventricular performances after coronary artery bypass surgery: a randomized controlled trial. J Clin Monit Comput. 2020;34(1):29–40.
https://clinicaltrials.gov/ct2/show/NCT00788242. Accessed 21 Jan 2020.
Priebe HJ. Pre-specified outcomes must be followed. J Clin Monit Comput. 2020;34(1):187–8.
Schulz KF, Altman DG, Moher D, CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. BMJ. 2010;340:c332.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.
Sun X, Briel M, Buss JW, You JJ, Akl EA, Mejza F, Bala MM, et al. Credibility of claims of subgroup effect in randomized controlled trials: systematic review. Br Med J. 2012;344:e1553.
Sun X, Ioannidis JP, Agoritsas T, et al. How to use a subgroup analysis: users' guide to the medical literature. JAMA. 2014;311:405.
Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13:282.
Goldacre B, Drysdale H, Dale A, Milosevic I, Slade E, Hartley P, Marston C, Powell-Smith A, Heneghan C, Mahtani KR. COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time. Trials. 2019;20:118.
The author declares that he has not received any institutional/hospital/departmental funds or funds from any other funding agency/source. Open access funding provided by Projekt DEAL.
Ethics approval and consent to participate
Consent for publication
The author declares that he has no competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Priebe, H. Problems of subgroup analysis in randomized controlled trial. BMC Anesthesiol 20, 186 (2020). https://doi.org/10.1186/s12871-020-01105-8
- Trial registration
- Outcome reporting
- CONSORT guideline