Institutional Review Board approval was obtained from the Lebanese American University and from the University of Illinois at Chicago. Participants were informed that their participation was voluntary; results were anonymized. Written consent was obtained prior to the educational intervention.
Participants
All PGY3 and PGY4 Lebanese anesthesiology residents were invited to participate in this education intervention. PGY1 and PGY2 residents were not recruited because of the minimal likelihood of BBN encounters at their stage of training. A pre-workshop survey focused on residents’ previous training and experiences with BBN and their self-perceived competence and comfort in BBN, as well as their perspectives about the need for associated training programs (Additional file 1).
Delivery of the education intervention
The four-hour session was conducted at the Clinical Simulation Center and was organized by two anesthesiologists, a clinical psychologist and a simulation educator. Following an orientation to the simulation setting, participants were randomly divided into groups of four. Each group participated in an immersive experience (case 1, described below) with the high fidelity simulator (HFS) (iStan, METI, CAE). This was followed by an individual encounter with an SP. After all residents participated in case 1, they participated in a teaching intervention. The intervention consisted of short lectures provided by the anesthesiologist and the clinical psychologist, role-play and group discussions. Both educators facilitated the debriefing and encouraged the participants and SPs to reflect on what went well in the conversation and to make suggestions for improvement. Following this teaching intervention, all participants were again randomly divided into groups of four and each group participated in case 2 (described below), followed by an individual encounter with an SP (Table 1).
All encounters with family members were conducted in Arabic, the national language and videotaped. At the end of the session, participants completed a survey about their perceived ability and comfort in BBN (Additional file 2).
The scenarios
Two scenarios describing situations of unexpected intraoperative anesthetic complications were developed by experienced clinicians (anesthesiologists), assisted by experts in simulation-based medical education (LAU-CSC educators) and a clinical psychologist. Both scenarios were based on the complication being from a purely anesthetic origin and not from the surgery. Intraoperative death from a purely anesthetic complication is an extremely rare event. The most common encounter for BBN for an anesthesiologist is caused by intraoperative complications that lead to sending the patient to the intensive care unit. Both cases occurred during a routine anesthesia induction in a healthy patient.
In case one, a young patient admitted for a surgery of the knee developed an anaphylactic reaction during the induction of anesthesia. In case 2, a middle-aged patient admitted for back surgery developed a sudden onset of atrial fibrillation after the induction of anesthesia. In both cases, and following the resuscitation efforts of the participants, the patient’s critical state was stabilized, which allowed his transfer to the intensive care unit. Each participant was then asked to inform a patient’s family member about this unexpected complication.
The assessment instrument
The GRIEV_ING is a 27-item instrument developed to focus on 8 competency areas concerning death notification (Additional file 3). This instrument can be divided into subscales that measure different components of the BBN skill: preparation, delivery and wrap-up [17]. After a thorough review by the organizing team, the general structure of this checklist was maintained, while some descriptors were modified to accommodate the notification of an adverse event or death in the anesthesiology context. This modification of the instrument was followed by piloting a few scenarios of BBN with SPs; the wording was further modified, based on feedback. The checklist was also sent to three anesthesiology program directors to ensure its applicability for assessing BBN skills. In addition to this GRIEV_ING checklist assessment form, participants were also assessed for communication skills by the SPs using a global rating instrument that was adapted from the GRIEV_ING Death Notification Protocol” (Additional file 4).
The raters
Five SPs were trained by the organizing team. SPs participated in a two-hour training session for their roles as family members and a four-hour training and calibration session to prepare them for their rater role. Immediately preceding the workshop, SPs participated in a review session to ensure that all instructions were clear. The two anesthesiologists independently rated the videotaped encounters using the GRIEV_ING checklist assessment form.
Study design
Residents’ skills in BBN were assessed with an SP encounter at two separate time points: before the teaching intervention with case 1 (anaphylaxis) and immediately post-intervention with case 2 (cardiac event). Residents were not assessed on the way they managed the intra-operative events, but only on the way they delivered the bad news to the family member. However, they were not informed about this fact to preserve the realism and the total immersion required by our intervention. We chose to use two different cases, one at time 1 and the second at time 2, for a number of reasons. First, given the educational intervention, it would be difficult, logistically, to conduct the 2 different simulations at the same time point. Second, and more important, we did not want our results to be confounded by case familiarity. While the patient presentation and associated management was different at the 2 time points, the resident task, breaking bad news, was the same. As such, the two assessments, before and after the intervention, were considered to be of equivalent difficulty.
Our primary hypothesis was that, compared to baseline, residents would perform significantly better on the skills associated with BBN after the educational intervention. Considering the GRIEV_ING total checklist score, a paired sample t-test (β = .2, α = .05), and a medium effect size of 0.5, our required sample size would be 25.
The assessments
Standardized patients completed both the modified GRIEV_ING checklist and the communication global rating assessment. The GRIEV_ING checklist is a binary assessment tool with equal credit given for each item performed. The communication global rating was provided on a scale from 1 = poor to 5 = excellent. To measure inter-rater reliability, six encounters from case 1 and from case 2 were randomly sampled and rated by the two anesthesiologists and the psychologist.
Data analysis
The intraclass correlation coefficient (ICC) was used to evaluate inter-rater reliability of the GRIEV_ING scores for each case, both by dimension and overall. Raters scored each GRIEV_ING assessment item on a yes/no basis (1 = performed item, 0 = did not perform item). A total score for each case was calculated by summing all items. The percentage of performed items was also calculated for all residents. The internal-consistency reliabilities (Cronbach alpha) for the communication global rating and the GRIEV_ING checklist were calculated for both cases. A paired-samples t-test was used to compare changes in mean trainee scores before and after the intervention. This was done separately for the total checklist scores and for the global communication rating scale. Pearson’s correlation coefficient was used to quantify the association between the GRIEV_ING checklist scores and the communication global rating scores.
Survey data were summarized by frequency counts and means. Data were analyzed using the Statistical Package for Social Sciences (SPSS) version 21.