Perceptions about the relative importance of patient care-related topics: a single institutional survey of its anesthesiologists, nurse anesthetists, and surgeons

Abstract

Background

Persistently variable success has been experienced in locally translating even well-grounded national clinical practice guidelines, including in the perioperative setting. We have sought greater applicability and acceptance of clinical practice guidelines and protocols with our novel Perioperative Risk Optimization and Management Planning Tool (PROMPT™). This study was undertaken to survey our institutional perioperative clinicians regarding (a) their qualitative recommendations for (b) their quantitative perceptions of the relative importance of a series of clinical issues and patient medical conditions as potential topics for creating a PROMPT™.

Methods

We applied a mixed methods research design that involved collecting, analyzing, and “mixing” both qualitative and quantitative methods and data in a single study to answer a research question. Survey One was qualitative in nature and asked the study participants to list as free text up to 12 patient medical conditions or clinical issues that they perceived to be high priority topics for development of a PROMPT™. Survey Two was quantitative in nature and asked the study participants to rate each of these 57 specific, pre-selected clinical issues and patient medical conditions on an 11-point Likert scale of perceived importance as a potential topic for a PROMPT™. The two electronic, online surveys were completed by participants who were recruited from the faculty in our Department of Anesthesiology and Perioperative Medicine and Department of Surgery, and the cohort of hospital-employed certified registered nurse anesthetists.

Results

A total of 57 possible topics for a PROMPT™ was created and prioritized by our stakeholders. A strong correlation (r = 0.82, 95 % CI: 0.71, 0.89, P < 0.001) was observed between the quantitative clinician survey rating scores reported by the anesthesiologists/certified registered nurse anesthetists versus the surgeons. The quantitative survey displayed strong inter-rater reliability (ICC = 0.92, P < 0.001).

Conclusions

Our qualitative clinician stakeholder survey generated a comprehensive roster of clinical issues and patient medical conditions. Our subsequent quantitative clinician stakeholder survey indicated that there is generally strong agreement among anesthesiologists/certified registered nurse anesthetists and surgeons about the relative importance of these clinical issues and patient medical conditions as potential topics for perioperative optimization and risk management.

Background

Persistently variable success has been experienced in locally translating even well-grounded national clinical practice guidelines, including in the perioperative setting [1, 2]. Recurrent barriers to health care providers adopting such guidelines include inadequate understanding, lack of agreement and perceived “real-world” practicality, clinicians’ concerns about loss of self-efficacy, low outcome expectations, and the inertia of existing practice [3]. Applicability can be enhanced by adapting clinical practice guidelines within the local context, and their acceptence can be improved by assessing barriers to their use [4].

At the University of Alabama at Birmingham (UAB), we have sought to achieve greater applicability and acceptance of clinical practice guidelines and protocols with our novel Perioperative Risk Optimization and Management Planning Tool (PROMPT™). A PROMPT™ is a merger of available published material with the equally valued expertise and consensus of institutional clinicians to arrive at current best practice. A PROMPT™ can serve as a real-time, front-line health informatics decision support tool, which can be applied by the health care team in their day-to-day operations and facilitate patient-specific, personalized care planning, decisions, and implementation. A PROMPT™ is not a static document but instead is essentially predicated on an iterative series of Plan-Do-Study-Act (PDSA) cycles that incorporate newly published data, concurrent institutional-level outcomes data, and local clinician feedback (Fig. 1). A prototypic PROMPT™ is the one we have developed at UAB for the management of postoperative nausea and vomiting (Fig. 2).

Fig. 1

Iterative series of Plan-Do-Study-Act (PDSA) cycles applied to develop, implement, and revise a Perioperative Risk Optimization and Management Planning Tool (PROMPT™)

Fig. 2

Prototypic Perioperative Risk Optimization and Management Planning Tool (PROMPT™) for the management of postoperative nausea and vomiting

While its efficacy has yet to be validly demonstrated, the PROMPT™ is intended to complement and to strengthen a surgical procedure-specific integrated care pathway. A patient-tailored “Perioperative Personalized Care Matrix” can be created by the amalgamation of all of the standardized elements of a surgical procedure-specific integrated care pathway and any applicable, patient condition-specific PROMPT™ (Fig. 3) [5]. This Perioperative Personalized Care Matrix is an integral component of our Perioperative Surgical Home model at UAB.

Fig. 3

Perioperative Personalized Care Matrix comprised of both elements of an Integrated Care Pathway (ICP) and any applicable Perioperative Risk Optimization and Management Planning Tool (PROMPT™) [5]

However, at the outset of our effort to develop, implement, and study our PROMPT™ concept, institutional stakeholder identification and prioritization of topics for a PROMPT™ were needed. A mixed methods research design involves collecting, analyzing, and “mixing” both qualitative and quantitative methods and data in a single study to answer a research question [6–8]. This mixed methods study was thus undertaken to survey our institutional perioperative clinicians regarding (a) their qualitative recommendations for (b) their quantitative perceptions of the relative importance of a series of clinical issues and patient medical conditions as potential topics for creating a PROMPT™.

Methods

This continuous quality improvement (CQI) study was formally reviewed by the UAB Institutional Review Board (IRB) (E110311001). This study was approved under “Exempt” status [as defined by United States Department of Health and Human Services (45 CFR 46.101(b)(2)] by the UAB IRB, because the research involved the use of survey procedures in which the information obtained was recorded in such a manner that human subjects could not be identified, directly or through identifiers linked to the subjects. A waiver of signed written informed consent documentation was also granted by the UAB IRB. Per the UAB IRB, informed consent was instead obtained from all clinician study subjects prior to participation via their affirmative response on the initial page of the two online electronic surveys.

Survey design

We administered two electronic, online clinician surveys.¹ Survey One was qualitative in nature and asked the study participants to list as free text up to 12 patient medical conditions or clinical issues that they perceived to be high priority topics for development of a PROMPT™.

A “summative approach” to qualitative content analysis was then applied, in which the textual material was approached as single keywords, which were categorized based upon a review of the literature (i.e., conventional clinical sub-categories). This analysis of the patterns generated an interpretation of the contextual meaning of the specific terms (medical conditions or clinical issues) [9].

The free-text responses from this initial qualitative survey were hence grouped into the five categories of “Preoperative Testing,” “Medications and Technologies,” “Optimization of Co-Morbid Diseases,” “Perioperative Risk Reduction,” and “Provision of Comprehensive Care.” A final roster of 57 total possible topics for a PROMPT™ was created.

Survey Two was quantitative in nature and asked the study participants to rate each of these 57 specific, pre-selected clinical issues and patient medical conditions on a 11-point Likert scale [10, 11] of perceived importance as a potential topic for a PROMPT™. This 11-point Likert scale of perceived importance ranged from 0 = Not At All Important to 10 = Extremely Important. Once an importance rating score was electronically assigned to a given clinical issue or patient medical condition, the survey respondent could not return to that item.

Survey validity and reliability testing

The two administered study surveys were questionnaires about clinician attitudes. The content and face validity of the two surveys were assessed and agreed upon by two experienced anesthesiologists (T.R.V. and K.A.J.). The inter-rater reliability of our quantitative clinician survey was assessed with an intraclass correlation coefficient (ICC) [12] between the importance scores for three generic topics, which were placed at the very beginning and at the very end of the list of the other 57 specific clinical issues and patient medical conditions. The individual clinicians’ paired ratings of the level of importance were significantly correlated for “hand washing in the operating room” (ICC = 0.93, P < 0.001), “patient hand-off communications” (ICC = 0.92, P < 0.001), and “first case on-time start rate” (ICC = 0.92, P < 0.001).

Survey administration

Potential clinician study participants were recruited from all of the faculty members in the UAB School of Medicine Department of Anesthesiology and Perioperative Medicine and Department of Surgery, as well as the entire cohort of certified registered nurse anesthetists employed by UAB Hospital. A master list of anesthesiologists, surgeons, and certified registered nurse anesthetists was created using our institutional electronic directory.

These identified anesthesiologists, surgeons, and certified registered nurse anesthetists were then invited to participate in this study via an e-mail from the principal investigator (T.R.V.). The e-mail described the purpose of the study and provided the recipient with a hyperlink to the sequential online electronic qualitative and quantitative surveys located on surveymonkey.com. To maximize the survey response rate, two qualitative survey email invitations and subsequently three quantitative survey email invitations were sent at 10 day intervals to all potential study participants. The clinician survey responses were completely anonymous. The same participant pool was used for the qualitative survey and subsequent quantitative survey; however, because the clinician responses were anonymous, the degree of overlap could not be determined.

Statistical methods

For the purposes of statistical analysis, because of our anesthesia care team approach at UAB, the quantitative survey responses from the anesthesiologists and certified registered nurse anesthetists were combined.

Continuous variables were reported using mean and standard deviation, or if the data were skewed, as a median and interquartile range. Categorical variables were reported using frequency counts and percentages. Parametric continuous demographic data were compared between groups using a t-test. Categorical demographic data were compared between groups using a Chi-square test or Fisher’s exact test. When indicated, a 95 % confidence interval was calculated for a point estimate.

A Pearson correlation coefficient was calculated and a scatterplot generated for the aggregated quantitative survey rating scores for each clinical issue and patient medical condition reported by the anesthesiologists/certified registered nurse anesthetists versus the surgeons.

The survey respondents’ self-reported quantitative clinician survey scores demonstrated a floor effect and ranged from 5.4 to 9.4 on the 11-point Likert scale of perceived importance. We established 20 % of this absolute score range of 4.0, equal to 0.8, to be a minimally important score difference. Of the 57 specific, pre-selected clinical issues and patient medical conditions, those were identified for which there was a mean difference of ≥ 0.8 between the anesthesiologists/certified registered nurse anesthetists and the surgeons. The aggregated quantitative clinician survey rating scores for each of these nine clinical issue and patient medical condition were compared between the anesthesiologists/certified registered nurse anesthetists versus the surgeons with an unpaired t-test.

Continuous data were assessed for normality with a Shapiro-Wilk test and by examining Q-Q plots, and if non-parametric, they were analyzed as such. No a priori sample size determination and power analysis was performed. For all univariate data analyses, a P-value of < 0.05 was considered significant. Statistical analyses were performed using SAS® Version 9.3 (SAS Institute Inc., Cary, NC).

Results

Demographics of clinician survey respondents

There were no statistically significant differences in the demographics of anesthesia care provider and surgeon survey groups (Table 1).

Table 1 Study participant demographics

Clinician qualitative survey responses

Of the 289 recruited anesthesiologists, certified registered nurse anesthetists, and surgeons, 64 (22 % response rate) completed the qualitative survey, between July 2014 and August 2014. Based upon the qualitative survey responses, a roster of 57 total clinical issues and patient medical conditions was created (Table 2).

Table 2 Perioperative Risk Optimization and Management Planning Tool (PROMPT™) qualitative survey results

Clinician quantitative survey responses

Of the 72 recruited anesthesiologists, 33 completed the quantitative survey (46 % response rate); of the 161 recruited surgeons, 35 completed the survey (22 % response rate); and of the 47 recruited certified registered nurse anesthetist, 15 completed the survey (32 % response rate), between September 2014 and October 2014. The aggregated quantitative clinician survey rating scores for all 57 clinical issues and patient medical conditions are reported in Table 3.

Table 3 Specific clinical issues and patient medical conditions sorted by descending order based on all participants’ mean importance score (0 = “Not At All Important,” 10 = “Extremely Important”)

A strong correlation (r = 0.82, 95 % CI: 0.71, 0.89, P < 0.001) was observed between the quantitative clinician survey rating scores reported by the anesthesiologists/certified registered nurse anesthetists versus the surgeons (Fig. 4). However, the scores for nine specific clinical issues and patient medical conditions met the minimally important score difference of 0.8. The scores for eight of these specific clinical issues and patient medical conditions were also statistically significantly different between the anesthesiologists/certified registered nurse anesthetists versus the surgeons. These nine items are rank-ordered by standardized difference score in Table 4.

Fig. 4

Scatterplot observed between the quantitative clinician survey rating scores reported by the anesthesiologists/certified registered nurse anesthetists versus the surgeons (r = 0.82, 95 % CI: 0.71, 0.89, P < 0.001)

Table 4 Specific clinical issues and patient medical conditions significantly differently rated by the anesthesia care providers (anesthesiologists/certified registered nurse anesthetists) versus the surgeons (0 = “Not At All Important,” 10 = “Extremely Important”)

Discussion

This mixed methods study was undertaken to survey our institutional perioperative clinicians regarding (a) their qualitative recommendations for (b) their quantitative perceptions of the relative importance of a series of clinical issues and patient medical conditions as potential topics for creating a PROMPT™. Our qualitative data generated a comprehensive roster of 57 clinical issues and patient medical conditions. Our quantitative results indicate that there is generally strong agreement among anesthesiologists/ certified registered nurse anesthetists and surgeons about the relative importance of these clinical issues and patient medical conditions as potential topics for perioperative optimization and risk management. The nine clinical issues and patient medical conditions, for which there were significantly divergent prioritization, likely represent topics for which further efforts at achieving stakeholder consensus are indicated.

In our prior experience with instituting a standardized protocol for eye care management during anesthesia to prevent corneal injury, there was a significant initial reluctance among practitioners to comply with the proposed protocol [13]. Practice change did not fully occur and become part of our culture until outcomes data, showing efficacy, were repeatedly provided to practitioners. Lack of input from the involved practitioners in the development of this earlier protocol was thought to be a reason for their initial resistance to change practice.

Our present approach to continuous quality improvement thus began with “Start with what.” With our PROMPT™ concept, we first sought input from practitioners to determine what they perceived to be the most important topics in the care of our patients. This prioritization can now be used as a reason why a particular best practice change was selected—thereby hopefully increasing initial buy-in (i.e., “Start with Why”) [14]. This will include identifying and capitalizing upon crucial clinician motivators for performance improvement.

Nevertheless, with introduction of the PROMPT™ concept and a PROMPT™ focused on postoperative nausea and vomiting (Fig. 2), we have encountered initial attending physician pushback, mainly an unwillingness to practice “cookbook medicine.” However, a PROMPT™ is not prescriptive, “cookbook medicine” but instead serves as a best practice-based decision support tool. The successful development and implementation of a PROMPT™ requires not only robust informatics, analytics and decision support but also sustained robust grassroots clinician input and “buy-in.”

In addition to clinicians’ perceived importance of a given clinical issue or patient medical condition, other factors should be considered when prioritizing PROMPT™ topics for development and implementation. For example, we have observed that when the level of evidence to support a given PROMPT™ includes robust randomized clinical trials, systematic reviews, and meta-analyses, the degree of initial clinician buy-in can be more substantial. The prioritization of PROMPT™ topics may also be modified by practical matters, including the required resources, the complexity of the PROMPT™, and local institutional political considerations.

Limitations

One weakness of our study was the relatively low overall qualitative survey response rate (22 %) and overall quantitative survey response rate (30 %), potentially undermining the validity of our findings due to non-response bias. It is likely that this minority of survey respondents were more motivated to describe the issues that they felt were important. Only their identified set of issues was in turn scored in the second phase of the study. However, non-response bias may be less of a concern in physician surveys than in surveys of the general public [15]. Likewise, higher physician survey response rates have not been associated with lower response bias [16].

Based on Rogers’ Diffusion of Innovation Theory [17], it is likely that our minority survey respondents are highly engaged Innovators and Early Adopters, and the majority non-respondents are Early Majority, Late Majority, and Laggards. Such early adopters could serve as “thought leaders” who subsequently increase buy-in from the whole group.

Our study might have been strengthened if we had more clearly specified that the clinician survey respondents rank the 57 items (clinical issues and patient medical conditions) from their own provider-perspective rather than from the patient perspective or the institutional/financial perspective.

The results of our clinician survey nevertheless may effectively serve as the impetus for achieving greater local stakeholder consensus on how to prioritize a series of clinical issues and patient medical conditions as topics for a PROMPT™.

Conclusions

Our qualitative clinician stakeholder survey generated a comprehensive roster of clinical issues and patient medical conditions. The results of our subsequent quantitative clinician stakeholder survey indicate that there is generally strong agreement among anesthesiologists/certified registered nurse anesthetists and surgeons about the relative importance of these clinical issues and patient medical conditions as potential topics for perioperative optimization and risk management. We intend to undertake a future research to demonstrate if rank ordering problems department-wide and working on them together increases compliance with a specific PROMPT™.