Investigating the Effectiveness of the Pre-anesthesia Induction Patient Safety (PIPS) Checklist: Comparison of Two Implementation Strategies
Event Type
Oral Presentations
TimeTuesday, April 132:00pm - 2:20pm EDT
LocationPatient Safety Research and Initiatives
Surgery and anesthesia departments regularly implement safety checklists to standardize care and improve safety. However, checklists are often designed and implemented ineffectively (Gillespie & Marshall, 2015). Many checklists are introduced by a single clinical “champion” and often without obtaining users’ buy-in or in a way that researchers describe as a “Moderately Effective” approach to checklist implementation (Conley et al., 2011). This “moderately effective” approach does not include any of the elements for successful implementation that have been identified in retrospective studies and large scale reviews and implementation manuals (Borchard et al., 2012; World Health Organization, 2009). Specifically, key components of checklist success include: multidisciplinary team communication, active leadership, deliberate enrollment, extensive discussion and training, piloting, real time coaching, ongoing feedback, and local modification.

In 2013, when the Anesthesia Patient Safety Foundation (APSF) recommended implementation of a consensus 11-item Pre-anesthesia Induction Patient Safety (PIPS) checklist ( Stoelting, 2013), we sought to determine, prospectively, if “Highly Effective” checklist implementation using the aforementioned key elements would yield improved and sustained compliance, acceptance, and outcomes. However, there are no prospective studies showing simultaneous implementation of the same checklist with and without these effective properties and their effects on checklist compliance or clinically relevant proximal outcomes. The primary purpose of the current study was to determine whether a “Dynamic” implementation of the PIPS checklist with a multidisciplinary committee of stakeholders created and empowered to gather user feedback and modify the checklist to “make it their own” would yield better outcomes than a “Static” approach with a fixed checklist implemented under the direction of a single clinical champion. Specifically:
Hypothesis 1: “Dynamic” implementation of the PIPS checklist will yield better outcomes than a “Static” approach with regards to (1a) compliance with checklist mandated behaviors, (1b) checklist related errors, (1c) checklist related near-misses, (1d) attitudes toward checklists, and (1e) team communication between anesthesia team members and other operating room personnel.

We conducted a prospective, pre-post, cohort trial observing simultaneous implementation of the same PIPS checklist in two hospitals within the same health system. Each hospital is similar in size (between 250-350 beds), employs a team-based model of care with similar staffing ratios, and offer similar services and resources. Further, both hospitals serve the same catchment area, and use the same surgical services rules and regulations. Participants were anesthesiologists and Certified Registered Nurse Anesthetists (CRNAs) from an anesthesia group practice.
At one hospital, we created a multidisciplinary committee of stakeholders to oversee implementation and provide active leadership, ongoing feedback, and local modification of the checklist (“Dynamic Implementation”; Hospital D). At the end of each post-implementation quarter, desired checklist modifications were incorporated into a new checklist (e.g., adding an item about blood availability, removing items about upper airway and NPO status, re-ordering some items, defining two items as ‘first case only’), which was printed and distributed to Hospital D at the start of the subsequent quarter. At the second hospital, we empowered a single clinical champion to implement the checklist in any way they desired except that the checklist itself could not be changed (“Static Implementation”; Hospital S).

The study proceeded for 12 months divided into four quarters. Quarter 1 observations were conducted before checklist implementation to establish baseline measurements and observations in Quarters 2-4 occurred after checklist implementation.

Checklist compliance, errors and near misses. We developed a structured, behaviorally anchored, rating instrument for observation of anesthesia induction to measure compliance, errors and near misses. Four observers were trained by an anesthesiologist. We assessed inter-observer agreement (Fleiss kappa = .91). We pre-defined checklist compliance with two components: a specific observable action and verbal affirmation. Observers coded each as binary (yes/no) for each checklist item. We recorded all errors, existing because of checklist non-compliance, at the time of induction (e.g., missing suction, unavailable backup airway devices) and noted consequences and actions taken during post-induction observation.

Team communication. Observers coded team communication as one or more of the following: Initiation (i.e., provider initiated checklist item completion without prompting such as “Let’s start the checklist”), Call Out (i.e., provider spoke aloud as checklist item was completed), Delegation (e.g., relegating task to team member such as “Go ahead and check that we have an ambu bag”), Verbal Request for Assistance (i.e., requesting further action from a colleague or coworker such as “Could you help me with this please?”) or Offer Assistance (i.e., providing assistance without explicit request such as “Would you like me to run through the checklist with you?”).

Attitudes toward checklists. We measured attitudes toward surgical safety checklists prior to Quarter 1 and again after each quarter via a self-report measure.

Barriers to effective implementation. At the end of each post-implementation quarter, we conducted semi-structured interviews about providers’ perceptions about PIPS checklist utility and implementation and the specific factors they believes helped or hindered them from using the checklist.

We observed 477 anesthesia induction events. The Dynamic implementation cohort demonstrated significantly greater checklist compliance compared to the Static implementation cohort in the first and third quarters after implementation: 90.6% vs 71.4% (p=.01) and 79.7% vs 54.2% (p<.001), supporting H1a. There were no significant differences between the two cohorts when comparing checklist related errors, near-misses, or attitudes toward checklists, indicating lack of support for H1b-H1d, although Hospital D had a significant within-hospital pre-post decrease in checklist-related errors from Q1 to Q2 (19.8% to 5.3%, p=0.02).

Team communication improved at both hospitals after implementation. This does not support H1e that proposed Hospital D would have a significantly greater improvement, but does indicate that the PIPS checklist is likely to result in more team-related communication for both implementation strategies.

Interviews showed that two of the major criticisms of the PIPS checklist, ambiguity and redundancy, were more effectively resolved with the “Dynamic” implementation. Barriers to use, such as time pressure and lack of team buy-in, were cited for both groups.

We found that a “Dynamic” approach to checklist implementation will likely result in greater compliance with checklist mandated tasks than a “Static” approach employing a single clinical champion and no modification of the checklist. This indicates that allowing organizations and employees more control over how mandatory checklists are implemented may help with buy-in and eventual use. More research should be conducted to grade the individual impact of each of the elements of a “Dynamic” implementation, and larger scale studies are needed to determine the full impact of the PIPS checklist on safety related outcomes such as near-misses.

Funding: (Institutional) Adventist Health System/Sunbelt, Inc. DBA Florida Hospital and The Joseph L. Riley Anesthesia Associates, Inc.

Borchard, A., Schwappach, D. L. B., Barbir, A., & Bessola, P. A. (2012). Systematic review of the effectiveness, compliance and critical factors for implementation of safety checklists in surgery. Annals of Surgery, 256, 925-933.

Conley, D. M., Singer, S. J., Edmondson, L., Berry, W. R., & Gawande, A. A. (2011). Effective surgical safety checklist implementation. Journal of the American College of Surgeons, 212, 873-879.
Gillespie, B. M., & Marshall, A. (2015). Implementation of safety checklists in surgery: A realist synthesis of evidence. Implementation Science, 10, 137-151.

Stoelting, R. K. (2013). APSF survey helps to establish pre-induction checklist. APSF Newsletter, 28(1), 11-13.

World Health Organization. (2009). Implementation manual WHO Surgical Safety Checklist 2009.