Assessing Surgical Teamwork Competencies During Moments of Uncertainty, and their Relationship to Adverse Events, Using OR Black Box
Event Type
Oral Presentations
TimeTuesday, April 133:50pm - 4:10pm EDT
LocationPatient Safety Research and Initiatives
DescriptionStudying Teamwork and Uncertainty to Improve Patient Safety

Despite 20 years of focused research, preventable adverse events (AEs) continue to occur. The majority of these events occur in the operating room (OR) environment, contributing to patient morbidity and mortality. Although they are often determined to be highly preventable, intraoperative surgical events occur more frequently than other AEs and concerningly, their consequences tend to be more severe.

The OR is a fast-paced environment, with several factors contributing to uncertainty (e.g., unforeseen complications, abnormal anatomy). Therefore, management of uncertainty is a central component in safe surgical care; poor management of uncertainty has been linked to “near misses” and adverse events. To counter uncertainty, surgical teams must rely on high-quality communication and teamwork to ensure the coordination of information and roles to optimize outcomes. Surgical team members not only require clinical knowledge and technical skills; they also need teamwork skills to support decision-making when faced with uncertainty.

Teamwork has been recognized in the literature as an indispensable tool for building more effective and patient-centered healthcare systems. Surgical teams must be able to anticipate and modify their actions and behaviours to overcome the challenges faced in the operating room. However, team research to date has focused narrowly on studying surgical team behaviour during routine procedures, failing to give full attention to the unique demands of uncertain situations and emergencies. Characterizing teamwork competencies during times of uncertainty allows for the opportunity to formulate specific and effective interventions that will help maintain high-quality care and improve patient safety.

Frameworks to Study Teamwork

To better comprehend and evaluate teamwork, several frameworks have been developed over the years. Frameworks for assessing teamwork are essential as they provide a way to assess teams and allow organizations the ability to choose specific interventions to increase safety. These frameworks also provide a conceptual model that enables teams and organizations to track improvement and effectiveness of the interventions on specific competencies. In previous work utilizing various assessment tools and frameworks has been shown to enhance teamwork skills, as well as improve safety outcomes. Unfortunately, many of these have been adapted from outside industries to try and fit healthcare needs and there is variation between the frameworks with regards to the competencies they include in the measurement of teamwork. In addition, many are limited in their abilities to suggest specific strategies or solutions to actively resolve the issues they observe.

Recently a framework was put forward by Gregory et al., in 2019 to identify the competencies of the ideal medical team. Their evidence-based model was created by performing an extensive literature search on teamwork in healthcare and pulling the most commonly cited attitudes, behaviours and cognitions (ABCs) in previous healthcare teamwork models. They concluded there are 7 key emergent states necessary for the team to react to complex situations and promote safety. Their comprehensive review includes competencies not commonly noted in previous models such as “Psychological Safety” which the literature has shown as being critical to improving patient safety and reducing unsafe acts.

A Novel Way to Study Teamwork

Improving safety is dependent on our ability to learn from past mistakes, but the operating room is rarely monitored and evaluated, creating a limitation to enhancing quality of care. Previous studies of teamwork have relied on chart-based reviews, simulations, interviews and field observers. These retrospective methods are limited by recall bias and have also previously excluded incidents that are not charted, such as “near misses”. Real-time observers who are trained to enter the operating room, may solve for some of these issues, but there are temporal and physical limitations of trying to make observations within an OR.

This study will aim to address these limitations by using innovative technology to study teamwork in the operating room. The OR Black Box® platform (Surgical Safety Technologies Inc., Toronto, ON, Canada) is a device that allows for the capture and subsequent analysis of synchronized audio-visual recordings of the OR environment. This could allow for a more accurate assessment of teamwork without the limitations described above.


This presentation will describe an exploratory study, built on a secondary analysis of previously transcribed data and codes for uncertainty developed from prospectively collected intraoperative video recordings. The original dataset of uncertainty data was coded from observations of OR Black Box intraoperative video recordings taken during laparoscopic general surgery at a large community academic hospital. Over a 10-month period, 70 cases were coded for instances of uncertainty defined as “Any moment the OR member experienced an apparent sense of doubt, either verbally and/or physically, when making a decision for which there was no clear best answer”.

The current study builds on this previous dataset of uncertainty by coding for teamwork competencies during recorded instances of uncertainty. Teamwork codes were created using the ABC model. A deductive approach was used to code descriptions based on the team competencies outlined. These include Coordination, Conflict Management, Psychological Safety, Team Leadership, Team Decision Making and Planning, Situation Assessment and Shared Mental Models and Backup Behaviour. The team member expressing the competency was also coded to allow for a more comprehensive understanding of the team. To ensure the validity of the coding, another researcher within the human factors team was recruited and inter-rater agreement was completed before a single coder coded the remaining data.

The research questions are:
(1) What types of team competencies are present or absent during specific instances of uncertainty during surgical procedures?
(2) How does the presence and absence of these team competencies vary as a function of procedure type and team member (nurse vs. surgeons vs. fellows vs. anesthesiologists) expressing the competency?
(3) Does the number of intraoperative adverse events vary as a function of the teamwork competencies expressed?

Preliminary Findings

Seventy ORBB videos of surgical cases were coded for teamwork competencies associated with incidents of uncertainty. Coding with the teamwork competency taxonomy achieved an inter-rater kappa score of 0.73 after review of 20 cases. A total of 17 types of surgical procedures and 2,627 instances of teamwork competencies has been coded. All 7 team competencies from the original ABC model were identified in the uncertainty data across the 4 key team member roles (nurses, anesthesiologists, fellows, and surgeons).

Preliminary results show that during instances of uncertainty, Coordination is the most frequent team competency affected in moments of uncertainty (1381 instances), followed by Team Leadership (660 instances) and Psychological Safety (245 instances). Surgeons are the team member most likely to exhibit all three of these competencies. Surgeons were also most frequently seen to lack Coordination and show negative instances of Psychological Safety. These initial findings may suggest that surgeons heavily impact the team dynamic during times of uncertainty and strategies specific to their role may be beneficial.

Research Impact

Teamwork is exceptionally important during times of uncertainty in the operating room, when team competencies are required to facilitate the collaborative problem-solving needed for complex situations. Studying teamwork in the past has been mostly limited to simulation observations and retrospective reviews. The OR Black Box presents a novel opportunity to gain a thorough understanding of team competencies that are used in the OR during times of uncertainty. The results will be able to inform the OR team of specific strategies they could implement to improve their teamwork and could act as a baseline measurement tool to test future interventions. Information gathered from the results could advance our knowledge about how team members rely on teamwork (or fail to employ teamwork strategies) during times of uncertainty and how this ultimately impacts patient safety.