Introduction

Current literature shows that historically it has been challenging to integrate lasting behavioral change in healthcare, even when evidence suggests benefit and need for change1. In recent years, analysis of reported medical errors shows that most errors can at least in part be attributed to deficient nontechnical skills such as leadership, effective communication and situational awareness2. In complex organizational settings, interventions that focus on collective versus individual behavioral change appear to be more successful3. Using crisis resource management (CRM) simulation training to address non-technical errors has consistently demonstrated at least short-term benefit4-6.

However, traditional CRM programs focused on promoting skills within only select members of the healthcare team or within trainees of individual professions, and have also been limited to simulation center contexts7. Furthermore, there remains no consensus regarding a universal methodology by which to train professionals8. While there has recently been increased focus on implementing team-based CRM simulation training, the extent of involvement from different disciplines in these initiatives has been limited in the existing literature. It remains to be elucidated how an interprofessional simulation training program, with involvement from all professions from conception to evaluation, would be received.

To best support the potential educational value of an interprofessional CRM-based training program and to promote long-lasting institutional change, we took a team-based approach throughout the entire process of the design, implementation, and evaluation of our program. To align most closely to ‘live’ clinical practice, our program was designed to be interprofessional (between professions), interdisciplinary (between disciplines of the same profession), in situ and for practicing health care professionals. Feedback from participants was integral to all stages of the program, and allowed the unique needs of each discipline to be identified and attended to.

Methods

Program Overview

Our simulation-based CRM team training program was intended for teams of health professionals who typically work in the operating room (OR) and post-anesthesia care unit (PACU). Potential participants included surgeons, anesthesiologists, respiratory therapists, Licensed Practical Nurses (LPNs) and operating room attendants. Figure 1 graphically depicts our methodology.

Figure 1

Pre-Intervention

Program Development

Senior-level staff from each of the professions intended to be involved in the program were contacted and requested to identify “champions” from within their department/profession. A one-day faculty development course on principles of CRM simulation and feedback was given by staff members from the Centre of Medical Education at McGill University, who had significant experience in these areas. Candidates who successfully completed the course were selected to be involved at all levels of the program recruitment, design, implementation and evaluation. In total, 2 surgeons, 3 anesthesiologists, 3 nurses, 1 respiratory therapist and 1 operating room aid successfully completed the full course of training. Following completion of the training course, leaders from the various disciplines contacted their own departments and solicited topics relevant to their profession that they wanted emphasized in the course. 

Curriculum development was a transparent process, with face-to-face discussions between champions from all professions to establish a shared mental model concerning aspects such as role definition and debriefing points. To successfully implement the selected educational endeavor, Kotter’s steps to transforming organizations was used as a conceptual framework. This framework outlines and incorporates steps such as establishing a sense of urgency, forming a guiding coalition, creating, communicating and consolidating vision, empowering others to act, and institutionalizing new approaches.

Interdisciplinary Scenario Design

Suggestions elicited from each of the participating champions including discipline-specific priority goals, objectives and sample scenarios were submitted to scenario writers. Additionally, an electronic needs assessment was sent to all members of the professions working in the operating room (OR) and post-anesthesia care unit (PACU) for further input on learning objectives to incorporate for each profession. Using this information combined with actual morbidity and mortality (M&M) cases from the workplace, simulation scenarios were generated by an interprofessional panel of content experts, medical educators and simulation experts. Drafted scenarios were sent back to the champions of each profession to review before finalizing. This was to ensure that the needs of all participating professions involved were met. Scenarios underwent rigorous simulator programming, piloting, revision and modification to ensure the simulations were high-quality and realistic. High-fidelity manikins were used as patients. A list of scenarios used in the educational program and their accompanying descriptions are included in Table 1.

Table 1

Ethics

The Research Ethics Board of the McGill University Health Centre (MUHC) granted ethics approval for the research study component of the program. Prior to completing the course, participants who wished to participate in the research component completed written consent forms for their participation in the study. After consent was provided, the participants completed pre-course questionnaires, participated in the educational program and debriefing and then completed post-course questionnaires. All data was kept confidential and no identifying information was kept. Only members of the research team had access to the results from the questionnaires.

Intervention

Study Setting and Population

Staff members from a tertiary care pediatrics center, the Montreal Children’s Hospital in Montreal, QC, Canada were recruited to participate in a variety of simulated scenarios over a two-year period. As part of the recruitment process, two joint surgical grand rounds were held. During these rounds, an explanation of the program was provided with an emphasis on the importance of CRM, interprofessional teamwork and the use of simulation for acquisition of these skills. The chairs of the Departments of Surgery and Anesthesia, in addition to the directors of the Department of Professional Services and Risk Management Committee hosted the rounds. All implicated professions were invited.

Staff members who were invited to participate came from a variety of professional backgrounds, fields of expertise and levels of experience. All simulations were completed in situ at the Montreal Children’s Hospital in the typical OR and PACU settings where these staff would perform their usual duties.

Simulation Sessions

All participants took part in 1 of 17 1-hour inter-professional and interdisciplinary simulation sessions. Each 1-hour session entailed a 15-20 minute high-fidelity in situ simulation, followed by a 40 to 45-minute debriefing facilitated by a team of trained debriefers, corresponding to the different professions represented in the program. Simulation required participation of an entire team, hence staff were paired with their usual colleagues. Teams were composed of 1 surgeon, 1 anesthesiologist, 2 OR or PACU LPNs, 1 respiratory therapist and 1 operating room attendant.

Debriefing Sessions

A structured debriefing segment followed each simulation. Discussions focused on identifying and exploring interdisciplinary-specific issues in CRM and systems-based issues specific to the site. These sessions were facilitated by a group of pre-selected staff experienced in interdisciplinary CRM training, representing the various disciplines involved in the scenarios. Potential solutions to problems identified during these sessions were recorded by the debrief moderators and submitted to the perioperative committee.

Post-Intervention

Data collection

Participation in the research component of this project consisted of participating in the education program in addition to submitting pre- and post- course questionnaires. However, deciding not to participate in the research component of this project did not preclude an individual from participating in the educational activity.

Each research participant was asked to complete a 4-question pre-course survey prior to taking part in the simulated scenarios. This survey consisted of questions assessing the participant’s initial thoughts towards the simulation (e.g. how much they were looking forward to the course and associated apprehension levels towards both the simulation, the debriefing, as well as the interprofessional and interdisciplinary nature of the course). All pre-course questions were answered on a 5-point Likert scale (responses ranging from 1 = Strongly Disagree to 5 = Strongly Agree).  A 28-question post-course survey was completed following the simulation debriefing. The post-simulation survey included items targeting: perceived course quality, the participant’s interest and overall experience, comfort level, feedback received and perception of their role throughout the session. In addition to 5-point Likert scale questions, the pre- and post-course surveys also included 6 items collecting demographic data such as profession, work experience and previous experience in simulation.

Data Analysis

Descriptive statistics were performed for the analysis of overall quantitative data. Quantitative data was then analyzed specifically looking at discrepancies between pre- and post-course questions addressing course quality and apprehension. Post-course questions assessed knowledge of CRM, communication skills and teamwork. To stratify results, Welch’s t-tests were performed to compare mean response scores between linked pre-and post-course questions, between physicians and allied health professionals, between individuals with prior simulation experience and those without, individuals with prior CRM experience and those without and prior interprofessional-CRM experience and those without. Regression models were created to compare results for non-dichotomous variables. Only differences in means of at least 0.5 points on the Likert scale were considered significant for analysis and 95% confidence intervals were created. Those with both upper and lower limits either completely residing to the right (positive) or left (negative) of zero among these were deemed to be significant.

For illustrative purposes, the 5-point Likert scale was converted to a 3-point scale, with “Strongly Disagree/Disagree”, “Neutral” and “Agree/Strongly Agree” composing the 3 points.

Results

Pre-Intervention

Program Development

During the development and implementation of this interprofessional training program, we identified challenges that prevented individuals from becoming involved in the program. Originally, the educational program was given during Grand Rounds or after normal working hours, which limited participation and required staff to be paid overtime. After extensive troubleshooting, it became clear that the success of this type of program depends on support from the entire institution. Hence, institutional approval was obtained for each surgeon to give up one hour of operating time every three years to participate in this type of program. This approval needed to come from the perioperative committee as it concerned patient safety. Furthermore, to ensure lasting cultural change, the program needed to commit to being truly interprofessional from beginning to end. 

Intervention

Study Setting and Population

127 health professionals participated in a total of 19 simulation sessions between May 2014 and January 2017 (distribution of health professionals provided in Table 2). Complete data was available for 9 of the 19 sessions, but program evaluation responses regarding course quality, feedback from debriefers, the course’s relevance to work, as well as qualitative responses were available from all 19 sessions.

Table 2

Participants from different disciplines and specialties reported varied levels of past simulation training experience. In general, fewer staff possessed CRM training compared to general simulation training and even fewer had experience in interprofessional CRM training (Table 3). Staff self-reported work experience ranged from 31% with less than 5 years of experience, 22% with 5-10 years of experience, 19% with 11-20 years of experience and 28% with 20 years or more of experience. However, distributions between professions varied, with nurses having the largest proportion of staff with less than 5 years of work experience (51%) and anesthesiologists having the largest proportion of staff with greater than 20 years of experience (55%) (Table 4).

Table 3

Table 4

Post-Intervention

Program Evaluation

Nearly all research participants (99%) found the session to be relevant to their work and greater than 95% found the simulation scenario to be interesting, of good quality and were likely to recommend the course to a colleague. Greater than 95% of the staff involved found that after the session they were more confident in their knowledge of CRM and of teamwork (Figure 2). No differences were found when comparing responses from individuals from different professions, with different work experience or with different levels of previous simulation training.

Figure 2

System-based Issues and Solutions

Examples of system-based issues and solutions that were identified during the debriefing session are given in Table 5. These ideas were brought back as feedback to the perioperative committee. Specific points of improvement were identified in addition to which groups they were most relevant to. Subsequently, lists of occupation-specific priorities were compiled. Champions of each given profession were responsible for working with their respective staff to implement changes based on these priorities. After these changes were supposed to have been implemented, the perioperative committee followed up to ensure that this was the case. 

Table 5

Discussion

The use of normalization process theory to examine successful behavioral change has shown that in order to successfully perpetuate long-term change in complex healthcare settings, interventions need to emphasize collective action and need to legitimize new practice norms through experience3. The importance of collective action in healthcare is well-known; the Institute of Medicine recognizes the importance of teamwork in providing patient-centered care and improving patient safety9. Further, the WHO encourages interprofessional education and collaborative practice to optimize health services10. The benefits of experience-based learning through simulation are also well-studied in medicine11. We strived to meet these characteristics in the design of our educational intervention by actively involving all stakeholders, including hospital administrative staff and frontline healthcare workers representing all professions involved in the care of patients in the OR and PACU. These individuals were instrumental in the design and subsequent execution of a novel, in situ simulation-based program, focused on interdisciplinarity and relevance to real clinical practice.

To our knowledge, this is the first study to detail this extent of interprofessional involvement in educational program development and provides insight into the experiential value of this type of training. In our program, each profession and discipline set objectives for the simulation, was involved in the scenario design, provided a champion, contributed to the debrief and evaluated the simulation. Needs of each profession and discipline were accounted for across the entirety of the project.

Previous studies have assessed the benefits and utility of interprofessional CRM simulations by measuring improvement in clinical performance12. A systematic review done in 2015 comparing simulation-based CRM team training to didactic case-based CRM training and simulation without CRM training demonstrated that simulation-based CRM team training resulted in significant improvement in CRM skill acquisition in all but two studies13. One study included in the review showed that workplace team behavior significantly improved after simulation-based CRM team training. Two other included studies showed that adverse patient outcomes were reduced after a single simulation-based CRM team intervention. While these studies focused on the outcomes of interprofessional simulation, our study focuses additionally on the methodology of the simulation as being entirely interprofessional from conception to evaluation and the reception of this methodology by participants.  

In one study, a physician and registered nurse worked together to develop a team training exercise with the similar goal of interprofessional collaboration from study design to completion14. Dr. Linda Spillane, a professor of Emergency Medicine and assistant dean for Medical Simulation at the University of Rochester, created simulation goals in partnership with nurse educators. However, the cases and the CRM framework used to assess team performance and for debriefing were selected by Dr. Spillane alone. In our study, sample scenarios were accepted from all professions and disciplines and accounted for their specific priority goals. In addition, debriefing sessions were led by a variety of staff from various disciplines with expertise in interdisciplinary CRM training, as opposed to being led by an individual physician. Like us, Norsen & Spillane acknowledged the benefits of collaboration and shared expertise in all stages of an interprofessional activity, from planning to execution to evaluation. In our study, each profession and discipline made valuable contributions during the entirety of the course, which allowed the course to be truly interprofessional.

Another unique aspect of our study was the implementation of changes post-simulation. The Kirkpatrick model, one of the best-known models for evaluation of educational and training programs, was recently adapted for use in health education15. According to this model, part of the adequate evaluation of an educational program is to act on recognized deficiencies by implementing changes which result in improvements in practice and which inform curriculum development16. Our study not only identified specific issues and points of improvement but fed them back to the correct individuals and ensured that pertinent changes were implemented. As per the Kirkpatrick model, this course fulfills all aspects of training evaluation and paves the way for a continuous cycle of learning and improvement.

Participating staff reviewed the course favorably. Almost all participants rated it enjoyable, relevant to practice and were willing to participate in future courses. The debriefing was found to be effective and professional, and the simulation was commended for being realistic. These findings, coupled with reporting of self-perceived improvements in CRM and teamwork skills, support the use of CRM training programs in empowering staff, as characterized in past CRM studies4. Furthermore, participants consistently expressed the need for more regular, longer sessions with more varied scenarios, suggesting that they recognized the value of this type of course.

Due to the nature of our study and the context in which the data collection took place, our study has several limitations. Currently, the sample size is too small for further analysis by individual profession. The creation of a culture of change on a systematic level could have influenced the results to favor an improvement in skills. Furthermore, because the study took place at the Montreal Children’s Hospital, the systematic issues identified may not be generalizable to all healthcare settings. For future studies, a larger sample size will be needed to see if trends continue or if other trends emerge.

Studying barriers to staff participation and conducting long-term studies on continued staff involvement in these types of educational interventions will aid in tailoring the ideal CRM course. Knowledge-translation studies will be required to determine whether skills gained from these types of training sessions indeed result in improved patient outcomes and whether they show a clinically-relevant improvement over existing types of team training. Furthermore, to realize the full potential of this type of study, the scope should be increased to include other areas of the hospital (i.e. emergency room, Intensive Care Unit and wards), multiple interacting teams and staff-to-staff handover to best simulate crises and enhance the preparedness of the individual, team and institution, while continuing to identify further areas of weakness.

Acknowledgements

We would like to acknowledge the contributions of the McGill University Simulation Centre and Institute of Health Science Education (IHSE) Centre to this work. We would also like to thank all participants for offering their time and energy to contribute to our project.

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.