Introduction

The Acute Critical Events Simulation (ACES) program is a two day course intended to provide knowledge, skills and behaviours essential in acute resuscitation. The goal is to promote better care of the critically-ill and decrease the likelihood of medical errors. This manuscript discusses its design and implementation.

ACES is complementary to excellent course such as Advanced Cardiac Life Support® (ACLS®) and Advanced Trauma Life Support® (ATLS®). However, it teaches strategies applicable for any critical illness, rather than stressing algorithmic solutions applicable only to certain diagnoses. Furthermore, while many courses focus on knowledge and procedural dexterity, ACES covers not only these potential sources of adverse outcome,₁ but goes further. For example, it is well accepted that error can occur as a result of poor communication and inexperience managing the evolving medical crisis.₁,₂,₃,₄ Remarkably, this skill set, referred to as ‘crisis resource management' (CRM), is rarely taught except to anesthesiologists. ₅,₆,₇,₈ Therefore, ACES includes such strategies as: how to recognize the sick patient; mobilize assistance; work within a multidisciplinary team; and how to act preemptively while a greater chance for recovery exists. In contrast, ACLS® resuscitation typically occurs only following full cardiovascular collapse; a situation from which outcomes are often abysmal.₉

As many as 98,000 Americans₁₀ and 23,000 Canadians₁₁ may die each year from medical errors. Although the exact numbers are debated₁, ₁₂,₁₃,₁₄, few deny that errors have a major effect upon patient outcome and costs. Inexperience, human fallibility and imperfect work environments mean that errors occur in all medical settings.₁₀,₁₁,₁₂,₁₃ Unfortunately, with the critically-ill, decisions are often made quickly, under stress, and with limited information. This can compound the likelihood of error precisely where consequences can be most dire. The Canadian National Steering Committee on Patient Safety outlined that medical education initiatives are essential to tackle medical errors.₁₄ Furthermore, they recommended incorporating “simulations of high risk health-care interventions”. ACES uses Laerdal medical simulators (see below), which allow practice without patient-risk. Video playback also encourages self-awareness of strengths and weaknesses.

Physicians ought to be more than just medical experts: but also communicators; collaborators; managers; advocates; scholars and professionals. These skills encompass the CanMeds objectives.₁₅ These laudable goals can be difficult to teach using traditional methods, but nonetheless have become compulsory for Canadian trainees. ACES addresses numerous CanMeds objectives including obtaining and synthesizing relevant information; consulting effectively; managing finite resources, working within a team, and responding where advocacy is required. Our experience might also provide a template for others to design courses for their specific needs and harness their desire to decrease medical errors.

Methods

The ACES Program was initiated by the Department of Critical Care Medicine (CCM) at the University of Ottawa following the identification of recurrent deficiencies in resuscitation. The format subscribed to ideals of adult education. These include helping learners to identify their own needs and be involved in mutual planning. We also wished to provide basic knowledge but maximize guided practice, and ensure feedback.₁₆ The four original modules were 1)airway management, 2)respiratory failure and breathing management, 3)circulatory shock and 4)CRM/simulation (Figure 1).

Figure 1

Figure 1: ACES Program Modules

A pilot course occurred in December 2001. Participants included target populations such as family physicians, anesthesia residents and internal medicine residents. Feedback helped refine the pilot course, which included a greater emphasis upon case-presentations and ‘hands-on experience' with mannequins, even though this reduced the amount of factual information. The refined course received Continuing Medical Education accreditation from the Canadian College of Family Physicians and the Royal College of Physicians and Surgeons of Canada.

The principles of ACES were presented to the National Residency Program Directors for CCM. The same resuscitation deficiencies were confirmed by the majority of centres. Therefore, a proposal to offer the course to all fellows in CCM was accepted, and faculty were identified nationwide. Faculty members and a section head were chosen for each module. Each section corresponded via email and conference-call to refine ACES into a peer-reviewed program. This was presented as the First National ACES Program for CCM fellows in Toronto of 2002. The program occurred during the fellows' first month of training to ensure a modicum of early competence. Participants were beginning their fourth-to-sixth year post-graduate year. A syllabus was mailed beforehand to facilitate pre-study.

Participants were asked to complete an anonymous Previous Training Questionnaire (Appendix 1). After introductory presentations, participants were divided into groups of up to six. For the next day and a half, participants rotated through half-day sessions in the Airway, Breathing and Circulation modules, and a 6:1 resident-to-faculty ratio was maintained.

The Simulation module for the First National course was carried out in the final afternoon. Laerdal SimMan® simulators were controlled from a laptop computer by an operator in an adjacent room. They include a microphone through which voice can be delivered, a modifiable airway that can be intubated; palpable pulses, and areas to insert lines and tubes. Respiratory Therapists and Registered Nurses assist the participant and are briefed, but told to act as they would in everyday practice. They wear headsets incase directions are required. Standard medical equipment including monitors with all usual parameters increase realism, and again can be modified from the computer.

Three high-fidelity simulator scenarios were given to participants in a standardized fashion. Participants worked as a two-person team. The other four residents in the group observed from an adjacent room via a video feed. Constructive CRM feedback was provided by faculty to all group members using the videotape of each performance.

Participants filled in the Course Evaluation Questionnaire at completion (Appendix 1). As it was a measure of participant perception and satisfaction, a global rating scale, anchored with a 5-point Likert-scale, was used. Descriptive anchors were present at each point.

The Second National ACES Program was held in Ottawa in July 2003; planning began in the spring. The 2002 evaluations were instrumental in revising the format and syllabus. Changes included decreasing the time allocated to Airway, Breathing and Circulation from four to three hours. This increased each individual's simulator time from one hour to three (Figure 2). Participants now also managed scenarios by themselves. New scenarios included greater emphasis on CRM and simpler medical problems. Of note, we found that complex medical problems distracted from teaching CRM. Participants often focused on knowledge rather than how best to orchestrate care.

Figure 2

Figure 2: ACES Program Schedule

In response to feedback, additional sessions on sepsis, antibiotics, and blood transfusion were added in 2003. In addition, shorter but more frequent, sessions occurred within each module to facilitate consolidation of skills and increased breaks. Faculty arrived a day early in order for rehearsal, and for “train-the-trainer” sessions. This included peer-driven presentations on how to give feedback and how to operate the simulators.

We also formalized the administrative structure. A not-for-profit company, the Canadian Resuscitation Institute/Institut Canadien de Réanimation, was established. A web site (www.cri-icr.org) also facilitates registration and the completion of questionnaires. It also serves as a portal to access study material. A manual entitled “Organizing an ACES course” has also helped faculty plan their own courses.

Resident demographics and previous training were analyzed using descriptive statistics. We computed mean and median scores for each of the modules and their respective sessions. Using the Student's t-test, the 2002 and 2003 evaluations were compared. Given that the Airway, Breathing and Circulation modules were shortened in 2003, a comparison was performed for the question: Was there enough time allocated for the presentation?

To determine whether the residents' previous training influenced the usefulness of the simulator sessions, we identified a priori four variables: training in anesthesia residency program (yes/no); number of ventilated patient managed per year; number of times vasopressors/inotropes used per year; and past experience with simulators (yes/no). We only used the 2003 database since participant surveys were not linked to the course evaluation in 2002. These four variables served as surrogate markers of previous training or experience in airway management, respiratory failure, management of shock, and CRM, respectively. Using multiple linear regression analysis, we determined whether past training and experience influenced the perceived usefulness of the Airway, Breathing, Circulation and Simulator sessions. We also used simple correlation to determine whether having taken a previous ACLS® or ATLS® course influenced the perceived usefulness of these sessions.

Results

Forty-nine CCM fellows received the course; roughly one-half trained in Internal Medicine, one-quarter in Anesthesiology, and >80% were males. The most common means of acquiring experience is on real patients during clinical rotations. Only half had any prior simulator experience, and one-third prior CRM training (Table 1). Ninety-seven % have received ACLS® and 48 % ATLS®. In contrast, the majority of fellows had attended <15 hours of didactic teaching on various topics related to the resuscitation of critically ill patients and even less supervised instruction.

Figure 3

Table 1: Demographic Information on Participants

The overall evaluation was very favourable (Table 2). On a scale of 0 to 5 (5 representing strongly agree), the rating was 4.38 (95% CI, 4.12-4.65) in 2002 and 4.44 (95% CI, 4.3-4.59) in 2003. Participants also felt that ACES was useful, with scores of 4.33 (95% C.I 4.01-4.67) for 2002 and 4.37 (95% C.I 4.19-4.55) for 2003.

Figure 4

Table 2: Comparison between average evaluation and perceived usefulness scores for 2002 and 2003

The 2002 and 2003 simulator evaluations were compared by a t-test in order to evaluate the participant driven changes. Results are summarized in Table 3. Changes in 2003 included more time for the Simulation/CRM module, and addition of transfusion and sepsis modules. Correspondingly, less time was available for the Airway, Breathing, and Circulation modules. Of note, a statistically significant number of participants did feel that there was insufficient time for airway and breathing in 2003 (p=0.0018 and p=0.0383 respectively). There was no difference for the Circulation module (p=0.35). However, the additional time for CRM/Simulation was associated with significantly increased satisfaction (4.01 in 2002, 95% C.I 3.71-4.31, versus 4.67 in 2003, 95% C.I 4.54-4.80, p = 0.0004).

Figure 5

Table 3: Participant satisfaction comparing changes made to course in 2003, as a function of question: “was there enough time for each module?”.

We found that previous training and experience in airway management, respiratory failure, management of shock, and CRM, had no effect on the perceived usefulness of the Circulation and Simulator sessions. Completion of residency training in Anesthesia did significantly decrease the perceived usefulness of the Airway session (R² = 0.208, p = 0.0148), but had no effect on the perceived usefulness of the Breathing, Circulation, and Simulator sessions. We also found that a greater experience in the management of shock decreased the perceived usefulness of the Breathing session (R² = 0.373, p = 0.0004), yet had no influence on the perceived usefulness of the other sessions, including Circulation. In addition, there was no correlation between having taken ACLS® or ATLS® and the perceived benefit of the Airway, Breathing, Circulation, and Simulator sessions.

Conclusion

ACES is a unique peer-reviewed educational program that focuses on the knowledge, procedural dexterity and behaviours required to perform early resuscitation. It is one of the first national courses to emphasize CRM, and to incorporate medical simulators. The reviews demonstrate that the program was very well received regardless of prior specialty training, or ACLS®/ATLS® experience. In fact, many participants, including anaesthesia trainees, reported ACES to be the best educational experience they had ever attended. We also demonstrated how course feedback was used to successfully modify the ACES program. This suggests that the program has the flexibility to provide ongoing Medical Education tailored to the specific needs of participants.

The questionnaire showed a notably lack of prior training in resuscitation. When one considers that this is a selected group who have chosen a specialty for which resuscitation is central to practice, this is concerning. When anesthesia trainees are excluded, deficiencies are even more apparent. This presumably reflects that most undergraduate and post-graduate programs, do not mandate training in this area.₅,₆,₇ Equally notable is the profound absence of CRM instruction. Deficiencies in CRM have been identified as a major source of medical error, and affect practitioners at all levels of experience.₃, ₄ While CRM training is increasing,₈,₁₇,₁₈,₁₉ again, most medical schools and residency programs, with the exception of Anesthesiology, do not explicitly teach it.₈,1₉,₂₀ Presumably, educators hope that trainees will gain these skills during residency. However, our data confirms that trainees are typically unsupervised during resuscitation. Furthermore, examinations typically focus on factual knowledge and curricula rarely include training in leadership, problem-solving, situational awareness, resource utilization and communication. It is widely accepted that these are essential facets of good physicians, but they are rarely addressed. ₈, ₁₇, ₁₈ One reason may be because there are few concrete strategies. This all suggests that ACES could be an important tool.

CRM is not unique to medicine. Indeed, most CRM research is from aviation, aerospace and nuclear power.₁₇,₁₈,₁₉,₂₀,₂₁,₂₂,₂₃,₂₄ As with medicine, errors in these professions carry disastrous consequences. Unlike medicine, these professions readily adopted simulation and insist on regular practice as a function of employment.₂₂,₂₃,₂₄ Barriers to teaching CRM have included the risk of learning on patients. Fortunately, high-fidelity medical simulators now obviate this.₂₅,₂₆ Recent studies demonstrate that residents perceive simulator cases to be highly realistic,₂₇,₂₈ and that errors committed during simulations are similar to those in real practice.₂₈,₂₉,₃₀,₃₁ Simulators have the ability to provide for repeated practice and immediate feedback. Furthermore, they enable training in rare conditions that demand proficiency but do not occur frequently enough to allow regular training. Examples include the surgical airway, bioterrorism, or even SARS.₃₂

ACES focuses upon non-punitive feedback rather than ascribing a pass-fail grade. This was deliberate given that our primary goal was to facilitate learning, rather than just to determine competency. Of note, only half of our participants had prior simulator experience. Therefore, performance could be hindered by a lack of familiarity as opposed to any deficient ability or skill set. Causing anxiety because participants are also being scored might further decrease the simulator's educational value. It remains to be seen if continued familiarity with simulators will allow them to be used as formal evaluative tools. Initial studies comparing simulation to standardized examinations suggest that simulator performance does not always correlate with written grades.₃₃ However, rather than dismissing the utility of simulators, this raises the question of which format better assesses competent delivery of care; or in fact whether they are testing different skills. Regardless, it presents a fertile area for research. Unfortunately, it has currently meant a reluctance to incorporate simulators into formal examinations. Of note, however, the airline and nuclear industries have widely incorporated high-fidelity simulators without the perceived need for validity studies.₂₂,₂₃,₂₄

Clearly a more comprehensive pre- and post-test could have been conducted. For example, the pre-course questionnaire did not formally assess pre-existing knowledge or procedural dexterity. Surveys do not assess competency, they simply document prior training. Although more testing and larger participant numbers would improve the quantitative assessment of the course, this would not asses the behavioral skills that are, after all, the focus of ACES. Thus, it may be difficult to more precisely quantify the benefit of ACES given the absence of a gold standard test for CRM. Complex prospective outcome studies comparing participants and non-participants are an option. Of note, however, ACLS® and ATLS® were never prospectively shown to improve outcome, but have been widely accepted nonetheless. For ACES, ongoing improvements are expected to continue from a combination of pre and post-testing, polling potential participants (i.e. needs assessment) and analyses of critical events and medical errors.

ACES is an extremely valuable experience for faculty. It provides an opportunity to develop and share educational material. It also fosters ties between institutions and has germinated other initiatives. These include the coordination of multi-centre educational research, and development of pediatric ACES. Given the importance of language to CRM, we are also finalizing the full ACES Program in French.

The ACES Program is obviously no panacea. However, it can help clinicians wishing to ensure exemplary acute care; administrators eager to decrease adverse outcomes; and educators wishing to tackle the CanMeds objectives. We hope our experience will inspire others to harness the breadth of medical educational expertise and enthusiasm.

Appendix 1

QUESTIONNAIRE DEVELOPMENT AND ADMINISTRATION – ACES NATIONAL COURSE

A peer-reviewed pre and post survey was developed by faculty.

The Previous Training Questionnaire was administered at the start of the program, on day one. It had the following objectives:

• Demographics of the participants (background specialty, seniority, gender)

• Experience in “ABC” skills training

• Experience in crisis management

• Instruction received in ABC skills implementation

• Instruction received in crisis management

• Experience with high-fidelity medical simulators

The Course Evaluation Questionnaire was administered upon completion. It had the following objectives:

• Overall evaluation/perceived usefulness

• Evaluation and perceived usefulness of airway module

• Evaluation and perceived usefulness of breathing module

• Evaluation and perceived usefulness of circulation module

• Evaluation and perceived simulation and crisis resource management module

• Satisfaction with time allotted to each section

• General comments and suggestions for course revision

The full questionnaires are available at http://www.cri-icr.org.

Acknowledgements

Role of Authors

Correspondence to

Dr P Brindley Capital Health Medical Lead for Patient Simulation Assistant Professor and Program Director, Division of Critical Care Medicine 4H1.22 University of Alberta Hospital 8440-112^th St, Edmonton, Alberta, T6B 2R7 Tel: 780-407-8822 Fax: 780-407-3340 Email: peterbrindley@cha.ab.ca