Introduction

Surgical education has long relied on the mentorship model as the chief architecture for knowledge acquisition and skills training. In the late 1990’s, the estimated cost of educating surgical residents in the operating room was at or above $50,000¹. At the same time, operative times are under increased scrutiny, clinical activities generate less per visit compensation, and federal funding has decreased to offset direct and indirect graduate medical education (GME) costs². Additionally, resident hours limitations and the ACGME mandate that training programs certify surgical competency, increased pressure is being applied to maximize the educational benefit of each experience throughout surgical education.

Concurrently, many surgical techniques have moved from traditional open techniques to minimally-invasive techniques. This has been driven both by the advance of various technologies and by the promise of decreased overall cost. Often, minimally-invasive techniques require additional training to achieve facility with equipment and technical modifications. Endoscopic Sinus Surgery (ESS) is one such area of advancement. One proposed method of achieving gains on these competing goals is skill development through surgical simulation.

Several high-fidelity systems exist, employing variations on virtual reality (VR) environments, haptic feedback system, and user interfaces^3-7. To date, limited availability and high front-end cost have kept high-fidelity simulators from becoming widespread throughout Otolaryngology training programs. This pilot study aimed at developing facility with bimanual endoscope use in medical student and resident populations.

Methods

Institutional Review Board approval was gained through the University of Texas Health Science Center at San Antonio prior to instituting this study. One human cadaveric specimen was used in the context of a “sinus lab” set up. Video monitor, 0 degree endoscope (Karl Storz), a freer elevator, camera head and video processor system. The faculty proctor provided verbal instructions on the sequence of maneuvers and demonstrated the technique, structures, and sequence to all participants. Verbal understanding of the task was verified. Under endoscopic visualization, each participant sequentially touched the septum, inferior turbinate, middle turbinate, uncinate process, bulla ethmoidalis, and vertical basal lamella. Each participant performed these tasks in the same order at 3 separate times on the same day of training. The primary outcome measure was time measured in seconds.

Data was collected by stopwatch in seconds while watching the surgeon perform basic maneuvers with a preserved cadaveric specimen. Standard endoscopic operating equipment was available for surgeons during this study.

Statistical analysis was performed with SPSS 11.0. Parametric data was analyzed with two-way T-test and one way ANOVA. Curve fitting was performed with Microsoft Excel Chart application.

Results

There were 4 mid-level otolaryngology residents (PGY3 and PGY4) studied along side 5 medical students. All residents had thorough training in endoscopic sinus surgery. At all three surgical attempts, residents were significantly faster to accomplish the surgical goal correctly (Attempt 1: 20.25 vs. 80.20 sec (p=0.002), Attempt 2: 9.00 vs. 26.40 sec (p=0.003), Attempt 3: 7.00 vs. 21.80 sec (p=0.008)).

Paired analysis of data showed that between the 1st/2nd and 1st/3rd attempts, surgeons had a significant improvement in overall surgical speed to complete the task (p=0.009 and p=0.006). The trend for improvement between the 2nd and 3rd attempt was not statistically significant. These same findings remained significant for subgroup analysis when residents and students were studied independently (FIGURE). Best curve fitting of the data showed a geometric rate of improvement between attempts (R^2 = 0.9516).

Figure 1

FIGURE: Learning curve evolution over 3 attempts for a basic endoscopic sinus surgery skill. The medical students (dotted line) were significantly slower than residents (dashed line) at all attempts. Typical asymptotic behavior of the learning curve is evident in both study groups.

Discussion

Since the first dissection of anatomic specimens, medical education has relied on low-fidelity surgical simulation as a component of training. When evaluating systems of surgical simulation, considerations must include fidelity of the system to the actual surgical experience, attainment of desired skills, transferability to operative practice, and overall cost of training. The use of a human cadaveric model attempted to maximize face and content validity. This low-fidelity system effectively trained several action measurements defined by the Metrics for Technical Skills Conference⁷. Psychomotor haptic and depth perception abilities, the task of camera navigation, the skills of instrument handling and bimanual dexterity, and the procedure of sinusoscopy were all effectively trained in this exercise. In both the medical student and the resident cohort, time to task completion significantly improved.

Although accuracy parameters were not evaluated, the primary outcome measure of time serves as a surrogate for overall ability to synthesize and perform the related tasks which can be termed “scope handling”. Skills developed fall into Pre-training, Basic, and Intermediate levels. In simulation work done in other surgical specialties, training of “part tasks”, or component parts of more complicated tasks, in low-fidelity systems are as good or better than richer environments for training novice learners^{8, 9}. Including various types of surgical simulation in a graduated curriculum has been shown more effective than standalone modules¹⁰. This supports a curriculum of ESS training that includes low-fidelity part skills attainment modules for novice learner who would later progress to higher fidelity, higher complexity models as their skills develop.

Although literature evaluating the direct effect of ESS simulation on operative performance is thin, improved operative performance has been demonstrated with training on laparoscopic surgery simulation platforms^11-13. Additionally, mastery of basic skills acquisition correlated with acquisition of more complex skills¹⁴. Caversaccio, et al failed to demonstrate positive effect of pre-surgical simulation planning in ESS, though significant study design issues may have influenced this result⁶. ^{Much work remains to be accomplished to establish improved surgical outcomes with ESS simulation.}

The ANOVA testing suggests that mid-level residents not only are capable of accomplishing surgical goals faster, but after three attempts, the medical students were still significantly slower surgeons than moderately-practiced residents. There are important considerations to this finding. Unlike other more simple procedures, endoscopic sinus surgery skills, even very basic ones as performed in this study, cannot be expected to be learned in a few attempts. These skill sets require complex hand-eye coordination that requires significant levels of practice to master.

The results of the paired analysis testing showing significant improvement between the 1st surgical attempt and subsequent ones suggests that there is a steeper learning curve for speed that shows immediate benefits at the second attempt.

Curve fitting of the data suggests a rapid rate of learning that is evidenced by a best-fit that is geometric in nature. Asymptotic decay of this form of model suggests that between the 6^th and 7^th surgical attempt, variations in surgical speed would be around 1 second and diminish from there. This is consistent with the findings of Uribe, et al. who demonstrated a step learning curve for the first 3-5 trials, with a plateau of achievement being reached within 4-5 additional trials¹⁵. The overall implication is that a geometric rate of decay means rapid learning, with minimal additional improvement in the primary outcome measure, speed, after a handful of attempts.

Overall evaluation of cost must include the fiscal cost of assembling a useable model as well as per use costs of the model, and the “time cost” of training on the model. The equipment used in this training can be purchased and assembled for $6,050. Many programs will have some or all of this equipment already available, which ameliorates this direct cost. Cadaveric specimens can be appropriately preserved in for limited number of uses. Unlike other training systems that rely heavily on faculty proctoring, this low-fidelity model does not require similar oversight, thus reducing the overall time cost of training.

Although this pilot study demonstrated significant improvement in task completion time, suggesting training effect, several limitations exist. First, the small number of participants leaves open the possibility of a Type II error. Future studies are planned to expand the test population and decrease the probability of this type error. Second, cost, limitation of the supply of fresh cadaveric specimens, and degradation of cadaveric tissue limit the access to and reuse of this model. Development of a static, non-degradable model would represent an improvement in subsequent modeling. Third, testing established significant improvement in immediate skill development and performance; however, intermediate-term and long-term follow up was not performed to verify skill retention. Future studies are planned to establish skill retention. Fourth, this model currently exists as a “stand alone”, rather than as an integrated part of a continuous curriculum. Extrapolating from simulation data in other surgical specialties, effect would be improved by creating a graded curriculum of simulated task that increase in complexity and fidelity as the learner’s skill develops. Fifth, participants in this study performed the same sequence of maneuvers. It is possible that improvement in times reflected familiarity with the task rather than improved endoscopic skills. Dubrowski, et al demonstrated that randomly sequenced task performance result in greater retention of skills than simple repetition¹⁶. Random sequencing will be integrated into future iterations of training. Finally, this data demonstrates only improved “endoscope skills”, but does not establish relationship to performance in the operating theater.

Conclusion

In recent years, concern has been raised about ethics and cost of resident surgical education being conducted completely within the operating room. These concerns, combined with increased pressure to maximize efficacy of resident education brought on by resident duty hour limitations, has motivated surgical simulation in resident education. In this pilot study using a low-fidelity endoscopic training system, senior medical students and mid-level residents improved their performance of programmed endoscopic tasks with repeated practice in a sinus lab setting. Mid-level residents are capable of accomplishing endoscopic maneuvers faster than medical students. Practicing endoscopic skills in a sinus lab setting can develop basic skills necessary to perform more complicated endoscopic surgical procedures in the sinuses. Low-fidelity simulation is an effective, low-cost modality that can be integrated into a graduated curriculum of endoscopic surgical skills training that may result in improved operative performance among trainees.