H Denz-Penhey, J Murdoch
H Denz-Penhey, J Murdoch. Organising a clinical curriculum in rural settings: implementing quality control. The Internet Journal of Medical Education. 2009 Volume 1 Number 1.
Evaluation has been defined as “a process of decision making about the object being evaluated and how it compares to some standard of acceptability.1 “ All innovative medical programs require evaluation to ensure that medical training standards are maintained and any mistakes rectified.2 Much evaluation is done of individual classes and/or teachers and/or programs3 with an emphasis on the quality of teaching.4 One of the aspects of evaluation is the importance of undergraduates’ perceptions of their teaching and training environments5 with the majority of studies depending on item based questionnaires or measures4 such as the DREEM questionnaire developed in Dundee and apparently useable across many cultures.6
It is recognised that students find new curricula stressful1 and that they also find new ways of delivering curriculum stressful.7 Student satisfaction with teaching and learning opportunities is one important aspect but satisfaction at the structure, organisation and supervision aspects must also be considered.4 These aspects of clerkship structure and systems which support it are much less frequently investigated.4 There is little in the literature about the establishment of systems to define and maintain the quality of education8 and also little that identifies the systems of curriculum delivery in settings that are both greatly different from the usual metropolitan-based hospital and which also vary considerably amongst the rural and remote locations in which teaching must be undertaken.
The Australian Commonwealth Department of Health and Ageing has provided funds for the Australian medical schools to establish Rural Clinical Schools.9 This workforce initiative has enabled medical students to learn in a diverse range of rural and remote healthcare settings.
The initial task, in 2003, was to deliver the 5th Year curriculum to 25% of the medical students in a number of rural and remote settings. The pre-existing University of Western Australia 5th year curriculum was structured into obstetrics and gynaecology, paediatrics, internal medicine, general practice and medical specialties of cancer and musculoskeletal in 8 week blocks with problem-based learning (PBL) sessions, tutorials, lectures and a certain expectation that common presentations will be seen within that 8 week period and assessed with tightly framed assessments.
Some of the difficulties inherent in organising such curricula have been emphasised by colleagues from Queensland10 particularly as they relate to distance from the base university and the isolation from the metropolitan centre. As the rural clinical schools were set up to encourage later practice in rural and remote areas, the clinical experiences students have must be of sufficient quality that they will perceive rural medicine to be an attractive career.11 It is a real challenge, however to keep students grounded academically, clinically and emotionally over vast geographic distances.12
How do we identify quality curriculum delivery processes in an environment of maximal diversity? The issues that the Rural Clinical School of Western Australia (RCS WA) faced were:
Setting up multiple sites in rural and remote locations that were 200 to 2400 km from the base university medical faculty.
Insufficient patients to undertake the conventional eight week rotational teaching and learning framework of the city.
Communities as small as 1500 people and as large as 35,000 people.
Sites of varying clinical educational capacity from 3 students to 10 students.
Sites with no specialist teachers at all, although no site had less than 60% generalist teaching.
Some sites were based in Aboriginal communities, some were pastoral, some were mining, others were tourist centres or some combination of these.
Some sites had only one medical coordinator, some had up to 6 part time coordinators and all had additional preceptors supporting the clinical teaching.
Some sites had hospitals staffed solely by private practice family physicians; some were staffed by salaried proceduralists, and one by the full range of specialists and residents.
In the first year, there were only two staff members experienced in teaching undergraduates and so there was a need for a fast professional development program for already busy rural clinicians who had taken on the teaching responsibilities in sometimes very remote sites.
The rural clinicians who took on the teaching role were clinically experienced and knew their sites strengths and limitations. Hence they had strong preconceptions of how the program needed to be introduced in their site.
In 2007 The RCS WA had students from a second university join the program. These were graduate students in a 4 year PBL- type program being trained alongside students from a disciplinary based 6 year program.
The challenge for the RCS WA was to structure its teaching and learning in a way that it met the Australian Medical Council learning objectives for the undergraduate, provided excellent teaching and learning, was manageable for medical coordinators given time constraints in areas of workforce shortage, and gave students an experience that would not only encourage them to come back to the country after graduation but would also transmit that excitement to more junior students so that following cohorts would want to come to the rural program.
In order to deal with these difficulties, we set some clear principles in place. We decided there was a need
for each site to be staffed by one or more medical practitioners who were also a member of the medical faculty at the base university;
for students to have end of year examination results equivalent to their city trained counterparts;
for a generalist focus to the year, rather than a specialist focus – we were training for their first postgraduate generalist years (compulsory in Australia) not residency training for their future specialty;
to respect the site specific knowledge of what was possible and appropriate given the local conditions;
to respect students’ reasons for coming to the rural placement and where possible to meet reasonable expectations;
to operate using adult learning and self -directed learning models, and
to offer additional support where necessary to address possible site specific deficiencies.
There were seven Emergent Themes or systems components that were consistently exhibited in those sites which were considered outstanding in any particular year.
The 7+3+1 Model
Students were more satisfied when they spent seven sessions in their week in clinical settings, three in academic sessions and one in self-care. (A session was a half day of three to five hours.)
The Western Australian medical student learning process in the tertiary hospital is self directed, and, while a certain level of attendance is expected, it is rarely compulsory on any particular day. In the rural setting, the students observed other clinicians, practised taking histories, performed examinations of patients, came to diagnoses and worked out management plans. In practice, this was usually six half-day clinical sessions during the normal workday week with an additional session (usually in an Emergency Department) on one evening or during the weekend. Each year there has been a site (different each year) which did not follow this structure, each year there were a few students who chose not to arrive at the clinic as expected, and each year those students who dropped down to 5 clinical sessions per week were in the two lowest quartiles in the mid-year formative clinical examinations. In addition, those students reported both to their coordinators and the evaluator being more stressed than we usually expected, often using their stress as justification for not seeing patients. When these students began to attend patients over more sessions per week, their stress levels subsided with their increasing clinical competence. It should be noted that they were not stressed by attending fewer sessions, but by gaining competency at a slower rate than they wanted or in comparison to the increasing competency of their colleagues.
Three sessions per week were sufficient to cover the academic requirements for the year in the form of tutorials, PBL sessions, Case Based Learning tutorials(CBLs), and lectures. Although many students said that they liked getting more teaching, we had no evidence across the five years that extending teaching to twelve or more hours per week improved marks or student attitudes.
One half day a week was required for students to undertake personal self care. Small rural communities often have limited shopping hours and students needed some time each week during shopping hours to do such things as their shopping, personal banking, getting a hair cut or seeing their own doctor. They could also use this time for writing up their work.
With the focus of 70% of time as clinical experience, it was important to ensure that the clinical time was at peak admission and surgical times when patients were available to be seen. In most hospitals this was Monday to Thursday in the mornings. Teaching then needed to be rostered later in the day when the throughput of patients was less. Peak times were sometimes different in differing environments. One remote Aboriginal Medical Service saw few patients after 2.30 pm, regardless of the day of the week. Another remote tourist town had hospital peak times at weekends and evenings after the day was over and so flexibility was essential for both students and staff to maximise learning opportunities. Timing of tutorials had to work around such learning peaks and troughs.
Structured academic learning
While students could cope with seeing whatever “came through the door” at the hospital or the local clinic, they needed some sort of structure around which their academic theoretical learning could be organised. With the huge number of learning objectives in their first full clinical year, students needed a way of organising this knowledge simply. The most effective structure in which to do this was to have a learning focus for a week or two weeks at a time for their tutorials and PBL sessions using one discipline at a time, such as obstetrics, with a logical learning sequence of tutorials over one or two weeks before moving on to a second learning focus, e.g. paediatrics. The learning would then cycle through all of the year’s disciplines, before coming back for a focus on obstetrics again.
What did not work from the students’ learning perspective was when each discipline was covered every week. Students became anxious and upset if they had a tutorial or PBL on several disciplines each week, no matter how well it was organised. This was identified to be the case in each site where it was undertaken across the five year period and in the face of coordinator satisfaction at their excellent organisation. Where coordinators organised their program well, student dissatisfaction would often erupt in non-related areas (for example, student-student, student-coordinator, student-hospital relationships) along with coordinator bewilderment as to why they should have such “difficult students this year”.
Students always wanted as much learning as possible from the most qualified person possible and took every opportunity to learn from specialists. However we found that over about 10 hours of tutorials in 3 sessions, the extra hours of tutorials got in the way of clinical experience and self-directed learning.
Modified Problem Based Learning (PBL)
Rural staffing issues and groups of students as small as three, made undertaking the formal PBL structure very difficult. Various sites trialled a wide range of adaptations over five years. In the first year the RCS dealt with the variations by improving the staff training in the conduct of PBL procedures. In the second year it was accepted that the problems and poor ratings of the teaching method were not related to the facilitators’ lack of knowledge of the process of PBL. By year five, all of the sites had modified the delivery of PBL sessions so that they became more like a progressive Case Based Learning approach. The same content was used, students read up about the condition prior to their tutorial but the tutorial was lead by someone skilled in the care of such patients as an additional resource person and teacher. As such, the new approach kept the integrated learning of patient presentation, history taking, pathophysiology, pharmacology and management. In addition, it improved both self-directed learning and clinical reasoning as the students were able to focus their learning of relevant matter and discuss the real clinical issues rather than “being side-tracked by their own ignorance and tendency to hare off down an irrelevant track,” as one student put it. Learning became more clinically focussed on the immediately relevant, and time was saved by students’ not reporting at length on their segment of knowledge to the acute boredom of their colleagues.
Students who had at least one other person with whom to share clinical learning reported greater satisfaction with their learning at their site than those who learned in isolation. Sometimes this was done by design with a buddy system set up at the beginning of the year; sometimes it was a by-product of sharing living accommodation in remote areas when students discussed their learning over preparing and eating meals. Having a congenial learning partner was particularly important where there were student-student personality clashes, as their “natural” tendency to share was compromised in those situations.
Generalist rather than Specialist focus
Learning from generalist practitioners maximised exposure to rural cases, but, in most sites, staff preferred students to learn in discipline-specific rotations. As a result, students, even in general practice settings, would considered themselves “doing a discipline”. Students in rural settings get to see a very large range of patients but they never know when they might present. Students who have a specialist focus very often do not “see” patients out of their rotations. For example, a student on an internal medicine rotation would not pick up on a paediatric case that they needed, because they were “doing internal med.” With smaller numbers of patients presenting in rural and remote areas, it is important that students learn from every case and not just those cases on their rotation. This silo thinking was sharply reduced when the clinical rotation length was reduced to one to two weeks at a time. Students preferred four-week rotations, but the Head of School believed the unintended consequences of silo thinking reduced the generalist focus of the year and were too negative to let student preference be the deciding factor on what was most useful as a teaching and learning process. The one exception to this was when students were rostered to the maternity wards which had sufficient cases to support their full time presence.
Some students were mature and comfortable in being fully self directed learners, others were much less so. The ideal was to encourage self directed learning and for students to have clinical contact with patients for most of the week. However it became evident that staff desire to maximise clinical exposure to cases, led to fewer opportunities for students to take charge of their own learning. Local coordinators had only three to ten students, knew them, their skillsets and learning needs very well, and became very anxious if the students were not seen in clinics or making the most of learning opportunities. This could lead to more and more micromanagement of student time to the detriment of self directed learning. In turn the students could become confrontational, avoidant or resentful, depending on their personal psychology, at being treated “like a child”. The school therefore moved to a negotiated approach to self directed learning where the student negotiated their learning with the site coordinator.
Student identification with the home site and community engagement
Metropolitan academic staff members are not generally concerned with whether or how much a student relates to his or her home community. However in the rural setting, far from their usual psycho-social supports, it was necessary for students to develop social connections in their new site in order to survive the year in some kind of psychological wholeness. It needed to become their new home. Those who did were happier than those who did not. Students from the site closest to the metropolitan centre who went “home” to the city on a regular basis reported higher loneliness scores than those who identified with the site in which they were studying .18 Students at the furthest sites (over 2000 km away from the city) were far less lonely than those at the closest site (200 km away,) and they also reported having many more social connections to their new home site.
Australia is just one country that is exploring the use of an integrated longitudinal curricula within a well thought-out pedagogy for its medical schools in such a way that students express satisfaction both with their medical education and within the community they practice.19 Experience in Rural Clinical Schools (RCS) around Australia has shown that students perform academically as well as their metropolitan-trained counterparts, 9 10 20-22 that they can learn specialty based content in generalist settings 23 24 but that rural practice, teaching and learning is different to that in tertiary based hospitals. 11 21 25-30 However little has been written about the efficiency of curriculum delivery in rural and remote settings with comparisons about what was perceived to have worked by both staff and students. The importance of clerkship organization, the learning context and activities have been less recognised than the role of the teacher.4 Yet, when setting up a new program or site it is exactly these factors which can determine student and staff satisfaction.
Harden 31 in his paper
What was very difficult for students was when we unintentionally strayed into a mix of trans-disciplinary clinical work (Harden’s Step 11 ) with isolationist academic teaching 31 (Harden’s Step1) where individual tutorials, unrelated to other academic content or their clinical experience, were delivered.
The strengths of this research relates to the multiple triangulations across years, across sites, across different coordinator profiles and with different cohorts of students.
However there are a number of limitations to this study. The first is that the ability to generalise these results to other RCSs around Australia and courses around the world would depend upon their university’s academic requirements. For example, the number of hours of tutorials required would depend upon the course prescription and examination style for that year of their training. A second limitation was the focus of the study. The focus was at the level of the RCS as a whole and therefore was concerned with student outcomes (were marks comparable to city trained students and did the students perceive their experience as generally positive?) and the ability to provide coordinators with recommendations about what was likely to be the most successful way to deliver the course despite various site specific issues and differences in coordinator approaches. Individual coordinators might take a different view based on a smaller number of criteria that suited their personal teaching style at their particular site as compared to the School requirements of meeting multiple teaching styles across a different range of environmental parameters.
Metropolitan tertiary hospital based training is not perfect, either in Australia or elsewhere, 32-34 but it is the baseline against which rural teaching and learning is compared. The curriculum content is largely determined by Medical Councils, and so it is only the order and the way in which it is experienced that is possible to change.
Good and excellent students will almost always be able to make up for lower quality teaching and learning opportunities, although it comes at an emotional and time cost to them. It has been our experience that we do not have to get everything right. For most students, it is enough to be “good enough”. However students do expect us to listen to their feedback and to attempt to make right when we discover unintended undesirable outcomes in the way we organise our curriculum delivery process. What we have endeavoured to do is to identify and articulate a system to deliver the curriculum in rural and remote settings where staffing and resources can be very variable, or missing altogether, even when students are difficult, demanding, manipulative or lazy and even if the medical coordinator is having a particularly difficult month or year.
Ongoing evaluation by means of Action Research methods has been a successful way of implementing quality control in the RCS WA. The use of these methods turned anecdotal concerns into qualitative data and the frequent iterations under very differing environmental constraints and staffing allowed strong usable curriculum delivery systems components to be identified and coded over time.