S Smith, N Roberts, M Partridge
S Smith, N Roberts, M Partridge. What Factors Influence Postgraduate Medical Trainee Attitudes To Computer-Based Learning?. The Internet Journal of Medical Education. 2009 Volume 1 Number 2.
Computer based learning is an attractive solution to the increasing problem of providing self-directed/directed continuing education for medical postgraduate trainees on multiple sites with fewer teachers and clinical resources available. This study explores the practicalities and views regarding e-learning of trainees working in three hospitals in the same NHS Trust in London. Use of nominal group technique indicated that ease of access to computers and the internet was ranked highly by all groups and may be a significant barrier to trainees' use of e-learning. Use of questionnaires to explore views in more depth demonstrated that not all junior postgraduate trainees made use of e-learning materials prior to planned teaching sessions, despite requests to do so; spoken and written reinforcement may be needed to increase use of e-learning materials. Telephone interviews with senior postgraduate respiratory trainees indicated that they seem receptive to the positive benefits of using e-learning as part of their postgraduate training as long as social interaction and face-to-face teaching on study days is maintained. The exact function and content of each e-module and whether it needs to be compulsory or optional need to be determined if there is to be successful implementation of e-learning.
Increasing numbers of medical students and the introduction of more formal structures of postgraduate medical education, are combining with other factors (such as the limitations based on working hours by the European Working Time Directive and a reduction in specialist autonomy) to reduce the staff time available for postgraduate clinical teaching. The need to meet this shortfall, coupled with a proliferation of authoring tools and an increased ease with which computer-based materials can be created and distributed to learners, has resulted in an increase in the number of computer-based, self-directed learning packages available for both undergraduates and postgraduates. Computer-based learning is perceived as a particularly attractive way of achieving both self-directed and directed continuing education of the postgraduate clinical community, where trainees may be geographically scattered and working variable hours1. E-learning packages are also being increasingly used in the education of other health care professions. These trends in postgraduate medical education have been explored in some detail by Harden2
Whilst many computer-based packages are carefully evaluated before being recommended for permanent inclusion in a training programme3 others are being developed and introduced as a short-term response to an immediate need, for example, the need to change the practice of primary care clinicians4. The methods used for evaluation are highly variable. The use of the randomised controlled trial is considered to be the gold standard in clinical research, but its use in educational studies, although not unknown3 is contentious for a variety of reasons5.
One problem is that, when used, randomised controlled trials comparing two or more different teaching formats rarely show differences in learning outcomes 6,7,8,9 neither do they consistently show variation in acceptability to learners of different learning formats 10,11. Indeed, it has been argued that a lack of demonstrable difference in outcomes between e-learning packages and other forms of learning indicates the need to move beyond comparative research in favour of studies on how and when (rather than if) to use e-learning12. For example, a study on final year medical undergraduates demonstrated that there was no difference in immediate transfer of respiratory knowledge between a computer package and tutor-lead formats, but use of the computer package conveyed a small, but statistically significant, advantage with respect to the interpretation of spirometry12. Despite the quantifiable, objective advantage of the computer-based package, in this particular study students displayed a strong preference for tutor-lead, interactive teaching of respiratory medicine. Importantly, the data did suggest that students who had prior exposure to the computer package were more willing to consider its use than those who had not13 suggesting that better exposure to computer-based learning might remove a significant barrier to wider acceptance of this type of educational tool, at least, amongst undergraduates. It is unclear whether or not this also applies to postgraduate medical trainees.
Since e-learning seems, in most studies, to be at least equivalent to and occasionally superior to, tutor-lead formats with respect to knowledge transfer and understanding, non-educational factors are likely to determine whether or not e-learning packages become a dominant feature of the medical curriculum both at undergraduate and postgraduate level. Such factors include the ease and cost with which the materials can be produced, and the willingness of both staff to produce suitable materials and of students to use them, since, if learners are not prepared to engage with a particular learning resource, its quality is irrelevant. In our studies on undergraduates, we have already identified lack of familiarity as one factor that may prevent them from engaging with self-directed e-learning13. Secondly, our experience shows that if a package is not embedded within the undergraduate curriculum, and the students are not informed that its content will be assessed, medical students are unlikely to use it, however useful it may be. Thirdly, e-learning formats, which are often designed for private study, may be better suited to individuals with particular learning styles.
Most published studies are on undergraduates, but the approaches to e-learning of undergraduate and postgraduate trainees may differ. Whether the same barriers apply to postgraduate trainees is unclear. Factors which could be important include the perceived relevance of the material to the learners, the level of postgraduate training, the specific specialism being studied and the geographical context. A search of the literature indicates that the variable which consistently transcends all others is, perhaps unsurprisingly, the perceived relevance of the material to the training programme being pursued14,15,16
In this study, we used a variety of research tools to explore the views of medical postgraduates on the implementation of e-learning into the postgraduate curriculum. We explore three specific areas: the extent to which postgraduate trainees use e-learning, the factors that influence whether or not they use e-learning, and their views on the integration of e-learning methods into the formal fabric of postgraduate training.
Do postgraduate trainees use e-learning in respiratory medicine?
In the UK trainees in general medicine who are less than 2 years post-graduation (UK Foundation Yr 2 [F2s]) are required to attend compulsory face to face training sessions17
For eight teaching sessions, all postgraduate trainees based at Hammersmith and Charing Cross Hospitals were contacted by email one week prior to a scheduled postgraduate respiratory training session and asked to access respiratory e-modules prior to the training session. Because some postgraduate trainees reported difficulty accessing NHS emails both an email and letter was sent for the last cohort of trainees in this study (n=21). A questionnaire was constructed by the authors and piloted for face validity on junior doctors attached to clinical staff in the home department. We checked for content validity by piloting on consultant members of the teaching faculty. We elected to not provide a tight definition of terms such as “minimal” and “moderate” as we were not interested in respondents’ absolute exposure to e-learning. In contrast, we were trying to gauge the trainees’ individual
What influences postgraduate trainees’ involvement with e-learning?
Specialist registrars at all of the Imperial College Trust hospitals (Charing Cross, St Mary’s and Hammersmith) were emailed and invited to participate in one of three nominal group technique sessions at Hammersmith Hospital and Charing Cross Hospital. The nominal group technique was designed to elicit opinions of e-learning amongst 21 postgraduate trainees who attended one of the three sessions. Nominal group technique is carried out in a group environment and utilises an “expert” group or panel. In this study
Respiratory trainees’ views about implementing e-learning into postgraduate training
Semi-structured telephone interviews were undertaken with 13 specialist respiratory registrars to discuss their views about medical postgraduate training and e-learning. The questions were piloted as before and the interviews were undertaken by one of the authors (SFS) who had no professional or personal relationship with any of the interviewees. Calls were recorded with the knowledge and permission of the interviewees and interviews transcribed and themed by all three authors independently.
Ethical review and informed consent
Ethical review was undertaken and permission to carry out these studies was granted by the Head of the North West Thames Foundation School. All subjects gave their informed consent prior to participating in the study.
Do postgraduate trainees use e-learning in respiratory medicine?
The postgraduate trainees (n=98, 66 females) who took part in the initial questionnaire were one (56/98, 57%), two (29/98, 30%) or more (13/98, 13%) yrs post graduation. Whilst 13% (13/98) had no previous experience, 22% had (22/98) minimal experience of e-learning and 63% (62/98) had moderate/high experience. One person failed to indicate their level of experience. Only 12% (12/98) had no internet access at home. Approximately one third of the group (37%; 36/98) used the internet daily for educational purposes (that is, for extending their clinical knowledge irrespective of whether or not this was in preparation for formal assessment), whilst 55% (54/98) used it one to four times per week. The remaining eight failed to respond to this question. A majority of trainees (64; 65%) spent between one to four hours per week updating their clinical knowledge using all modalities.
Only 40 participants out of the total of 98 (41%) accessed the e-learning modules prior to respiratory teaching sessions. Previous exposure to e-learning did not affect the proportion accessing the materials. Of those with low/minimal exposure, 36% (13/35) accessed the modules, whilst 44% (27/62) of those with previous moderate/high exposure did so (p=0.451 Mann Whitney). This is illustrated in Figure 1.
Of the 77 trainees contacted by email alone, only 33% (25/77) accessed the modules, but of the 21 trainees who received both a letter and email reminder, the access rate was higher (71%, 15/21, p=0.002, Mann Whitney. When asked why they had not accessed the e-learning material, most stated that they did not know about it, or did not receive the email that had been sent, had no time, or had no access to their hospital email away from the hospital site (Table 1).
What influences postgraduate trainees’ involvement with e-learning?
On average, participants (n=21) in the nominal group technique study had 6yr postgraduate training (mean age 31 years; 11M, 10F). Approximately one third (38% 8/21) had had moderate/high previous exposure to e-learning. Half (52%; 11/21) were graduates from a UK medical school. Table 2 shows the top 10 ranked responses from each of the three nominal groups. Ease of access and availability of the internet was identified by all three groups, as well as the ability to learn in your own time to fit your schedule. Objective self-assessment was cited highly by two groups. Negative points about e-learning were cost, the loss of personal interaction and a preference for working in groups, technical problems and difficulty reading on-screen.
Respiratory trainees’ views about implementing e-learning into postgraduate training
Of the 13 trainees [age 34±3; 11F, 2M] who volunteered to take part in a telephone interview, 10 (77%) were UK medical school graduates, and six (46%) graduated before 2000. All of the trainees stated that they had either minimal or no exposure to e-learning as undergraduates. All had internet access at home and 10 (77%) used the internet for educational purposes for two to four hours per week. Six (46%) spent between two and four hours updating their clinical knowledge per week. Three out of the 13 trainees (23%) downloaded educational material to an iPod. A majority (9; 69%) thought that e-learning should be embedded into the training programme although some suggested that initial piloting and training were required.
When trainees were asked how many training days could be potentially replaced by e-learning, responses varied from none to a
“I quite like e-learning because I can take a lot on board. But I also like the social interaction in meeting the other respiratory registrars so I think may be a maximum 50%”. SPR-2
The ideal length of an e-module was thought to be under two hours by the majority of respondents (54%). A significant minority, (5;39%) suggested between two and four hours, but this was dependant on the topic and whether users could interrupt the module and restart where they had left off without loss rather than return to the beginning. Most respondents (9; 69%) thought that e-modules should be a compulsory component of post graduate certification. If this were put in place, nearly all (12; 92%) would complete the modules at home because a lack of free, uninterrupted study time made e-learning difficult to complete at work.
“There is no reason why it shouldn’t [be a compulsory component of post graduate certification]. At the moment we are having to compulsorily attend lectures to finish, to be signed off at the end so there is no reason why in the future. I mean obviously all these things would have to be piloted and at the beginning you wouldn't want to make them compulsory and you know you want to test them and make them user friendly first” – SPR-8
Specific diseases, bronchoscopic anatomy, physiology, lung function, radiology and interpretation of x-ray or spirometry results were suggested as areas that would lend themselves to delivery via e-learning.
Individuals involved in medical teaching face increasing pressures compared to past generations. Advances in health care delivery have increased demands on academic resources, resulting in less teaching time19. There is also less access to a wide range of teaching cases when increasingly care of long term conditions is being delivered in a community-based setting20. Many research studies, including a previous study within our department13, have shown that undergraduate students are happy using e-learning and have high learner satisfaction rates, especially once they have gained some familiarity with the technology21, 22. In contrast, there is a smaller body of literature examining medical postgraduates views’ and attitudes about e-learning23, 24 despite the fact that many learned societies and professional bodies make extensive provision of e-learning materials for their postgraduate trainees (e.g. Royal College of General Practitioners25; European Respiratory Society26). Most e-learning studies focus on cost-effectiveness, increasing satisfaction rates27, and measuring increase in knowledge. Our study set out to examine in more detail whether trainees used materials, whether they rated e-learning tools and whether these could be implemented into postgraduate medical training
Use of e-learning
Our study suggests that there may be a mismatch between self-reported use and actual use of e-learning materials. Whilst 92 % of junior trainees reported using the internet for educational purposes at least once a week, less than half actually accessed the e-modules prior to their teaching session, despite requests from the teaching administrators. The initial questionnaire showed that only a minority of junior trainees made use of e-learning materials prior to planned teaching sessions. This may in part reflect contact problems with employer-based email accounts since, when a letter was used to contact trainees, use of the materials increased. (In our hospital group, when using hospital networked computers, there is access to hospital or affiliated university email accounts only.) Our results also show that encouragement to access e-learning materials may need spoken and written reinforcement. Even with such methods, however, usage was only 70% - other reported reasons for non-usage included lack of access, technical difficulties and a lack of time.
If e-learning is to become a significant element of the junior trainee curriculum, it may need to become compulsory. Novel ways of embedding e-learning into the curriculum may need to be utilised.
Perceived value of e-learning
The nominal group and telephone studies confirm the perceived advantages and disadvantages of e-learning and the need for optimal methodologies. The importance of access was reinforced by the nominal group work which indicated that ease of access to computers and the internet was ranked highly by all groups. A lack of access may be a significant barrier to trainees' use of e-learning. Cost (paying for e-module access) was cited as a barrier by 2 of the 3 groups. We were particularly interested to discover that many postgraduate trainees choose to pay for online materials even when free-to-access material of equal quality is available.
Overall, it appears from the nominal group work and telephone interviews that senior postgraduate respiratory trainees seem more receptive to the positive benefits of using e-learning as part of their postgraduate training than juniors. However it is important to note that the former were volunteer participants in our study, whereas the juniors were attending a normal timetabled training session.
Implementation of e-learning into postgraduate medical training
Specialist trainees reported that in some disciplines, viewing an e-module is regarded as compulsory (e.g. viewing a module of cardiotocography prior to obstetric trainees starting in the labour ward; SpR-2, personal communication). Where emphasis is being placed on full documentation of training (e.g. Royal College of Obstetricians and Gynaecologists28), online modules may be regarded by NHS Trusts as a verifiable and cheap method of demonstrating that trainees have received adequate training in particular subjects. However, without careful assessment, there is no evidence that the training session has been completed and the necessary understanding adequately internalised by the trainee.
It is important to consider the intended purpose of the e-modules. The decision whether each e-module needs to be compulsory or whether it should be viewed as a resource to enhance trainees’ self-directed learning is a crucial one. Provision of e-modules to all trainees and specialist registrars to ensure equal access to respiratory knowledge is admirable, but enforcing compulsory completion of e-modules will not necessarily ensure that they achieve practical competence. The approach needed may vary according to subject matter and speciality. For example, trainees may be encouraged to voluntarily view some material on “Emergencies in Cystic Fibrosis” or “Criteria for lung transplantation” (rare topics to which they may not have had much clinical exposure), but be
During the telephone interviews, trainees suggested a multitude of different topics that would lend themselves to delivery by e-learning, in particular, bronchoscopic anatomy, physiology and interpretation of x-ray or spirometry results. What is now needed is for careful thought regarding which parts of a speciality training curriculum are best delivered by clinical apprenticeship and observation, which are best delivered by seminars and face-to-face teaching and which are best delivered by self-directed learning, including use of e-learning modules.
The extent to which computer assisted learning may be substituted for true clinical experience is limited by the fact that the majority of learning materials provide no opportunity for practical skills acquisition. Whilst use of simulators in craft training is increasing29, the availability of simulators able to replicate complex scenarios is currently limited and costly. In contrast, introduction to new knowledge or subject areas may be more usefully undertaken using e-learning. One example could include acquisition of background theoretical knowledge regarding complex areas (such as orphan lung diseases or selection for lung transplantation). Another could include preliminary exposure to interactive video recordings and modules concerning practical techniques (such as thoracoscopy, use of non-invasive ventilation and intercostal tube placement). Other subject areas drawn from respiratory medicine that may lend themselves to e-learning modules are the understanding of bronchoscopic anatomy and the interpretation of lung function tests, where different trainees work at different rates and the ability to go back over uncertain areas may be advantageous.
Embedding of such materials within the training curriculum may be done by simply, for example, by suggesting to trainees that one e-module be completed per term. Use of the materials should in some way be related to an opportunity for discussion of the content with teachers and other trainees. This could be an online exchange, a face-to-face discussion of a particular module, a presentation by trainees on what they gained from the module, or by the use of interactive quizzes.
In some specific circumstances it might be necessary for safety reasons to mandate that a particular module be undertaken (for example, before assessment of competency to insert chest drains). However, there is no certain connection between compulsion and outcome and such practices may be defensive rather than being well supported by the educational literature. As the technical problems associated with e-learning become fewer, it appears that trainees’ willingness and acceptance of their use becomes greater. Time to utilise such materials is limited during the working day however, so the timescale for their use needs to be generous. Furthermore, the materials must be produced in such a way that they are attractive and welcomed by trainees.
Loss of social interaction
A recurrent theme in studies with specialist registrars was the social element of training days. All participants rated the social interaction and face-to-face teaching on study days as highly important. Interestingly, most recognised the need to undertake e-learning in their own time, but wanted private study to be an adjunct, rather than a replacement, for training days.
Weaknesses and limitations of the study
Junior trainees were attending timetabled training days, whilst the senior trainees who contributed to the nominal group and telephone studies were volunteers, who may have had a particular interest in e-learning. Thus, unlike the juniors, the senior trainees may be less representative of their peer group as a whole.
As with all studies in medical education, we are limited by the specific context in which the postgraduate education is being delivered, in this case, two London-based hospitals within a single National Health Service Trust. Previous studies show that there is variation between European countries and that British trainees are more reluctant to engage with e-learning than many of their European peers23. Thus, our results may not be generalisable to other parts of Europe or even to other hospital contexts with the United Kingdom.
Our preliminary investigations using the nominal group technique indicated that difficulties of accessing internet materials and computers can be a significant barrier to the use of e-learning, particularly in the workplace since, in the United Kingdom, NHS computers are usually protected behind firewalls in order to maintain a high degree of security. Provision of less heavily guarded internet-linked computers might be one method of facilitating the use of e-learning materials at work. Our questionnaire based studies with junior trainees demonstrated that spoken and written reinforcement may be needed to increase use of e-learning materials. In contrast, senior postgraduate respiratory trainees were attracted by the ability to access materials at home in their own time and were open to the advantages of e-learning, to the point of accepting a requirement to complete e-learning modules as a formal element of their specialty training. However, it was clear that they greatly prize the social interactions with peers and supervisors provided by study days and were not prepared to completely forfeit this aspect of their training.
The detailed content of each e-module and its role in the trainees’ programme needs to be clear to learners for optimal embedding of e-learning to be achieved.
This study was funded by an educational grant from the European Respiratory Society.