Introduction

Internet technology has brought worldwide communication to our desktops resulting in a very dynamic educational tool [₁]. As standards of web-based media delivery such as XML or SMIL are realized and as the networks grow faster and wider, development of interactive educational solutions are developing. Distributed educational material can be presented to a wider forum, allowing many to share teaching experiences worldwide. Using this technology the opportunity also exists for encapsulation of an entire course with lectures, text, testing, grading and feedback within an educational module [₂]. As these can be cumbersome to navigate for already busy medical students, some organization of materials seemed warranted.

We sought to develop a software engine that allows integration of teaching materials, facilitates the addition of new materials through wizards, automates the administrative components of the radiology clerkship, and provides a web-based forum for educational discussion. Additionally, we wanted to facilitate evaluating the students by interactively testing them online with automatic grading and feedback. The student's clerkship projects have also been migrated to the web to enhance peer education.

Materials and Methods

In the final year of their curriculum, medical students at our institution are required to participate in a formal radiology clerkship. The immediate objectives of teaching radiology to medical students at our institution are to give senior medical students instruction in requesting radiological investigations sensibly as well as gaining an appreciation of what imagers do and how they function in the health care enterprise. The methods traditionally used in achieving these goals include a wide variety of materials including: web based teaching material, small group interactions, one on one view box experiences with residents, fellows, and staff radiologists. Didactic lectures are provided 2-3 times per day along with directed rotations in the various radiology department sections. Teaching files, books, slides, laser discs, and videotaped teaching materials are also available for self-study. RadClerk aimed to inculcate these traditional means with a web based backbone.

Pre and post clerkship multiple choice examinations (40 questions each) and a film interpretation quiz (9 films with fill in the blank questions) were the standardized means of evaluating each student. The exam questions were randomly chosen from a database of over 2500 questions by computer, pre- and post-test questions were linked with regard to degree of difficulty and content.

To encourage peer learning and preclude repetition of teaching topics, at the end of the clerkship the student uploads a written project through a dynamic interface. This HTML (Hyper Text Markup Language) document with text and images is graded, and at the clerkship director's discretion, the project is transferred to the web server. Once there the project is placed in a database, and distributed via active web components including Active server pages and script modules to serve as a teaching file.

Student evaluations of the elective experience consisted of an evaluation of the lectures and other interactions with the faculty as well as general comments that we used to improve our end of the exercise. In addition, as part of the post test, an evaluation of the entire clerkship was performed with the results analyzed by the Dean of Academic affairs' office. All these data were analyzed to determine any patterns of specific problems in the clerkship as well as the students' overall experience in the clerkship.

Hardware

The server is a Pentium III 450 MHz system equipped with intranet 10-based TX Ethernet connections and a TI Internet connection through a partial firewall. Server operating system is Windows NT server 4.0, running Microsoft's Internet Information Server 6.0, and Microsoft's Front Page extensions (Microsoft, Bothel, WA).

Software design

RadClerk is a modular set of digital web-based applications, which are integrated to provide a complete learning environment for the medical student in radiology. By combining a powerful and extremely flexible user interface, with learning modules, tests, imaging tools, databases and Web technology we have created an all-encompassing educational tool. RadClerk features literature-based reading selections with pre and post evaluation exercises to help develop and strengthen radiology learning. Being online also allows learners to send and receive feedback, and publish their work using the online publishing tool.

Clerkship directors can quickly and easily monitor and assess student activities through cross-tabulated screens after a secure log-in.

RadClerk is divided into Administrative tools and Student Tools. The administrative tools modules are organized into four main components: Site Manager, Exam Manager, Calendar Manager, and Project Manager. (fig. 1)

Figure 1

Site Manager is a module that is used to establish the site for local use.

Exam Manager is for the organization of questions and allowing and disabling testing. Calendar Manager allows the entering and distribution of online events pertaining to the clerkship, including the daily lecture and event schedules and, if necessary, any changes thereto on line. Project Manager is a modular library of tools that can load image and text data from different file formats into the application and thus create the student's presentation and its subsequent incorporation into a web-based teaching file.

Web-based software was designed using Visual Studio from Microsoft Corporation on a Pentium based NT system. Software design was deliberately simple to ease use by new computer users and to support most major browsers. The modular design of the software allows for integration of new components and fast updates. The test was built using a two tiered security system and requires secure logon to assess the test or enter a project. The test must also be activated prior to administration for the active students.

The program is both Internet and Intranet ready, with built in security features that can be enhanced through the utilization of secured sockets layer. The testing module is online, and simple statistical analysis is automatically performed. Cases and questions for the test can be submitted on-line by faculty, and then approved by the clerkship director. Cases are displayed through online publishing of student projects, which they can create anywhere they have web access. Directed learning pages and cases are also preset for the utilization of the students. Student's schedules, email updates and feedback are all acquired on-line.

Administration Tools Design

Database registration of the students for the clerkship can be done remotely. Changing and posting schedules of sessions and learning experiences is permitted. All collected information is entered into a backup file at specified time intervals.

Students Project Design

After the oral presentation the clerkship director decides on its merit. A secured single record is made available to the student to enter his or her project. The written or text component is translated into HTML while images are digitized, converted to jpeg file format, and uploaded to the server. The web pages are then created on the fly by using active server pages and scripting to query the database. The database response creates the page in keeping with the general design of the system. The text is formatted and uploaded to the new page. The images are then embedded into the newly created web page as thumbnails, which can be viewed at higher resolution.

Statistical Analysis

Students paired T testing was performed to compare the pre and post test scores[₃].

Security

A two-tiered security was set up to protect student scores and information as well as testing material. First, a secured log-in must be obtained. This will allow viewing of course materials if they have an account that is active. Second, the test proctor must have the exam enabled in order to administer it. Viewing of the online projects is open to the general public.

Results

A total of 254 students were administered pre- and post- clerkship tests to evaluate effectiveness of the teaching process. A total of 508 conducted test evaluations; of a possible 508 evaluations, all were returned as no grade was given if the evaluation was not completed. Students' pre and post clerkship scores rose an average of 22.3% from pre- to post-testing. (Paired T analysis: p<.0001). The evaluation by the Dean of Academic affairs office reported an overall increase in the level of satisfaction with the radiology clerkship from 53% to 80%.

Discussion

Intrinsic changes in educational systems are being mandated by changes in web technology. Proper implementation of these systems provides a cohesive way to train and evaluate the students and the program. The environment to entice students to fully participate in the process must allow them inclusion. The existing on-line radiological educational projects are not integrated toward building or enhancing a curriculum (₄,₅,₆).

We created RadClerk to address the educational needs that we faced, and in doing such found that we had created a robust educational engine. These efforts mark our continuing steps toward a strategy to construct an Internet-delivered curriculum, offer Internet-delivered learning solutions that combine the viewbox approach with key Internet interactive learning benefits including research, online interaction, and collaboration with peers and/or experts (ref. 5).

Student feed back was excellent with almost 96% preferring the online component over its paper based predecessor with comments like faster, less stress, more convenient. As these results were validated by an external evaluation conducted by the medical school Dean of Academic affairs office by showing the overall level of satisfaction with the radiology clerkship rising from 53% to 80%, we feel that RadClerk has been validated prospectively.

The test was found to be a standardized, reliable and valid instrument that can assist in determining the degree of learning and as such point to defects or shortcomings in the material being taught in the clerkship. Uniformly the students performed 20-25% better on the post-test. Using this new technology RadClerk allows us also to automate some of the most time consuming and unrewarding parts of teaching. Additionally, we are producing substantial utilizable material for feedback to both the student and the program, and in the process creating an interactive growing repository of student created cases that give students pride in their accomplishments and can be used as a teaching aide for subsequent radiology clerkship students.

There have been, to our knowledge, heretofore no published reports of previous substantial attempts to digitize the clerkship. With the web as the forum, there is the possibility of collaboration and expansion of similar teaching tools to further enhance the medical student experience. This might also lend itself to the establishment of a lose consensus of what information is needed by non-radiologist, and thus taught in medical school. Presented is an effective tool for measuring student outcomes in radiology clerkships, and to continue across the board improvements in student performances both subjectively and objectively. We were able to continue improving the clerkship with timely on-line feedback.

In addition, as we showed RadClerk to our scientific and other colleagues, we realized that the tools were applicable to many other educational areas. Anywhere that structured educational activities are carried out, RadClerk can provide the engine to help establish and populate an educational. Some applications include: other clerkships in medicine, nursing, other allied health, basic sciences, technical programs, engineering, and many others. Finally, the major advantage actually turns out to be that RadClerk users do not have to understand how the technology works; they can focus instead on learning.

Author for primary communication