Introduction

The changing face of medical education with an emphasis on clinical competence has resulted in the development of new directions in the content, delivery and evaluation of information. Amongst these is the use of standardized patients (SP) versus real patients (RP). Although simulations are the norm in North America₂ and have been in use in medical settings for some 30 years₁, the use of unstandardized real patients remains a feature of undergraduate and postgraduate assessment in the United Kingdom ². The latter is deficient as it tests applied knowledge (“knows how”) rather than performance (“shows how”) in the Miller's pyramid of competence ², ₃.

In the United Kingdom, several of the medical Royal Colleges have introduced an OSCE component using SP into their postgraduate membership examinations. The Royal College of Psychiatrists has recently proposed changes to the existing membership examinations with a view to increasing their reliability and validity. The main changes are to the Part I examination, and from Spring 2003 the existing individual patient assessment will be replaced by an OSCE, in which SP are likely to be used. The Part II will essentially retain the same format ¹ .

In August 2002, SP were used as short cases for the first time in Part 1 Doctor of Medicine Psychiatry postgraduate examinations at the University of the West Indies, St. Augustine campus and for the first time in postgraduate examinations in the Commonwealth Caribbean.

The Doctor of Medicine Psychiatry postgraduate program is a four year program and consists of three parts: Part 1, Part 11 and Part 111. The Part 1 lasts one year and at the end of this period the postgraduate resident is examined in neuroanatomy, neurophysiology, psychology, psychiatry and neurology. In the clinical examination in psychiatry the candidate must examine a long case and a short case and present his findings to the examiners. The Part 11 lasts two years and the examination comprises two written papers, a clinical examination in psychiatry and an oral examination in psychiatry. The Part III is a research component and it is done in the fourth year of the program.

The aim of this study is to examine the use of SP in postgraduate psychiatry examinations and the feasibility for continuing use.

Method

Three SP, 2 females and 1 male, were trained by lecturers in psychiatry to play the roles.

Three candidates, an Indo-Trinidadian, an Afro-Barbadian and an Afro-Nigerian appeared for the short case in the Doctor of Medicine Psychiatry Part 1 examinations. They were each given thirty minutes to assess the SP. All candidates and some of the examiners were blind to the fact that SPs were being used. Candidates were blind as to whether their tested patient was an SP or RP.

A board of examiners, comprising one external examiner, the university examiner and at least two internal examiners, examined each candidate after the short case was evaluated. The examiners comments were noted.

Six weeks after the examinations a semi-structured interview was conducted with each candidate. The following information was sought from these interviews:

1. What were their opinions on the use of SP for Postgraduate psychiatry examinations?

2. At what time did the candidate recognize that the patient was standardized?

3. How did the candidate know that the patient was a SP?

4. Did the candidate witness the SP coming out of role?

5. Were these candidates in favour of using SPs for the Part II D.M. Psychiatry examinations?

Results

All candidates were male and were doing the D.M. Part I psychiatry examinations for the first time.

All candidates found SP as being very appropriate for testing in postgraduate psychiatry. The 3 candidates felt that the SP patients were more problem focused, were easy to talk to, were believable and did not give irrelevant answers? During the interviews with the SP none of the candidates realized that their cases were SP. Two candidates suspected that their patients were SP before being tested by the examination board. One candidate witnessed his SP coming ‘out of role' shortly after the clinical interview and remarked that he was now behaving differently. Another candidate was concerned about the exaggerated acting of his SP. This candidate stopped the interview and recommended immediate hospitalisation of the patient. This candidate enquired whether his was a ‘real patient' before the board of examiners saw him. The three candidates felt that SP should be utilized in the future. All candidates were in favor of using SP for the D.M. Parts I and II postgraduate examination in psychiatry. Some examiners felt that whilst it was acceptable to use SP for non-psychotic disorders, it was more difficult to train SP to simulate psychotic patients. One examiner was not in favor of using SP in this examination.

Discussion

Despite some skepticism, the use of SP was acceptable to both postgraduate students and the majority of examiners. Students of the Part 1 Doctor of Medicine Psychiatry examination found that SP were more focused and appropriate for eliciting clinical symptoms. Generally examiners agreed that well trained SP enabled more objective assessment thereby raising the standard of clinical testing.

A pertinent argument was that the SP problem-focused approach; concomitant ease of communication and information gathering resulted in an over-simplification of the examination process. It is argued that at the postgraduate level students should be competent in extracting relevant information from a muddled mind. Contrary to this view and as proposed by the Royal College of Psychiatrists, SP are being utilized in short cases at the MRCPsych Part 1 postgraduate examination. The rationale is that this is only one component of the examination that is problem-focused and designed to test safety, competence and communication. Further discussions that SP cannot truly simulate psychoses and will hamper psychodynamic formulations are valid. While actors cannot reasonably present psychosis however, they are certainly able to portray a relative of such a patient, or a depressed patient or a recovered patient who has come for counseling. There is a need to be creative.

The use of SPs in undergraduate medicine is becoming the norm ². The question arose whether it matters to the student if the patient is standardized or real. Several studies on this issue have been inconclusive₄. More importantly, there is an ethical question about withholding from students what is actually happening. Students should be told that the patients are role-playing.

Our three postgraduate students were blind to the use of SPs. They were blind in the sense that they were informed before hand that they may be tested on a SP but did not know if their index patient was indeed an SP. None of the candidates suspected role-playing during the interview. However, one candidate stopped the interview and recommended immediate hospitalization of the patient who was overacting. A second candidate, on seeing his patient sitting in the waiting room, after the interview remarked that he was behaving differently. This affected his clinical conclusions. It is indeed essential that students should be informed before their examination interview that patients are role playing, despite findings that students were unable to differentiate between persons simulating a patient role and those who presented a real history. Proper sequestration of students and subjects away from each other before and after the interviews are also essential .

The introduction of SPs to a postgraduate examination format in the Caribbean is new. On the background of more than twelve years experience in the use of standardized patients for undergraduate students in Trinidad, the Psychiatry Unit took a bold step to introduce it into the evaluation of postgraduate students. SP's were introduced because of the difficulty in finding the required number of suitable cases required for examination purposes. In addition, the department had a bank of simulated patients that have been trained and performing for more than ten years at the undergraduate level.

Our experience has been encouraging. The three candidates tested were in favour of the use of standardized patients. There was more objection from the examiners who traditionally are wary of change. As the process develops, refinement is needed in extensive training of role players to guarantee reproducibility and consistency in clinical presentations. The issues of overacting and exhaustion must be addressed.

This preliminary report is limited by the small number of candidates and SP. However, all candidates found that the use of SP was more appropriate. Not only were the simulated patients more problem-focused and reliable as historians, but the three candidates of varying nationality found their examination subjects more culture fair and they approached their examiners with more confidence.

Conclusion

SP presents an opportunity to achieve a gold standard for the assessment of student performance. It lays emphasis on clinical competence, safety and ease of communication and facilitates a more objective assessment by examiners. It facilitates the elimination of cultural bias. Our experience in its use in postgraduate psychiatric examination has been encouraging. It can be usefully employed in short case assessment. There is a need for creativity and refinement of the process in the future.

Publication Note

Acknowledgement