Introduction

The teaching and assessment of communication skills has become integral to the overall process of undergraduate and continuing medical education. Physician communication and interpersonal skills are a critical determinant of quality patient care.^{1, 2} Research illustrates that patient satisfaction,³ compliance,^4,5 recall and understanding of treatment advice,^6-8 disclosure of psychosocial information^9,10 and attendance at scheduled appointments^11,12 increase as a function of improved physician communication skills. Moreover, the importance of physician communication skills has been emphasized in guidelines and objectives for educational programmes,^{13-17, 39,40} consensus statements,^18-20 and professional practice standards.^21,22 Accordingly, effective communication skills are considered a core component of the clinical interview.

Typically, three methods are widely used for assessing communication and interpersonal skills: 1) checklists of observed behaviours during clinical interactions, 2) patient surveys, and 3) assessments of learning (oral, essay, multiple choice, OSCE).²³ Checklists are the most commonly used of the above methods. For example, in the ten-year period from 1986 to 1996 more than 25 communication and interpersonal skills checklists were detailed in the communication-skills literature.²⁴ Furthermore, educators have identified a need for age-specific communication instruments.^25,26 This is particularly evident for adolescent patients.²⁷

Medical students report low levels of confidence in interviewing adolescent patients, and that adolescent patients seem uniquely difficult to interview.²⁸ This may be due to the fact that adolescents can occupy a variety of positions within the psychological, cognitive, and psychosocial domains.²⁹ The adolescent years are a critical period in the development of behavioural traits associated with both positive and negative long-term health outcomes.^30-32. In dealing with the presenting problem alone, physicians may fail to identify underlying reasons for the adolescents’ visit. How then to prepare medical students to conduct an effective adolescent interview?

Several years ago, two of the authors (KB and KM) studied the improvement of senior medical students’ adolescent psychosocial interviewing skills, and found that structured feedback from an adolescent and mother standardized patient (SP) led to significant improvements in the students’ performance.²⁵ This study also served to highlight the need for a structured guide that could facilitate SP-delivered feedback. We developed our guide by adapting a well-known communication guide, the Calgary-Cambridge Guide.³³ We called our tool the Structured Communication Adolescent Guide or SCAG²⁸. The SCAG incorporates adolescent-specific psychosocial data-gathering along with communication skills that medical professionals are expected to use when conducting an adolescent clinical interview.^27,29

In the years since it was first developed, the SCAG has been refined through an on-going research process that has involved adolescent input as a central component for evaluating its effectiveness. Currently the application of the SCAG extends well beyond a simple feedback guide, as research has shown it is a reliable and valid tool for trained adolescent standardized patients (SPs) to use in assessing medical students, residents, and physicians in practice.^28,34 The SCAG is currently employed as a learning tool for medical students in their Paediatric clerkship at five Canadian Medical Schools, where it provides a framework for students to follow when conducting an adolescent interview.³⁵

The SCAG is a 29-item communication and interview skills checklist, and is divided into four sections: 1) Getting Started, 2) Gathering Information, 3) Teen Alone [lifestyles], and 4) Wrap Up (Appendix 1). Embedded in each section of the SCAG are key adolescent interviewing skill domains that should be addressed by a physician during each phase of the interview. For example, Section 1 includes such items as, ‘Discusses Confidentiality’; Section 3 includes the risk taking domains in an adapted HEADSS format³⁶. HEADSS is a well known mnemonic used to remember the key risk-taking elements of adolescent-hood (see Appendix 1 - Teen Alone Section of SCAG). The SCAG also includes General Ratings, (adolescents prefer this term to Global Ratings), for each section. This allows the adolescent to give an overall score for each section. Further, each section provides room for the adolescent to critique the physician. Clear instructions and examples to facilitate the writing of these examples are given (Appendix 1 – right hand column).

The current design of the SCAG has been achieved through formal research and informal feedback from users and physicians.³⁷ This iterative development has produced a concise and simple guide, which is at a grade 7-8 reading level (12-13 years old).

The study goal and research questions

Research questions:

Method

Participants

School Visits

Physician-Adolescent Clinical Interview Videotape

Questionnaire

Ethical Approval

Data Analysis

Results

Participation rates were high from all three schools. From a total of 228 eligible students, 183 participated (80.3%). The mean age was 13.4 years (+/- 0.51) years and included male (N = 94) and female (N = 89). Seventeen females and 28 males were unable to participate, either because they were absent, engaged in other activities or had no consent signed consent form.

The mean total SCAG score (Figure 1-A) and the mean score of the General Rating (Figure 1-B), were not significantly different when compared across age or sex of the adolescent participants.

Figure 1

Figure 1: Mean SCAG Scores by Age and Sex.

Participants’ age, sex and/or school did not significantly affect the SCAG ratings for each section (Table 1).

Figure 2

Table 1 – SCAG by Section Analysis for Age, Sex and School

The General Ratings and total scores showed overall high reliability (Figure 2) both for male and female adolescent scores. When calculated for each section, the reliability remained moderate to high [Section 1 (‘Getting Started’), 0.44; Section 2 (Gathering Information), 0.61; for Section 3 (‘Teen Alone’), and 0.78 Section 4 (Wrap Up) 0.65]., with the exception of Section 1)

Figure 3

Figure 2: Reliability of the SCAG.

Participants’ mean total score for the interview deviated from the ‘Gold Standard’ score by a value of 0.75. This difference was not significant (P = 0.29). The Gold Standard mean scores for individual sections, General Ratings and total score did not differ significantly from the adolescent mean scores (Figure 3).

Figure 4

Figure 3 - Analysis of Deviation from ‘Gold Standard’

Adolescent Written Examples about the physician (Appendix 1, right hand column of SCAG)

Of the 183 SCAG forms completed in the research study, 106 adolescents wrote feedback for the physician in the “comment box” on the right hand side of the SCAG: 47 male and 57 female (106/183 = 58%). A research assistant transcribed all comments. These were then grouped by two of the researchers (KB and SW) to produce themes that typified both the positive and negative comments (Table 2). The adolescents who gave the higher scores of general ratings supported this with positive written examples about the physician, 90% of the time. The adolescents who gave the lower scores of general ratings supported this with constructive comments 82% of the time.

Figure 5

Table 2 : Adolescent written examples grouped by theme.

Questionnaire Results

All participants in the study completed the questionnaire (5 Likert style questions) and the mean values of their ratings are presented in Figure 4-A. Ratings were generally high, ranging from the lowest mean score of 6.0 (sd = 2.7) for question 3 (I would use the SCAG if it was in my physician’s office) to 7.8 (sd = 2.2) for question 4 (I was comfortable with the questions). Males tended to give lower ratings than females (Figure 4-B) however, the ratings did not differ significantly across age groups.

Figure 6

Figure 4: Questionnaire completed by adolescents on their experience with the SCAG.

Of the 189 participants, 103 (55 %) completed some or all of the four open ended questions on the questionnaire. For Question 1, What did you like about the SCAG?) 65 responded: 32 discussed the ease of using the SCAG and 33 dealt with the important issues covered by the SCAG. For Question 2, What did you not like?, 28 responded: 14 wanted more rating choices and 14 described the SCAG as too long and/or too hard to understand. For Question 3, Are there any questions you would change or remove?, and Question 4, Are there any questions you would add?, 91 adolescents reported that they would not have changed or removed any questions while 85 reported that they would not have added any questions.

Discussion

The findings of our study support the view that adolescents, aged 12 – 15 years, can easily and reliably complete the SCAG while watching a videotape of a physician-adolescent interview. Further, the scoring was not significantly different when age, sex and school locations were examined. Scoring of the SCAG by the adolescents was not significantly different from that of the ‘Gold Standard’ scores; lastly, participating adolescents found the SCAG usable and relevant.

We were concerned that the SCAG scoring would differ in the males compared to the females, given what we know about the maturity of the sexes. Also, the female patient in the videotape may have hindered the ability of the male adolescents to put themselves in the “female SP adolescent’s shoes”. This may have been one of the reasons that the reliability of Section 1 ( Getting Started) was lower. Given all the results, however, our study supports the use of the SCAG by adolescent males, a finding not previously reported.

Our reliability analysis suggests that adolescents who have not been formally trained can reliably use the SCAG to rate a patient-physician interview on a videotape.

Results from the Gold Standard showed that, on average, adolescents produced an overall SCAG score that was not significantly different from one produced by a group of SP adolescents, a pediatrician and an SP trainer for the same interview.

The questionnaire that the adolescents completed after they had used the SCAG revealed positive experiences. The lowest scored item was that the adolescents would be less comfortable about using the SCAG in their physicians’ office. This may reflect that adolescents are not sure about discussing these types of issues with their physicians. Many studies support this and the fact that the physicians should be asking the adolescents for their opinions and pr providing a tool like the SCAG to the adolescent for anonymous feedback.

In initiating this research, we had reasoned that there would be numerous advantages to having a structured communication guide that could be used by adolescents for assessing the communication skills of medical students, residents and physicians. Medical students in their clerkship at our University are required to have at least one adolescent patient use the SCAG to assess them. Our study lends indirect support to the use of these assessments in the medical students’ evaluations. We suggest that the medical students should be required to have a number of adolescent patients assess their communication skills to ensure maximum reliability.

We also believe that the SCAG may be a useful tool for physicians in practice to use toward improving their adolescent clinical interviewing skills. In this role, the SCAG provides an easily utilized feedback measure that physicians’ could give to adolescent patients they see in their practice. Our study suggests that adolescent ratings using the SCAG could be reliably used to provide formative feedback to physicians.

A number of the adolescents who have been involved with developing the SCAG have asked “Can I get a copy of the SCAG and use this with my physician”? This is an interesting and unexpected finding of our research. The use of the SCAG as an educational tool for the actual adolescent patient was not directly addressed in this study. However, we would like to emphasize the SCAG’s potential in this regard in part because adolescents often do not receive sufficient preventative counselling and screening service^{37, 38}. Adolescent interviewing needs to be addressed through training of both medical students and physicians in practice³⁹ and further, from a patient perspective ⁴⁰ .

The SCAG is a well studied guide to learning and to implementing a basic adolescent clinical interview; it includes important questions that deal with health-related behaviours that will influence health later in life. Introducing current and future physicians to the SCAG may provide educators with a simple and inexpensive tool that could assist physicians with their vital task of not only getting, but also keeping their adolescent patients healthy.

Evaluating patient-physician communication is a complex and subtle task. Current checklists of observed behaviours are widely used as they are thought to capture essential facets of the patient-physician interaction. Recent research, however, suggests that patient perception of physician communication does not always accord with ratings derived from OSCE-style checklists.³⁵ Hence, in certain instances, it may be more informative for both learners and evaluators to have access to patient (or SP) derived ratings that can serve as an adjunct measure of physician communication skills.

Limitations

Firstly, this was a school-based study of 12 – 15 year olds used as a convenience sample due to the start and finish of junior high school. It is possible that with a sample having a larger age range the SCAG reliability may revealed more substantial differences. However, this is the middle adolescent period where the risk-taking often starts and this age range is a good target for intervention. Secondly, our study included an evaluation of the SCAG based on participants’ ratings of one videotaped interview featuring a female physician and a female SP adolescent only. This may be one reason that a larger number of female participants provided narrative comments in the “examples section” of the SCAG than did male participants. A third limitation of this study is that although participants were from urban, suburban and rural settings, all were from the Halifax regional municipality. This makes it difficult to determine if the results can be generalized to broader cultural populations, or to adolescents who speak different languages. In summary, the overall results from this study were promising; it is clear, however, that additional work is necessary to address these limitations.

Conclusion

Adolescents aged 12-15, who have not been formally trained in using the SCAG, can easily and reliably rate a videotaped physician-adolescent interview. Age, sex, and school location did not significantly change the adolescents’ ratings, nor did their ratings differ significantly from the “Gold Standard”. Overall adolescents found the SCAG easy to use, and the majority would not have changed or added any questions. That the adolescent participants in this study were untrained in using the SCAG suggests that in real-life settings, such as on the wards, or in out-patient clinics, the SCAG may be useful as a structured source of feedback on adolescent clinical interview skills for learners and physicians alike.

Appendix 1

Figure 7