Introduction

Routine psychiatric outpatient treatment usually consists of a variety of psychological treatments, different pharmacological treatments and a combination of these two primary interventions. Many of these treatments are evaluated in efficacy studies employing the randomized clinical trial (RCT) design. However, each one is usually assessed separately and often for specific disorders in specific settings. At the same time there has been some criticism of these research designs, raising the question of whether the results are generalisable to routine psychiatric practice. Therefore, it has been proposed that there is a need for more effectiveness studies in naturalistic settings (1-4).

Within the last decades an abundance of research has been produced concerning the effects of psychotherapy, and the results are consistent in showing that psychotherapy is beneficial (5-7). There are several ways of describing psychotherapy outcome, and a common measure is that of Hedges and Olkin’s d (8). Cohen (9), stipulated regarding effect sizes, that a large effect is d = 0.80, a medium effect is d = 0.50, and a small effect is d = 0.20. Most meta-analyses and reviews concerning psychotherapy found or showed an effect size within the range of 0.75 to 0.85 (6-7). Further, most research showed that there are small or no discrepancies between different therapeutic schools regarding outcome, and that they all show about the same effect size, a finding often referred to as the Dodo verdict, i.e. ‘all have won and all should have prices’ (6, 10-13).

In many efficacy studies regarding psychopharmacology, the most common method of estimating outcome is to measure whether the patient is considered to be a responder or not. Effect sizes, as in psychotherapy research, are more infrequently used but estimations of antidepressants and anxiolytic medications show a modest effect size of about d = 0.2 to 0.4 relative to placebo (14-15). However, since psychological ingredients also are at hand regarding psychopharmacologic treatment, the cumulative research indicates just about the same ‘all in all’ effect as in psychotherapy. There are also studies comparing various psychotherapeutic orientations and pharmacotherapy, and the results found in these studies indicate that there are small or no difference in effect between different treatments (2, 16-20). However, many of these studies are efficacy studies and clinical trials, and the authors also argue for more effectiveness studies in routine psychiatric settings.

Another question discussed is whether a combination of psychotherapy and psychopharmacology is better than the two treatment forms being delivered separately. Theoretically, there could be an additive combining of the effects but evidence for synergy has not been identified in previous research (15). There has also been a discussion as to whether there is any combination effect at all or if the treatments are equivalent when delivered separately. Research shows somewhat contradictory results, but in a research overview, Thase and Jindal (15) argue that psychological treatments have a significant benefit when added to psychopharmacology for more severe symptoms and for some mental disorders such as schizophrenia, chronic depression and bipolar affective disorder. There are also indications that patients with personality disorders including comorbid disorders will benefit more from combined therapy (21, 22). Although some new studies have found an additive effect (23-24), there is today no evidence that combination therapy is better in depressive and anxiety disorders, which may be the most prevalent diagnoses in routine outpatient psychiatric settings (15-16, 18).

In most outcome studies the outcomes are measured with statistical significance using average effects showing that these are not due to chance. However, statistical tests alone do not provide evidence of clinical significance. Arguments have been raised that the results can be misleading since they fail to reveal the complex distribution of outcome. Some patients may gain substantial effect, some little effect and some may be impaired, but the extent of the individual patient’s improvement after therapy, compared to well-functioning peers, is not captured. Furthermore, a revealed statistically significant effect may be of no practical and clinical meaningful importance. However, a statistically meagre result may also imply meaningful changes in patient functioning, especially if the change implies that the patient moves over the threshold, from a dysfunctional to a functional state.

As a conclusion, the outcome research from each treatment direction shows that there is a considerable effect gained from the respective treatments compared to non-treatment and the effect sizes appear to be rather alike for the various treatments. The research also indicates that an extra combination effect of psychological and pharmacological treatment seems rather modest. However, most studies have been in the tradition of efficacy studies and clinical trials, with diagnostic groups and treatments strictly specified in advance. There is a lack of naturalistic research with clinically representative patients and treatments. This was an exploratory study trying to meet these needs and the aim was to study the effectiveness of psychological, pharmacological and combination treatment in a routine psychiatric outpatient unit where the treatments the patient received were decided, not in advance, but by the normal routines at the unit and were performed by a variety of staff, representative of a routine psychiatric outpatient unit. Furthermore the aim was to evaluate whether the patients improved significantly and whether the change was clinically significant.

Material and methods

Participants and setting

The sample of patients was selected from a psychiatric outpatient unit in the main town of a medical-care district in southern Sweden. The unit had the responsibility for patients with all kinds of psychiatric diagnoses in a catchment area of about 55,000 inhabitants, both rural and urban. No acute admission was available at the unit because acute admission was centralised to a unit responsible for a greater geographic area. Also, most of the new-onset psychoses were admitted to a treatment unit specially designed for psychoses. The patients could be referred to the unit from other caregivers or they were admitted by self-referral. The unit was staffed with a multi-professional team consisting of psychiatrists, psychologists, nurses, social workers, an occupational therapist, and physical therapists. The treatment provided by the unit offered a mixture of common pharmacological treatment and psychologically/psychodynamically orientated therapy. All the personnel at the unit had access to supervision.

The study included patients who were measured on two different occasions. Firstly, all new patients admitted during a period of approximately five months were asked to participate in the study. Patients admitted for just one planned consultation were excluded, as were patients who could not understand oral and written instructions, i.e. patients who were too sick to participate or who could not understand the Swedish language. A total of 181 patients were newly admitted to the psychiatric unit during the research period. However, 32 of them met the exclusion criteria; two were too sick to participate (acute, severe psychosis), three could not understand Swedish, 12 could not participate due to severe dementia/dyslexia/mental retardation/organic brain damage, and 15 were admitted for just one consultation. A total of 149 patients were asked to participate in the study and 122 (82%) completed the first measure assessment.

The second measure assessment took place consecutively when the patients had finished their treatment. Since some psychiatric treatments take a very long time and for some patients in psychiatric treatment, are never completed, the time for the latest follow-up was set to be 18 months and measurements were only taken on patients who had finished their treatment by that point. Patients who attended the unit only once or twice were excluded since it was conceived that no therapeutic process was initiated and fulfilled with so few attendances. Of the initial 122 patients 16 were excluded (only one or two visits) and of these, seven were directly referred to other caregivers, whereas nine chose to directly terminate the contact with the unit. Of the 106 patients who met the inclusion criteria 17 were still in treatment and 13 did not want to participate, therefore the sample of those who completed assessments at intake and 18 months consisted of 76 patients (85%). Of these 76 patients, 27 came to the treatment unit by self referral, 24 was referred from primary care units, 18 from other psychiatric units and 7 from other somatic care settings.

The group of 17 patients still in treatment after 18 months consisted diagnostically (ICD) of; one patient belonging to the strata F10-F19 (mental and behavioural disorders due to psychoactive substance use), one to F20-F29 (schizophrenia, schizotypal and delusional disorders), ten to F30-F39 (mood, affective disorders), four to F40-F49 (neurotic, stress-related and somatoform disorders) and one to the F50 (eating disorders) strata. Of these 17 patients, six belonged to the pharmacological treatment group and eleven to the combination treatment group.

Of the 13 patients who dropped out, one belonged to the strata F10-F19, one to F30-F39, nine to the F40-F49, one to F50 and one to the F60-F69 (disorders of adult personality and behaviour) strata. Of these 13 patients, three belonged to the pharmacological treatment group, five to the psychological and five to the combination treatment group.

Figure 1

Figure 1. Flow chart and number of participating patients

Procedure

Instruments

Treatments

Data analysis

Results

The demographic data are presented in table 1 and all the different variables in BSI and IIP are shown in table 2. There was no significant difference between the three treatment groups concerning the intake data regarding all the variables in BSI and IIP showing that the treatments groups began the treatment with roughly equivalent levels of disturbance.

Figure 2

Table 1. Characteristics of the sample (n = 76)

Figure 3

Table 2: Intake and post-treatment data, effect sizes and p-values

The demographic data concerning the treatment groups did not differ regarding gender, but regarding age, a one-way ANOVA showed that the psychological treatment group differed from the pharmacological treatment group (p=.001) and from the combination group (p = .035) in that the patients were younger. Concerning the diagnostic variables and the treatments groups, the analyses are unsure due to there being too few patients in the cells. However, a Fisher’s exact test showed that there was a significant difference (p = .001) from a representative distribution. This can be largely attributed to the fact that there were more patients belonging to the mood (affective) disorders patients in the pharmacological treatment group and neurotic, stress related and somatoform disorders in the psychological treatment group.

No significant difference between the psychological (m = 8.1) and pharmacological (m = 5) groups was found regarding the number of treatment sessions, but the combination group differed from the others in that it used more treatment sessions (m = 16.8, p = .001).

The output measures, effect sizes and p-values are presented in table 2. When comparing the three different treatment methods, the effect size for the General Severity Index (GSI) was for the psychological treatment group d = 0.73, for the pharmacological group d = 0.90 and for the combination group d = 0.75 and for the total score in IIP the corresponding effect sizes were d = 0.50, d = 0.48 and d = 0.32. Paired-samples T-test between intake- and post-treatment data regarding the treatment groups was also made and the p-values are shown in table 2. The results showed that there were significant changes in many outcome variables, and that the treatment showed better effectiveness in the symptom based outcome measures (BSI) as compared to the interpersonal measures (IIP).

An ANCOVA was used to analyse whether there were any differences between the various treatment groups concerning the outcome. The three treatment groups were used as the independent variable and the outcome of the different variables measured at post-treatment was used as the dependent variable and the data at intake as a covariate. Further, the variables that differed between the treatment groups, age and ‘number of treatment sessions,’ were used as covariates. The analysis regarding the General Symptom Index in BSI (GSI) showed that there was no significant difference between the three treatment groups, F (2, 70) = .53, p = .593. Similarly, on the total score in IIP no significant difference between the various treatment groups was found, F (2, 70) = .58, p = .56. Further, the same analyses were made on all the different subscales in the BSI showing that there was no difference between the treatment groups F (2, 70) = .08 – 1.99, p = .144 – .925. Also for the variables in IIP, no difference was found F (2, 70) = .04 – 0.88, p = .418 – .958. The same ANCOVA analyses were also made with only the psychological/psychodynamic treatment versus the combination treatment as independent variable. The results showed that there was no significant difference between the two treatment groups in any of the subscales regarding BSI; F (1, 59) = .1 – 1.65, p = .204 – .759 and IIP; F (1, 59) = .002 – .80, p = .375 – .965.

Another approach to the question of effectiveness is the use of clinical significance. The results regarding CS and RCI are presented in table 3 and 4 and the statistical analyses were made by Fisher’s exact test.

Figure 4

Table 3: Clinical Significance (CS) as defined by Jacobson and Truax (1991)

Figure 5

Table 4: Clinical significant change according to the RCI (reliable change index) as defined by Jacobson and Truax (1991).

The results showed no statistical difference between the treatment groups with respect to problem severity at intake. Within treatment analyses from intake to post-treatment, regarding clinical significance (CS), showed that there was a significant difference for the combined treatment group (p = .044) but not for the psychological/psychodynamic (p = .064) nor for the pharmacological treatment group (p = .371). These results indicate that there was a significant change in number of patients from a dysfunctional group to a functional group in the combined treatment group, however not in the psychological/psychodynamic and the pharmacological groups. Regarding RCI there was no evidence of any differential treatment effects among the three groups.

Since not all patients who started their treatment finished their treatment after 18 months, an additional analysis, intent-to-treat analysis, was made. In this analysis, the patients from the intake sample who were still in treatment (n = 17) or who had dropped out (n = 13), were also included. The intake data was used as post-treatment data for those patients. The analysis showed that the effect size for the General Severity Index (GSI) was for the psychological treatment group d = 0.58, for the pharmacological group d = 0.61, for the combination group d = 0.71 and for the total score in IIP the corresponding effect sizes were d = 0.39, d = 0.35 and d = 0.28. An ANCOVA was also performed and the results showed that there were no differential effects between the treatment conditions.

Discussion

This study was an outcome study investigating the effectiveness regarding three different treatment groups, psychologically/psychodynamically-oriented-, pharmacological treatment ant its combination in a routine psychiatric unit. A consequence of the consecutive assignment of patients to the study was that the treatment groups varied in size. Especially the size of the pharmacological sample was very small and therefore it is important to notice that in the subsequent discussion any non-differences for this group versus the other treatment groups may be due to the very small sample. Altogether, the results from the study showed that the patients improved substantially. The effect sizes on average correspond to previous findings from efficacy and effectiveness studies (6-7, 14-15) indicating that treatments performed in a routine practice may also have a substantial impact. The results also showed that there was no difference between the treatment groups, a finding that have also been found earlier in efficacy and effectiveness research (14-16, 18).

There are previous studies showing that different psychological orientations are equally good in naturalistic settings (30-31). The present study supports these findings, showing that psychodynamic treatment in a routine psychiatric practice produces effect sizes similar to other psychotherapeutic orientations. Another interesting result is that the present naturalistic study supports previous findings from RCTs that there is no or only small additional benefit from combined treatments compared to single treatments with these kind of diagnoses (15-17). Furthermore, the finding that the effectiveness of the psychological/psychodynamic and the pharmacological treatment is similar is also supported by recent research regarding both depressive and anxiety disorders (17, 32).

The only results where equal effectiveness between the treatment groups was not found concerned the clinical significance (CS). Here, a weak tendency to a difference between the treatment groups was found. The analysis indicated that the combination treatment group showed a significant change in number of patients reaching the functional level after the treatment but this was not found in the psychological/psychodynamic and pharmacological groups. However, the results are very unsure due to the small sample size.

Since no control group was used in this study and the treatment groups differ in various aspects (including diagnosis and treatment length), it is impossible to compare their relative efficacy. However, all treatments proved to be rather efficacious in this routine psychiatric practice and they all showed about the same effectiveness. The fact that different psychotherapeutic orientations have equal effect has led to an interest in theoretical understanding of the process of change in the patients and the most common explanation is the common factors approach (6).

However, in this kind of routine psychiatric setting there may be supplementary explanations. One reason most likely present in this study is that there might have been a selection in the clinical assignment of the patient, based on the clinical judgment of the most suitable treatment for him or her. Another interpretation of the results may be that the patients received what they need to improve to a satisfactory or good enough level of outcome. It is evident from the results regarding CS and RCI that the patients improved, but it is also evident that many patients still had significant problems after the termination of their treatment. In a time of limited resources, the staff typically tries to offer the patients a treatment that gives the patients what they need to accomplish at a satisfactory level of outcome. Instead of being randomly assigned to a treatment or a treatment that had to follow an in advanced determined manual or a pre-determined programme, the patients were offered a therapy which, from the initial assessment and along the course of treatment, could respond to the patients’ immediate needs. There were no fixed conditions such as the number of treatment sessions, or whether the patient should receive pharmacological or psychological treatment; the patient could have both if he or she needed it. The results indicate some sort of regulatory processes, and there has been a theoretical discussion regarding responsiveness and self-regulatory processes by the patient and therapist for achieving a good enough outcome (30, 33).

The above reasoning seems to be consistent with the results from the present study. This reasoning is also consistent with the finding that the combination of psychological and pharmacological treatment did not show an additive effect. Instead the combination group required a significant increase in the number of treatment sessions as compared to the other treatment groups; at the same time the combination group also reached the same level of outcome when the treatment was terminated. A reasonable explanation of this phenomenon is that the patients needed more treatment (both pharmacological and psychological and more treatment sessions), and that the treatment was not finished until the patient reached a good enough level of outcome. The logic of this reasoning regarding responsiveness and a satisfactory level of outcome does not argue against the common factors explanation. Instead, it suggests that an appropriate treatment is a treatment that offers the patient what he or she needs, given the available resources. An important question that arises from the results is what characterises the patients in the combination group and the group of patients not yet terminated (i.e., the patients that use the most resources). The results showed that these groups consisted of patients that needed more treatment. One hypothesis may be that these patients had more severe problems; however, the results from the present study give no answer to what kind of problems or what characterises these groups. Instead, the results showed that none of the measured intake variables differed between the treatments groups. This indicates that there may be other factors in the patients, not captured by the GSI and IIP that determine the treatment required, which is an area for future research. The only intake variable that differed was the variability in diagnosis among the treatment groups. However, it was shown in another study regarding the same sample, that no difference in outcome between the diagnosis groups at the unit in any of the outcome variables was found (29). No statistical analysis whether there was any difference in outcome between the treatment groups in relation the various diagnosis groups was made due to the small number of participants in some of the groups.

There are many limitations in the study. First, a limitation is that the treatment groups were fairly small and also the unequal sizes of the treatment groups. This leads to a restricted power and a risk of type II error, implying a limited ability to detect group differences. This applies to all treatment groups but is especially evident concerning the pharmacological group and any non-differences for this group versus the other treatment groups may be due to the very small sample. Also the analyses regarding CS and RCI are very unsure due to the small sample sizes. Further, no control group was used which means that all positive outcome may be credited to a natural remission of symptoms and cannot with certainty be attributed to the treatments. Threats like maturation or other external influences may be present. However, each treatment is separately shown in randomised trials as having a documented effect, so this is not very likely. Further, the fact that no random assignment of patients to the treatments was made implies that selective referrals cannot be ruled out. However, it would be ethically impossible to withhold treatments from patients in this kind of public service psychiatric setting and randomisation would also undermine the clinical representativeness. Instead, the routine psychiatric unit with its heterogeneity of staff and patients representative for this kind of setting, and the sampling method, consecutively selecting all patients within a certain time period, and a high participation rate, should not have inferred any systematic bias. Rather, it may have strengthened the clinical representativeness. The unequal distribution of diagnosis across the treatment groups may nevertheless be a limitation to the generalisability of the study. Also, a limitation may be the impact of the dropout group. However, the intent-to-treat analysis did not reveal any differences concerning the results, indicating that the dropout did not seem to affect the validity of the study in a decisive way.

Conclusion

The results showed that that the patients improved substantially and that there was no difference in outcome between the different treatment conditions. The results also indicate that in these kinds of psychiatric treatments the patients are offered and tend to achieve what they need to obtain a minimum satisfactory or good enough level of outcome. A hypothesis may be that the treatments are determined, in an effort to offer the most suitable treatment for the patients, by the responsiveness and regulatory processes of the staff and patients, rather than from a pre-determined programme of treatments and methods.