What is probably the most important factor in determining the success of psychotherapy?

The Art of Finding the Right Control Condition for Psychotherapy Studies

Various forms of psychotherapies have been shown to be efficacious and cost-effective for a wide spectrum of mental disorders. The reference policy of the American Psychological Association (APA) developed for Evidence-Based Psychotherapies (EBP) [1] follows the guidelines by Sackett et al. [2]. This APA policy emphasizes the need to integrate the best available research with clinical expertise in the context of the patient’s culture, individual characteristics, and personal preferences [3]. Concerns have been raised about the generalizability of the findings from randomized controlled psychotherapy trials, since the conditions and characteristics of such efficacy trials differ significantly from routine clinical practice (effectiveness perspective). This is all the more important in the field of psychotherapy research, as there are factors to patient success, such as the individual patient-psychotherapist relationship, which can influence the observed outcomes without directly depending on the novel technique or therapeutic approach under investigation. In contrast, double-blind randomized controlled trials (RCTs) are still considered as the gold standard for medical research in general. However, behavioral interventions, and in particular disorder-specific psychotherapy techniques, differ in their complex nature in many ways from the classic application of double-blind RCTs seen in drug trials.

Recently, we have discussed the issue of possible placebo effects in psychotherapy [4] with respect to: (a) nonspecific factors shared with drug therapy (context factors), (b) nonspecific factors shared among all psychotherapy traditions (common factors), (c) specific placebo-controlled options with different psychotherapy modalities, and finally (d) nonspecific control options for the specific placebo effect in psychotherapy. Additionally, there are also particular differences, for example, double-blinding cannot be applied and the control condition is usually not just an inert placebo pill.

Gold et al. [5] have developed a decision framework for control conditions for RCTs of behavioral interventions in psychiatry. They suggested a list of frequently used control conditions in trials of behavioral interventions (see Table 1).

Table 1.

Frequently used control conditions in behavioral intervention trials (adapted from Gold et al. [5])

As a clear implication, the authors explain that investigators should: (a) describe in detail and (b) justify their choice of the control conditions for a planned trial explicitly in their trial protocol and subsequently in the trial publication. When choosing a control group population, the respective risks should be considered for each trial phase. They stratified their decision framework based on low, intermediate, and high participation risk. The authors also claim that no behavioral treatment should be included in treatment guidelines if its effectiveness is supported only by trials consisting of a waitlist control group or by meta-analytic evidence conducted on such trials. Finally, the authors stress the fact that indirect comparisons of efficacy and effectiveness based on trials of behavioral and pharmacological therapies, tested against different control conditions, are problematic.

As a conclusion, the authors of this framework demand that for the choice of a control condition in a behavioral intervention, the type of intervention, patient population, available resources, and finally the respective trial phase should be clearly considered and reported.

In their methodological recommendation for trials of psychological interventions, Guidi et al. [6] reflect the crucial role of the study design and the choice of the respective control group condition. In line with Gold et al. [5], they state that a waiting list control condition may be only suitable for preliminary testing of an experimental procedure and for pilot studies. In addition, the authors point out that treatment as usual (TAU) control conditions are often not properly manualized, as well as insufficiently monitored and supervised. A potential solution for this aspect might be the optimized TAU condition (TAU-O) where researchers try to include and monitor the best available evidence derived by the actual treatment guidelines for specific disorders [7, 8]. However, for patients with newly developed psychotherapy conditions undertaking such a structured TAU-O control condition could be a particularly strong competitor and may lead to two active psychological treatments with nonsignificant differences in statistical tests. On the other hand, such a study design makes participation in the study ultimately more attractive for participating patients and may help to minimize drop-out rates in the control arm.

Both, Gold et al. [5] and Guidi et al. [6] also line out the possibility to use a nonspecific component control group. Such a control condition includes the same dosage of therapy sessions as in the experimental treatment, but no ingredients regarded as therapeutic (see Table 1). This has been sometimes also referred to as an “attention placebo,” while it should be considered that such conditions might suffer from lower acceptance and high dropout rates [6]. Moreover, Guidi et al. [6] also emphasize that it is important to differentiate so-called clinical management control conditions from nonspecific factor component control (see below for an in-depth example and discussion of such a clinical management control condition).

Based on the recommendation from Guidi et al. [6] dismantling studies (i.e., comparing a full intervention to a control intervention where specific components of the full intervention are not applied) is another important option when designing psychotherapy trials. A psychotherapeutic approach is likely to include multiple treatment components. As a result, dismantling studies may determine which components may yield additional value to the treatment and again the crucial question is, which component condition serves as control group.

A specific approach has to be considered when studying interactions of experimental psychotherapy with pharmacology. There are four models of interventions involved: (1) addition (the effect of two interactions combined equals the sum of their individual effects), (2) potentiation (the effect of two interventions combined is greater than the sum of their individual effects), (3) inhibition (the effect of two interventions combined is less than each individual effect), and (4) reciprocation (the effect of two interventions combined equals the individual effect of the more potent intervention) (see [6]).

In the future, studies with adaptive designs and interventions may play a more prominent role in the times of individualized and precision therapy. Such a targeted and gradual intervention could then focus even more precisely on specific characteristics of the individual patient, which could come from the entire spectrum of bio-psycho-social factors and also include former patients’ response to first-line treatment. Such complex and individualized study approaches could also benefit from the rapidly growing research on artificial intelligence in the near future. However, this will require even broader interdisciplinary expertise, which will also demand the involvement of data scientists [9].

In addition, the distinct differences in healthcare systems between different countries should also be taken into account in the consideration of an adequate control condition in the future. Especially with regard to the question of financing psychotherapy, this can depend on the socioeconomic level of the country, but also on very different traditions. In Germany, for example, there is a long tradition of psychosomatic medicine with a broad range of inpatient and day care services [10]. In addition, Germany has a long tradition of financing outpatient psychotherapy, including psychoanalytic, psychodynamic, as well as cognitive behavior psychotherapy. In such a treatment environment, TAU can be much more intensive than in other healthcare systems, as these services may not be available as TAU control conditions.

Another factor that is usually not taken into account is the interaction of diagnostics and therapy. Particularly in the case of those mental disorders for which a structured monitoring of symptoms is an integral part of therapy, an intensive process and multistep outcome monitoring can also have a certain therapeutic effect. Such aspects of self-monitoring should be discussed at least in the case of significant control group effects [11].

Challenges for Trials Investigating Digitally Enhanced Psychotherapy

In recent years, there has been an increasing challenge for providing digital health [12] and subsequently also in the field of mental healthcare [13]. As a consequence, the challenge for the design of control conditions of psychotherapeutic interventions can be seen in the complex mix of components in the context of digitally enhanced treatment approaches. For example, if active treatment conditions are delivered as a combination of face-to-face psychotherapy with an (intersession) e-health component, such as an online application of mindfulness-based stress reduction (MBSR), it is of crucial importance to monitor control group participants for any use of such (potentially confounding) online applications that are available on the market. However, monitoring such use of confounding applications may be of limited value because it is usually reliant on self-reporting. Study participants may, for example, not recognize any use with potentially confounding applications. Although technically feasible, the alternative of web tracking the use of potentially confounding (online) applications by control participants faces serious ethical and data safety constraints.

When the Control Condition Is the More Effective Intervention

In this context, it becomes particularly interesting when the supposed control condition is significantly more successful than the actual experimental psychotherapeutic intervention.

One specific explanation for such a constellation might be that the experimental treatment entails adverse effects [14]. However, this can also happen under active control conditions, which are particularly close to the mechanisms of action of the disorder and sometimes include additional optimization of the therapy according to current guidelines.

In the field of eating disorders, this has been seen, for example, with the so-called Specialist Supportive Clinical Management (SSCM). SSCM was initially developed as an atheoretical, unspecific active control condition for an RCT on the efficacy of cognitive behavior therapy versus interpersonal psychotherapy in the treatment of anorexia nervosa (AN) [15]. In this trial, it proved more efficacious than the two specific interventions and has since been used in other RCTs on AN treatment and has even been included in national treatment guidelines [16]. It has been speculated that SSCM might work as it incorporates a few “general” components which induce change, i.e., a focus on core pathologies while using a supportive approach and a positive working alliance. Especially in the case of severe disorders like AN, where prognosis is still limited and novel treatment approaches are urgently needed [7], it is actually highly beneficial if treatment options can be broadened by an intervention which unexpectedly targets the disorder.

This particular constellation brings us to the current study by van Passel et al. [17] in this issue of Psychotherapy and Psychosomatics. They present data from an RCT in which they have investigated the efficacy of cognitive remediation therapy (CRT) as an adjuvant therapy prior to TAU for obsessive compulsive disorder (OCD) and AN. The rationale behind this trial entails that patients suffering from both disorders typically show cognitive inflexibility which might reduce the potential benefit from TAU, which targets the core symptoms of the disorder. In a rigorous RCT study design, the group has conducted CRT in a decent sample size of OCD and AN patients, while a central innovative design aspect lies in the implementation of a novel active control condition called SAT which closely resembles CRT without addressing aspects of cognitive inflexibility. Other strengths of the trial include comparably low immediate dropout rates from the adjuvant treatments, suggesting acceptability of the interventions, as well as a 12-month-follow-up assessment. Taking a look at the primary outcome, in the long run, most of the patients who were available for follow-up assessments were improved and many remained unchanged or stable. However, there was no evidence for CRT to enhance the efficacy of TAU. On the contrary, there was even a trend for the active control condition SAT to result in superior outcomes in combination with TAU for OCD patients. From this perspective, the trial by van Passel et al. [17] is at first glance a so-called negative study, as the trialed intervention was not beneficial in terms of the primary outcome. Nevertheless, the trial is promising as the active control condition seems to have more potential than initially expected and should be investigated further, similarly to SSCM.

Are Negative Studies Really Negative?

There is yet another aspect which might have contributed to the control condition being superior in the Van Passel trial (and other trials before), which has also been emphasized recently in The Lancet Psychiatry [18]. It is the choice and especially the assessment of the primary outcome that can be very challenging in the field of mental health where there is often no clear biomarker or “objective” measure available and that disorders might be characterized by different key symptoms [18]. This may lead to circumstances in which a trial turns out “negative” on an outcome which has been chosen more or less arbitrarily from different potential equivalent outcomes. Especially in the field of investigating novel interventions for mental diseases, this might imply that secondary outcomes should gain much more attention. They might also tell a lot on whether an intervention might be beneficial for patients. This has also led to the call for integrating the perspectives of individuals with lived experience into the development of clinical trials [18] as they might help to choose outcomes beyond the mere “RCT logic” which make a difference in the lives of individuals living with mental health issues and their carers.

Conclusion

Together with the Lancet Psychiatry Commission on psychological treatments research in Tomorrow’s Science [19], we are convinced that dedicated effort is needed for conducting state-of-the-art trials to assess psychological therapies and inform policy and practice. Accordingly, the design and conduct of these trials warrants scrutiny and quality improvement. An important factor is the selection of an adequate and suitable control condition. This is especially the case for the complex interplay of active and confounding components within digitally enhanced psychotherapeutic interventions. Clinical researchers should be trained even better and earlier in this growing aspect of clinical research. Meanwhile, models for such systematic training have been tested and proven [20].

Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.

What is the most important factor in the success of psychotherapy?

What is an important factor in determining the effectiveness of counseling?

What are the three main factors that predict whether therapy will be successful?