Enhancing Quality of Care Through Evidence-Based Practice: Training and Supervision Experiences

Mental health problems represent a major public health concern for individuals across the life span. Among adults, psychiatric disorders are one of the most common causes of disability (1). Approximately 21% (52.9 million) of U.S. adults ages 18 years and older experience a mental illness, and roughly 5.6% (14.2 million) experience a seriously debilitating mental illness (2). Among children and adolescents, 10%–20% experience mental disorders worldwide, with approximately 50% of all mental health conditions beginning by age 14 (3). To combat the burden of mental illness, researchers and clinicians have developed empirically supported treatments (ESTs), which are manualized treatments established via randomized controlled trials for specific populations and disorders (4, 5). Although several ESTs for children, adolescents, and adults have been developed and proven to be efficacious via randomized controlled trials (3, 6), effective implementation of ESTs in community clinics and outpatient and inpatient settings, rather than in controlled academic research environments, has been an ongoing challenge (7, 8). Thus, ESTs have become part of a broader concept called evidence-based practice (EBP), or “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences,” that is intended to assist psychotherapy providers in optimizing patient outcomes (5).

Psychotherapy providers who receive training that emphasizes EBP during their graduate education are more likely than those without such training to use ESTs in routine clinical care and to seek additional EBP training throughout their careers (9). However, psychotherapy providers may not have the opportunity to receive adequate EBP training, which limits their understanding and use of ESTs and their ability to incorporate the best available research related to intervention strategies, assessment, clinical problems, and patient populations into their routine clinical practice (5, 10). The amount and quality of EBP training acquired often vary on the basis of when and where therapists complete their graduate training. For example, graduate students across master’s- and doctoral-level mental health programs (e.g., social work, counseling, and clinical psychology) receive highly variable training on EBP (11). Variability in training is understandable given the difference in time needed to complete different graduate degree programs (e.g., a 2-year master’s program vs. a 5- to 7-year doctoral program) (12). Furthermore, one study found that only 38% of master’s-level social work training programs required coursework and clinical supervision on any EST (13). Similarly, psychologists trained before the release of the 1995 American Psychological Association task force reports regarding EBP, and specifically ESTs, are likely to have had highly variable (if any) EBP training during graduate school (14, 15).

Many training programs have increased their emphasis on EBP training (16). However, large inconsistencies remain in both the type of training and the type of supervision therapists receive. The field of implementation science focuses on identifying strategies to efficiently and effectively increase use of EBP through training on ESTs across mental health settings (10, 17). For example, 1- to 2-day training workshops are often ideal, given their brevity and their ability to increase providers’ knowledge of ESTs; however, workshop training alone does not result in proficiency in using ESTs or therapists’ integration of new knowledge and skills into routine patient care (10, 18, 19). Focused consultation and supervision after in-person training have been shown to result in greater treatment adoption (20) and competency among therapists (21), with more consultation and supervision predicting greater therapist treatment fidelity and skill (22). Furthermore, unless ongoing supervision occurs after training, skills learned are often not maintained (22).

Although implementation research has documented the importance of supervision as part of training and implementation of ESTs, supervision best practices (e.g., supervision with structure, supervisor modeling of skills, and ongoing feedback) common in academic settings are understudied and difficult to implement in routine inpatient, outpatient, and community settings (23, 24). Moreover, to ensure consistency and mastery of treatment delivery, evaluation of psychotherapy in the context of ESTs typically emphasizes a high degree of fidelity monitoring and intensive supervision, which are incongruent with the typical service delivery environment in clinical settings (6, 25). In outpatient and acute inpatient settings, several barriers have been identified that contribute to low implementation of and therapist fidelity to ESTs, including therapists’ lack of time and financial compensation to attend training sessions as well as insufficient opportunities for ongoing EBP supervision, training, and support (26–28). Despite the perception that low EST implementation rates across mental health settings are a problem, few studies have documented the number of providers who have previously received or are currently receiving EST training and supervision. Furthermore, understanding the training histories of psychotherapy providers in outpatient and inpatient settings is essential because it assists administrators in identifying who may benefit from additional training on ESTs. Such information is necessary to help guide decision making about the types of training and supervision needed to support effective implementation of ESTs throughout a mental health center.

This study aimed to explore current training experiences related to ESTs among a sample of psychotherapy providers, a majority of whom were licensed social workers or licensed clinical counselors, within a department of psychiatry and behavioral sciences. This work was intended to serve as a needs assessment before implementation of EST training programs. Two ESTs—cognitive-behavioral therapy (CBT) and dialectical behavior therapy (DBT)—were used as exemplars. The results provide insight into the current state of training and supervision regarding ESTs and help inform the types of training and supervision needed to improve the overall quality of care.

Methods

Setting and Population

This study followed a descriptive research design. Psychotherapy providers within a psychiatry and behavioral sciences department at an academic institution, a majority of whom were licensed social workers or licensed clinical counselors, were eligible for inclusion in the study, which began during winter 2019. Participants had to be currently providing mental health therapy services to patients across the life span to be eligible. Participating therapists were recruited from an adult outpatient psychiatry clinic; a children’s mental health clinic; an outpatient psychiatry clinic that serves patients throughout the life span; hospital-based child, adolescent, and adult inpatient services; and hospital-based child and adolescent day hospital services. Participation was voluntary, and no exclusion criteria were applied. Retrospective review of the survey findings was approved by the Johns Hopkins Medical Institution Review Board.

Research Design, Data Management, and Analytic Plan

Therapists were asked a series of categorical and open-ended questions related to their training on CBT, DBT, and other ESTs. Training variables included coursework and supervision experiences during and after graduate school as well as current training and supervision experiences. Information collected included graduate degree, location of services, age of population served, formal training received (CBT, DBT, other EST), type of training received during and after graduate school (i.e., coursework, supervision), EST training during current employment, current EST supervision, and general interest in EST supervision and training.

Statistical analyses were completed by using SPSS, version 25. Summary statistics were calculated by using frequencies and percentages for categorical and count variables and means and standard deviations for continuous and count variables. Therapists were asked to complete items on the basis of their engagement in different aspects of past and current EST training. Given the nature of the survey, frequencies were calculated on the basis of the number of participants who reported receiving certain aspects of training. Missing data, as well as participants who did not complete items because of lack of training, were excluded from analyses.

Results

Electronic surveys were distributed to 108 mental health therapists, and 64% (N=69) completed the survey. Of those who completed the survey, a majority of the providers were master’s-level therapists (N=65, 94%). The remaining therapists included one (1%) with a doctoral degree (Ph.D. or Psy.D.) and three (4%) with an “other” degree or certification. Approximately 39% of therapists focused primarily on treatment of children and adolescents, whereas the remaining focused on treatment of young adults, adults, and older adults or treated people of all ages. Therapists had an average of 11.0 years of work experience after graduate school, with experience ranging from 0.5 to 35.0 years (Table 1).

The proportions of respondents who received CBT-, DBT-, and other EST-focused coursework, supervision, or both through graduate and postgraduate training are outlined in Table 2. Of the therapists who reported receiving some form of CBT training during graduate school (N=67), 85% reported completing coursework in CBT, and 33% received supervision of CBT cases. Of the therapists who reported receiving postgraduate training, approximately 60% completed coursework in CBT, and 31% received supervision of CBT cases. Overall, 49% reported receiving any supervision of CBT cases to date. Regarding current supervision, 15 therapists reported receiving some form of supervision, and four reported receiving formal supervision of CBT cases.

A similar and more striking pattern was found for DBT. Of the therapists who reported receiving some form of DBT training during graduate school (N=42), 52% reported completing coursework in DBT, and 24% received supervision of DBT cases. Moreover, whereas 67% completed postgraduate DBT coursework, 17% received postgraduate supervision of DBT cases. Overall, 24% reported receiving any supervision of DBT cases to date. Nine therapists reported receiving some form of DBT supervision from their current employer, and two reported currently receiving formal supervision of DBT cases.

Two-thirds (N=46) of our sample reported completing some form of coursework related to ESTs other than CBT or DBT. When the type of “other” EST was closely reviewed, a variety of treatment approaches were listed (e.g., interpersonal psychotherapy, acceptance and commitment therapy, psychoanalysis, motivational interviewing, mindfulness-based stress reduction therapy) or not enough information was available to properly characterize the treatment as an EST (e.g., “certificate in evidence-based practice with children and adolescents”).

Overall, a majority of therapists reported having interest in receiving training on ESTs (N=60 of 63, 95%) and interest in implementing and receiving supervision on ESTs (N=56 of 62, 90%).

Discussion

EBP in mental health treatment aims to improve patient outcomes through the integration of science and clinical practice (5). This integration is important because it helps therapists gain a deeper understanding of why a particular treatment works and how they can integrate the best research available with their own clinical expertise to optimize patient outcomes in an efficient and effective manner (5, 29). Since the release of the 1995 American Psychological Association task force reports regarding EBP, the mental health field has focused on improving understanding and implementation of evidence-based treatment components, specifically ESTs, in routine practice to improve patient outcomes (10, 18, 21, 26). Although training on ESTs within mental health settings is an important component of EBP, supervision of EST implementation, in addition to treatment fidelity, are critical for therapists to develop and maintain a strong EBP skill set (22, 30–32). Evaluation of therapists’ training and supervision histories is an essential first step in the improvement of patient outcomes because it helps mental health centers identify the types of training and supervision models needed to support effective center-wide implementation of ESTs and ultimately improve the quality of patient care.

In a sample of predominantly master’s-level therapists, we found that although a majority reported completing some form of CBT coursework during their graduate school training, only 33% reported receiving supervision of CBT cases. After graduate school training, only 31% received supervision of CBT cases. Overall, only 49% reported receiving any supervision of CBT cases to date. A similar and more striking pattern was found for DBT. Of the therapists who reported completing any DBT coursework during or after graduate school training, only 24% reported receiving any DBT supervision throughout their careers. This statistic is particularly salient given that DBT training explicitly emphasizes the importance of ongoing supervision for maintaining treatment integrity, especially when working with high-risk patients with complex problems (33).

Taken together, our findings support previous research highlighting the need for increased supervision of therapists who implement specific ESTs across a variety of mental health settings (34). Furthermore, increased supervision efforts in workforces that predominantly hire psychotherapy providers with 2-year master’s-level training backgrounds (e.g., vs. 5- to ≥7-year doctoral-level training backgrounds) may be especially beneficial, because shorter training programs may limit therapists’ opportunities for structured EST supervision experiences. Therefore, future research is needed to evaluate graduate school training models and their applicability to different training programs (e.g., social work vs. psychology) of varying lengths (e.g., 2-year vs. 5- to ≥7-year programs) to help minimize training gaps in EBP across graduate programs.

Although adapting educational programming will help provide long-term solutions, in the immediate future, providing effective and efficient training will be key to improving the quality of current psychotherapy interventions. However, given the time constraints of practicing therapists, the feasibility of increased EBP education and supervision efforts will remain low without systematic changes to organizational practices. A majority of providers in this study reported being interested in receiving EST training and supervision in the future; however, barriers to receiving training and supervision were noted in our sample via therapists’ self-reports. For example, one provider wrote, “I cannot express the need for formalized supervision in EBP once trained. This is a huge missing component, and it is not affordable to seek outside formal supervision.” Another provider wrote, “Training would be greatly appreciated, but it is not a realistic item at this time” because of financial and time constraints. Our findings highlight that well-documented barriers to training and supervision continue to exist. Thus, evaluation of past and current EST training and supervision experiences of staff, current training needs, and associated training targets (e.g., education, supervision) is encouraged within routine care centers to improve quality of care. Beyond improving quality of care, increasing self-efficacy within therapeutic practice via increased EST training and supervision may also help to alleviate work-related stress that often contributes to provider burnout (35).

Multiple models are used to train psychotherapy providers in clinics and health systems, such as training for the entire staff, “train-the-trainer” approaches, and online training without supervision. Recent studies suggest that the train-the-trainer approach may be a particularly sustainable and cost-effective model of training, because therapists can be continuously trained and supervised over a prolonged period, with qualified trainers and supervisors embedded within a mental health center (10, 36). However, sufficient time must be allocated to activities that are commonly part of supervision in EST research, such as ongoing skill building through practice, observation, and feedback (24, 37). Given the organizational challenges associated with implementing ESTs, future research that explicitly explores the financial costs and cost-benefit analysis of different training and supervision models is warranted to determine whether these models are worth the cost of implementation in terms of treatment fidelity and patient outcomes.

This study was intended as an initial needs assessment to guide future implementation of training and supervision programs in community mental health clinics, outpatient clinics, and acute inpatient settings within an academic medical center. It is worth noting the paucity of doctoral-level clinicians in our sample. The aim of this project was to serve as a needs assessment before implementation of a train-the-trainer program for therapists to be trained in ESTs and, in turn, to train others within their service centers. Therefore, the focus of the survey was on the therapists in the department, a majority of whom were licensed social workers or licensed clinical counselors. Because the sample was predominantly composed of master’s-level therapists, our findings may not generalize to other clinical settings with a more diverse group of providers. However, it is worth noting that far more clinical social workers than psychiatrists and doctoral-level psychologists make up the current mental health workforce; thus, clinical social workers represent the largest group of mental health service providers (38–41). In addition, the amount of exposure to ESTs in the context of EBP among the therapists in our sample may differ from that among therapists in different care systems. Important factors that were not examined in the current study warrant further examination in future research. In particular, knowing the specific types of training models providers have engaged in would aid in assessing the adequacy of providers’ previous training and supervision and their current training needs. In addition, direct observation or self-report tools (with specific questions related to the therapeutic techniques used during a session) to assess therapists’ use of and fidelity to ESTs are strongly recommended because providers’ self-labels of orientation (e.g., CBT) may not be congruent with the actual skills utilized in therapy (42). Observational methods are the current gold standard for determining therapists’ use of evidence-based therapy techniques (43), but feasibility challenges make the use of such methods nearly impossible (35). Therefore, future research is warranted on implementing service models that increase the number of mental health specialists (i.e., clinical psychologists) serving as consultants and supervisors within mental health centers in order to help provide cost-effective internal support for EBP supervision.

It is important to note that although this study focused on the implementation of training and supervision on ESTs in naturalistic settings, other forms of therapy may also be worth exploring. Although initial lists of ESTs included mostly CBTs, over the past 20 years, the number of efficacy trials has increased for psychodynamic therapy, family-based therapy, and interpersonal therapy, among other treatments (44–46). In addition, research in recent years has highlighted the growing awareness of and problems with the replication crisis (47), which has led to a greater emphasis on study quality, better study designs, and improved reporting of findings. Although the evidence base for therapies is still evolving, it is critical that psychotherapy providers utilize the best available research to help inform treatment efforts (48). Thus, high-quality training and supervision related to ESTs in the broader context of EBP can teach psychotherapy providers how to be informed consumers of intervention trial research and administrators of evidence-based assessment tools in order to enhance clinical practice (49).

Conclusions

Taken together, our findings highlight that although research from the past decade has supported the need for improvements in EST training—especially supervision—the problem of limited exposure to training and to supervision related to ESTs still exists. Our findings have implications for how mental health centers can evaluate staff members’ EST training and supervision experiences, training needs, and associated training targets to improve quality of routine care. In addition, this study highlights potential options for how mental health centers can work toward improving quality of care through increased training efforts. Finally, quality improvement efforts to understand how organizations and supervisors currently use supervision are important. Modification to supervisory practices within mental health settings may improve the quality of care provided by therapists, especially in centers where staff have previous EST training but minimal supervision experience.