Using Learning Objectives for Psychotherapy Supervision
Abstract
Although learning objectives, often in the form of competencies, are now standard for training mental health professionals, they are not generally used to guide psychotherapy supervision. Nevertheless, when learning objectives are not used to guide supervision, supervisors and supervisees often remain uncertain about the goals of supervision, how those goals should be attained, and how they should be assessed. In this paper we review the literature on learning objectives for psychotherapy training and supervision, outline reasons for using learning objectives in psychotherapy supervision, and suggest ways to use learning objectives in training.
Introduction – The Basics about Learning Objectives
What are Learning Objectives?
In 1949, Ralph Tyler, then the chair of the department of education at the University of Chicago, first described the need for learning objectives within education in his landmark book, Basic Principles of Curriculum and Instruction. A former high school science teacher, Tyler questioned standard methods of testing that relied on memorization, advocating instead assessments based on objectives designed to guide every aspect of the educational process. He wrote:
Many educational programs do not have clearly defined purposes. … No doubt some excellent educational work is being done by artistic teachers who do not have a clear conception of goals but do have an intuitive sense of what is good teaching, what materials are significant, what topics are worth dealing with and how to present material and develop topics effectively with students. Nevertheless, if an educational program is to be planned and if efforts for continued improvement are to be made, it is very necessary to have some conception of the goals that are being aimed at. These educational objectives become the criteria by which materials are selected, content is outlined, instructional procedures are developed and tests and examinations are prepared (Tyler, 1949, p. 3).
Tyler’s “educational objectives” are now generally referred to as learning objectives, and are recognized to include three basic types—knowledge, skills, and attitudes (Bloom 1956). They are now used widely in educational enterprises from kindergartens to graduate schools.
Constructing Measureable Objectives
To be as useful as possible, learning objectives must also be measureable. For example, saying, “trainees will be able to use a cognitive behavioral formulation to guide treatment,” is not as useful as saying, “trainees will be able to use a cognitive behavioral formulation to guide treatment, as evidenced by their ability to write a short paragraph describing the way that the formulation guided the chosen interventions in a video-taped session.” Only the second gives the educator a way to measure attainment of the objective, and, thus, only the second gives the educator the ability to use this information to accurately assess the trainee’s skills, alter teaching methods, and change curricula.
In psychotherapy training, knowledge, skills, and attitude objectives are all required. Here are examples of each type of objective in psychotherapy training:
Knowledge
Trainees will be familiar with the three major transferences (mirror, idealized, twinship) discussed in Kohut’s developmental theories, as evidenced by their ability to describe them to his/her supervisor.
Trainees will know the meaning of the medical model of depression as it is used in Interpersonal psychotherapy (IPT), as evidenced by their ability to write a few sentences about this during class.
Skills
Trainees will be able to interpret the patient’s resistance, as evidenced by their ability to do this during a videotaped session.
Trainees will be able to conduct a chain analysis during a dialectical behavioral therapy (DBT) session, as evidenced by their ability to do this while being observed through a one-way mirror.
Attitude
Trainees will feel more comfortable discussing medication with a patient during a psychotherapy session, as evidenced by their ability to discuss this with a supervisor.
Trainees will be more willing to recommend psychotherapy to patients after a thorough evaluation, as evidenced by their ability to write about experiences of making recommendations in an in-class writing assignment that is discussed by the group.
The clearer educators are about 1) whether they are looking for their trainees to attain knowledge, skills, or new attitudes and 2) how they will assess attainment of those objectives, the more useful those objectives will be. Note that measurement can take many forms—class discussion (as long as everyone is involved), short writing assignments, individual discussions with supervisors—as long as they are specified, all of these methods can be used.
Using Learning Objectives to Plan Teaching Activities
Once educators know their learning objectives, they can use them to plan teaching activities. This is called backward design (Wiggins & McTighe, 1998), since it involves first understanding the endpoint goals for learners and then planning teaching activities that have the best chance of achieving them. For example, let’s consider this learning objective from the last section:
Trainees will be able to interpret the patient’s resistance, as evidenced by their ability to do this during a videotaped session
If this is the goal, we have to plan a teaching activity that will help the learner achieve this skill. If we think that this could be well taught in supervision, we design a teaching activity in which the supervisor and supervisee watch a tape of a session together, think together about the resistances that the patient is using, and role-play ways that the supervisee could interpret them. Note that we are doing more here than saying that supervision is the teaching activity—that would be like taking an objective and saying that the teaching activity is “being in a classroom with a teacher.” This is a key point—if the objective is to be achieved via supervision, the teaching activity that will be used in supervision must be specifically outlined and taught to supervisors—otherwise the supervisory experience is unlikely to be guided by the objectives. If the teaching activity is linked to the method of measurement, all the better.
Learning Objectives and Competencies
In graduate medical education and other mental health training programs, learning objectives are often conceptualized as competencies. Competency-based education has also been referred to as outcome-based education, since the educational curriculum is guided by the outcomes that learners are expected to gain (Frank et al., 2010). Competency-based curricula are designed by beginning with desired outcomes (e.g. knowledge, skills and attitudes expected at the end of the training,) and then developing instructional methods and assessment tools aimed at achieving those outcomes. Competency-based training has been the standard in education since the 1980s when the United States Department of Education mandated that all accredited educational endeavors include outcome measures (Humphrey et al., 2013).
From Competencies to Milestones
Competencies are end-point measures—they specify what trainees should attain by the end of training. As such, they do not address the developmental trajectory that learners should take to reach those goals. The concept of competency-based education is evolving in order to better capture the “multi-dimensional, dynamic, developmental and contextual nature” of learning (Frank et al., 2010). This modern-day conceptualization of competency is greatly influenced by the Dreyfus model of skill acquisition, which outlines the progression of competency through five levels: novice, advanced beginner, competent, proficient, and expert (Humphrey et al., 2013).
In recognition of this developmental learning process, the Accreditation Council for Graduate Medical Education (ACGME), which oversees residency training for psychiatrists in the United States, has started a multiyear initiative to delineate educational milestones for each medical specialty. The goal of this “milestone initiative” is to establish a core set of specific learning objectives along a developmental trajectory (Nasca 2012). The milestones include learning objectives in psychotherapy for each progressive stage of medical training, from the first year resident to the graduate ready for independent practice. Developmental learning objectives or competencies like the milestones help educators to plan phase-appropriate teaching activities and assessments.
Learning Objectives in Psychotherapy Supervision
Despite the centrality of learning objectives, or competencies, in mental health training programs, they are not generally used to guide one of the field’s most ubiquitous teaching modalities—psychotherapy supervision.
Although many educators agree that good supervision is based on learning objectives, this is not always practiced (Kilminster et al., 2007). In fact, several papers have argued that without objectives, the goals of supervision remain opaque to both trainees and supervisors (Azuonye 1997; Cabaniss & Arbuckle, 2011; Kernberg 1986; Pegeron 1996; Rojas et al., 2010).
Generally, the objectives/competencies that exist for psychotherapy are not specifically designed to guide technique and assessment in supervision. For example, in psychiatric training, the ACGME milestones establish specific developmental learning objectives for psychotherapy that could be accomplished with many different types of learning activities, such as didactic courses, case conferences, group or individual supervision. Other illustrations of this include Goldberg’s objectives (1998) for psychodynamic psychotherapy and Sburlati’s objectives for Interpersonal Psychotherapy (IPT) training—both are comprehensive for training in those psychotherapies but do not specifically address the role of supervision in achieving the objectives (Sburlati et al., 2012).
Over the years, however, many mental health educators have striven to delineate learning objectives that are specific for supervision. For example, many psychoanalytic educators have independently defined learning objectives for psychoanalytic supervision (Moga & Cabaniss in press; Schlesinger 1995; Weiss & Fleming 1975). Learning objectives have also been conceptualized for supervision in family therapy and Cognitive Behavioral Therapy (CBT; Carrigan & Bambrick, 1977; Rubenstein 1982; Sudak 2003). A 2004 survey of the Canadian psychiatric residency programs found that learning objectives and competencies for supervision of psychotherapy are used at several universities in Canada including University of Toronto and McMaster University (Ravitz & Silver 2004). Other groups of educators have specifically developed psychotherapy competencies for the supervision of trainees, but without specifying how attainment of those objectives should be measured (Boswell et al., 2010; Farber et al., 2010).
Adherence Scales and Learning Objectives
In evidence-based psychotherapies, adherence scales, developed to measure the adherence of already experienced psychotherapists to the manualized treatment being delivered and studied, are often used to teach new trainees, either by having supervisors or the trainees themselves rate adherence using audio or video tapes of the trainees’ psychotherapy sessions. Thus, in practice, adherence scales often substitute for learning objectives, serving as a list of techniques that trainees should be able to perform by the end of their training. For example, CBT training programs have used the Cognitive Therapy Rating Scale (CTRS; Young and Beck, 1980) in this way (Martinez & Horne, 2007, Sudak 2003). At Columbia, the Dialectical Behavior Therapy (DBT) Basic Skill Rating Scale, adapted from the DBT Adherence Rating Scale, is given to therapists at the beginning of their DBT training and then used intermittently to evaluate their videotaped sessions (Brodsky, personal communication). In a recent effort of the department of Veterans Affairs to nationally train therapists in Acceptance and Commitment therapy (ACT) for depression, the ACT Core Competency Rating Form was used at three intervals during the six-month training period to assess therapist learning and implementation of ACT in counseling sessions (Walser 2013).
However, efforts to study psychotherapy training using manualized treatments have demonstrated that increased adherence does not correlate with increased therapist competence (Beutler, 1991; Henry et al., 1993), and simply being able to perform the techniques outlined in adherence scales does not demonstrate competence in many of the more conceptual skills necessary to be a good psychotherapist (Vakoch & Strupp, 2000). For example, trainees may be able to perform certain psychotherapeutic techniques, but may perform them unnecessarily or at inappropriate moments in the session (Vakoch & Strupp, 2000). This suggests that even when adherence scales are available, educational objectives for skills that are observable when watching someone conduct psychotherapy, as well as skills that involve conceptualization and formulation (not necessarily observable on a tape or when watching a session), are still necessary for psychotherapy training (Binder 1993; Dobson & Shaw, 1993).
Evidence for the Importance of Learning Objectives in Supervision
There is ample evidence that using learning objectives to guide psychotherapy supervision enhances the supervisee’s experience. Several studies (Cabaniss et al., 2003; Rojas et al., 2010) and papers (Pegeron 1996; Kernberg 1986) suggest that without clear objectives, supervisees are uncertain about the goals of the process and are anxious about evaluation. One study found that when counseling students and their supervisors were asked to rate what supervisors’ behaviors were most important for good supervision, their respective opinions did not match: supervisors thought that good supervision involved feedback about the supervisee’s counseling abilities; supervisees rated supervision as better when their supervisors taught them counseling skills and encouraged trainees to try out the skill as way of incorporating them into their own style and repertoire (Worthington & Roehlke, 1979).
The use of learning objectives in supervision is also supported by cognitive models of learning psychotherapy that describe the importance of both declarative (knowledge such as facts and concepts usually taught in the classroom) and procedural learning (knowledge such as various psychotherapy skills, timing of interventions etc.). Several authors have suggested that the gap that often occurs between the two types of knowledge may be due to the lack of coordination of classroom teaching and supervision, leading to “inert knowledge” that is not accessed in the room with the patient (Binder 1999; Vakoch & Strupp, 2000). These authors suggest that knowledge is easier to modify once made explicit, thus making implicit learning of skills explicit by specifying learning objectives facilitates behavior change in trainees (Vakoch & Strupp, 2000).
Learning Objectives for Supervision—Advantages beyond Individual Learners
Psychotherapy supervision is an educational endeavor—thus it should be guided by clear, measureable learning objectives. But learning objectives have advantages for psychotherapy education that transcend the individual learner. Here are the advantages that we find central.
1. Linking Didactics and Supervision
When students learn one thing in the classroom and another in supervision, the result often is confusion (Rojas, et al., 2010). Previously, we argued that psychotherapy, like other skills, is best learned via a coordinated “course and lab” approach that links didactics and supervision, much as a chemistry lecture and the corresponding lab are carefully linked. Students use the concepts learned in the classroom with their patients, and this clinical work is discussed in supervision. In order for these concepts to be consolidated optimally, the basic teaching message used in both learning venues must be coordinated. One of the best ways to do this is to have psychotherapy learning objectives that guide both seminar instructors and individual supervisors so that they present a uniform approach to the psychotherapy, not only in technique, but also in priorities and expectations at different levels of learning. For example, if a program director determines that the priority for first-year students is that they be able to evaluate patients and forge a therapeutic alliance, this can be communicated to the classroom teachers and supervisors, letting supervisors know that proficiency in using multiple CBT manuals is not what he or she should be stressing in the individual supervision during that year.
Although the learning objectives may be used for both classroom learning and supervision, the teaching activities that will be used in these two venues will be different, as will the method of measurement. This needs to be clearly specified to the teachers in the two educational modalities (see below).
2. Standardized Supervision
Each psychotherapy supervisor brings his/her personality and experience to the supervisory hours. While this is enriching for students in some ways, it can also make the program’s overall supervisory experience uneven. One supervisor might emphasize one type of technique, while another does something very different, leading their supervisees to have a very different training experience. Offering supervisors common learning objectives helps to even out these differences, making psychotherapy supervision less “hit or miss.” Supervisors, like classroom teachers, know what they should “cover,” allowing them to look for “teachable moments” with which to do this. For example, specifying objectives both for technique and for formulation tells supervisors that they need to attend to both sets of psychotherapy skills. If a psychodynamic psychotherapy supervisor is supervising a case in which the transference is not explicitly discussed by the patient, he or she knows to discuss “what if” questions that allow for broader learning.
3. Guided Evaluation
Too often, supervisors have difficulty knowing how to evaluate their supervisees. Trying to maintain a supervisory alliance can impede a supervisor’s ability to comment objectively about the supervisee’s strengths and difficulties. Without guidelines for evaluation, supervisors often fall back on discussing how well the patient did, rather than how well the supervisee did. Statements such as, “Jane was a pleasure to supervise,” do little to help the supervisee or training director know what Jane actually did or did not learn. Offering supervisors clear learning objectives for supervision can remedy this problem. An objective that states, “the supervisee will be able to conduct a behavior chain analysis with a patient in a videotaped session that is reviewed with the supervisor,” tells the supervisor that he or she should specifically comment on the supervisee’s ability to use this technique. Learning objectives can be distributed not only to the supervisors, but also to the students, making the evaluation process transparent and decreasing the “surprise” element in evaluation.
4. Facilitated Research and Program Analysis
Having specified learning objectives for psychotherapy supervision allows individual programs or groups of programs to study supervisory methods and guide improvement in supervisory techniques and curricula. Programs may want to find out whether students are learning a particular technique, or whether they grasp a core psychotherapy concept. If these are specified as objectives of the supervision, they can be tracked and studied. Without clear objectives, desired outcomes are unclear and empirical research on this process is not possible.
5. Connection with the Field
We already know that having common learning objectives for psychotherapy allows educators from different programs to come together at common meetings to learn best practices about teaching—the same is true for supervision. If a group of programs elects to use common objectives, their supervisors can come together to learn from each other about everything from how to teach technique to how to conduct assessments.
Creating and Using Learning Objectives for Psychotherapy Supervision—Strategies for Success
Some training programs, such as psychiatry residencies, have common, extant learning objectives for psychotherapy training that are determined by outside accrediting bodies. Other training programs may have their own objectives, while still others may have no learning objectives in place. In our experience, groups of educators are more likely to use objectives if they conceptualize them, taking into consideration their educational priorities, theoretical proclivities, and faculty strengths/limitations (Moga & Cabaniss, in press). Thus, rather than offering our list of learning objectives for psychotherapy supervision, we offer the following basic guidelines to help training programs to construct their own learning objectives for psychotherapy supervision that are likely to be used to guide supervision and to enhance the learning experience of their trainees.
1. Begin with a Set of Overarching Learning Objectives for Psychotherapy Training
These may be existing objectives determined by an outside accrediting body. If you are constructing your own, we recommend you construct objectives that
• Are Developmental and Concise;
Supervisory objectives need to be phase appropriate and clearly indicate what knowledge, skills, and attitudes supervisees should have at each stage of training. (See Cabaniss & Arbuckle 2011, and Feinstein & Yager 2013 for examples.) In addition, we strongly believe that supervisory objectives will be used regularly by learners and educators only if they are concise—pages and pages of objectives may seem complete to administrators, but are daunting and alienating to the supervisory dyad. Short, practical objectives that are accessible and relevant to the day-to-day work of supervision serve to guide every supervisory session.
• Include Common Factors as Well as Specific Techniques;
There is strong evidence that the common factors of psychotherapy, such as alliance, collaborative goal setting, and good adherence to boundaries, are critical to outcome (DeFife & Hilsenroth, 2011; Horvath, 2001; Luborsky 1975). Thus, the teaching of common factors needs to be explicit in the objectives for psychotherapy training regardless of therapy type. Program directors may want to prioritize teaching common factors in supervision for junior trainees (see Feinstein & Yager 2013 for examples). Teaching common factors helps students to conceptualize the many commonalities among the psychotherapies, and how to oscillate between the (so called) “stem” techniques and those more specific to the individual psychotherapy (Plakun et al., 2009).
• Include Technique and Formulation;
Adherence scales that rate trainee performance of psychotherapeutic skills only are inadequate for assessing psychotherapy competency. In addition to making interventions, trainees need to be taught ways of thinking about things such as formulation, therapeutic strategy, and the use of transference and countertransference. Thus, these must be clearly included in the supervisory objectives. For example, supervisors could be instructed that, periodically in supervision, after watching a video clip of a supervisee conducting psychodynamic psychotherapy, they should systematically ask the supervisee to describe: 1) what he/she heard in the session; 2) how he/she reflected on the material therein to make choices about therapeutic strategy; and 3) how he/she would label his/her interventions. This ensures that both technical and conceptual skills are assessed. For CBT, Sudak et al recommend using the Cognitive Formulation Rating Scale (CFRS), which assesses the clinician’s ability to conceptualize the case and to plan treatment and the CTRS, which is an adherence scale (Sudak, Beck, & Wright 2003).
• Are Measureable, and Include Clear, Easy-To-Use Measurement Methods.
Programs may use extant measurement tools, such as the CTRS and CFRS, to measure attainment of supervisory goals provided a) these tools match the program’s supervisory objectives; b) supervisors and supervisees know how to use the tools; and c) both technique and formulation objectives are included. Where no tools are available, programs should create measurement tools that are easily used and should teach supervisors and supervisees how to use them. These assessment tools may come in many forms, including guided discussion and written exercises.
2. Choose Objectives You Think are Best Suited for Being Achieved and Assessed in Supervision
These objectives relate to the ability to use psychotherapeutic techniques, to formulate a case, and to use formulation in guiding intervention choice. For example, objectives relating to learning theory may be more suitable for classroom learning. Objectives that relate to common factors, such as how to forge a therapeutic alliance and to set the frame, are very well suited to supervision.
3. Create Teaching Activities and Assessment Methods Based on the Objectives for Supervision and Designed for Teaching and Learning in Supervision
Saying that an objective should be achieved in supervision is not an adequately specific way to delineate the desired teaching activity. There are myriad teaching activities that can be used in supervision, these include:
| • | guided discussion—supervisor and supervisee watch a video segment of a psychotherapy session conducted by the supervisee and the supervisor asks specific questions about the session, such as what the supervisee heard, how he/she reflected on what was heard, and how he/she used this to guide the chosen intervention; | ||||
| • | role play—supervisor and supervisee role play scenarios to help the supervisee choose and articulate interventions; | ||||
| • | joint formulation—supervisor and supervisee step out of the action of the session to consider why the patient responded the way he/she did. | ||||
Any of these could be directly turned into a measurement tool, for example,
Trainees will understand the way that their formulations guide their therapeutic strategy, as evidenced by their ability to discuss their thought process after making interventions during role-play in supervision.
Using language that is specific to supervision for teaching activities and methods of assessment will increase the utility of the learning objectives for supervision and the likelihood that supervisors will use them. Note that objectives may have teaching activities suitable for the classroom and for supervision if they can be approached using both teaching modalities.
4. Make Learning Objectives an Explicit Part of the Supervisory Experience
Finally, program directors must explicitly tell supervisors and supervisees the objectives of the supervision, reinforce that these objectives must be used, recommend teaching activities to achieve the objectives, and explain how achievement of objectives are measured. For example, if an objective of the supervision is that the supervisee learns to make appropriately timed interpretations, the supervisor needs to be on the lookout for a good time to teach this ability. We previously discussed how this helps alleviate the “hit-or-miss” aspect of supervision (Cabaniss & Arbuckle, 2011). In other words, the supervisor cannot simply say, “it didn’t come up in our supervision.”
We recommend using faculty development meetings to familiarize staff with the learning objectives for supervision, to introduce them to the types of teaching activities that best achieve those objectives, and to teach the program’s preferred methods of assessment. Role-play and small-group work using vignettes can turn faculty meetings into active learning sessions for supervisors. And we strongly suggest distributing the objectives in writing to supervisors and supervisees during meetings in which the objectives and their use are discussed.
Concluding Remarks
Psychotherapy supervision is an educational activity, and as such it needs to be guided by learning objectives. Using these objectives, educators can design teaching activities and measurement tools specific to supervision. Use of objectives in psychotherapy supervision enhances the supervisee’s experience of supervision; helps supervisors direct learning in supervision; assists supervisors, supervisees, and program directors to know whether trainees are attaining goals; and aids in the alteration of supervisory techniques in a program. Rather than specify supervisory objectives, we offer these guidelines with the hope that they help programs to create sets of objectives for psychotherapy supervision that match the program’s objectives and are realistic for both supervisors and trainees in order to maximize use of this important modality in psychotherapy education.
(1997). Educational supervision sessions between consultants and trainees. Psychiatric Bulletin, 21, 154–155.Crossref, Google Scholar
(1991). Have all won and must all have prizes? Revisiting Luborsky et al.’s verdict. Journal of Consulting and Clinical Psychology, 59, 226–232.Crossref, Medline, Google Scholar
(1993). Observations on the Training of Therapists in Time-Limited Dynamic Psychotherapy. Psychotherapy, 30(4), 592–598.Crossref, Google Scholar
(1999). Issues in teaching and learning time-limited psychodynamic psychotherapy. Clinical Psychology Review, 19(6), 705–719.Crossref, Medline, Google Scholar
(1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.Google Scholar
(2010). Competency in Integrative Psychotherapy: Perspectives on Training and Supervision. Psychotherapy Theory, Research, Practice, Training, 47, 3–11.Crossref, Google Scholar
(2003). Candidate progression in analytic institutes: A multi-center study. International Journal of Psychoanalysis, 84, 77–94.Crossref, Medline, Google Scholar
(2008). Becoming a school: Developing learning objectives for psychoanalytic education. Psychoanalytic Inquiry, 28, 262–277.Crossref, Google Scholar
(2011). Course and lab: A new model for supervision. Academic Psychiatry, 35, 220–225.Crossref, Medline, Google Scholar
(1977). Introducing Novice Therapists to “Go-Between” Techniques of Family Therapy. Family Process, 16, 237–246.Crossref, Medline, Google Scholar
(1973). Training Family therapists by setting learning objectives. American Journal of Orthopsychiatry, 43, 439–446.Crossref, Medline, Google Scholar
(2011). Starting off on the right foot: common factors elements in early psychotherapeutic process. Journal of Psychotherapy Integration, 21, 172–191.Crossref, Google Scholar
(1993). Training of cognitive therapists: What have we learned from treatment manuals? Psychotherapy, 30, 573–577.Crossref, Google Scholar
(2010). Humanistic-Existential psychotherapy competencies and the supervisory process. Psychotherapy Theory, Research, Practice, Training, 47, 28–34.Crossref, Google Scholar
(2013). Advanced Psychotherapy Training: Psychotherapy Scholar’s Track, and the Apprenticeship Model. Academic Psychiatry, 37(4), 248–253.Crossref, Medline, Google Scholar
(2010). Competency-based medical education: theory to practice. Medical Teacher, 32(8), 638–45.Crossref, Medline, Google Scholar
(1998). Structuring Training Goals for Psychodynamic Psychotherapy. The Journal of Psychotherapy Practice and Research, 7, 10–22.Google Scholar
(1993). Effects of training in time-limited dynamic psychotherapy: Changes in therapist behavior. Journal Consulting and Clinical Psychology, 61, 434–440.Crossref, Medline, Google Scholar
2001. The Alliance. Psychotherapy, 38(4), 365–372.Crossref, Google Scholar
(2013). Assessing competencies during education in psychiatry. International Review of Psychiatry, 25, 291–300.Crossref, Medline, Google Scholar
(1986). Institutional Problems of Psychoanalytic Education. Journal of the American Psychoanalytic Assosication, 34, 799–834.Crossref, Medline, Google Scholar
(2007). AMEE Guide #27 Effective educational and clinical supervision. Medical Teacher, 29: 2–19.Crossref, Medline, Google Scholar
(1975). Comparative Studies of Psychotherapies: Is it True That “Everyone Has Won and All Must Have Prizes”? Archives of General Psychiatry 32, 995–2008.Crossref, Medline, Google Scholar
(2007). Setting up and evaluating a cognitive-behavioural therapy training programme for psychiatric trainees. The Psychiatrist, 31, 431–434.Google Scholar
(2014). Learning objectives for supervision: benefits for candidates and beyond. Psychoanalytic Quarterly in press.Google Scholar
(2012). The next GME accreditation system–rationale and benefits. New England Journal of Medicine, 366(11), 1051–1056.Crossref, Medline, Google Scholar
(1996) Supervision as an analytic experience. Psychoanal. Quarterly, 62, 588–614.Google Scholar
(2009). The Y-Model: An integrated, evidence-based approach to teaching psychotherapy competencies. Journal of Psychiatric Practice, 15(1), 5–11.Crossref, Medline, Google Scholar
(2004). Advances in Psychotherapy Education. Canadian Journal of Psychiatry, 49(4), 230–237.Crossref, Medline, Google Scholar
(2010). Don’t leave teaching to chance: learning objectives for psychodynamic psychotherapy supervision. Academic Psychiatry, 34, 46–49.Crossref, Medline, Google Scholar
(1982). Learning objectives in family therapy training. The Canadian Journal of Psychiatry, 27(7), 556–558.Crossref, Medline, Google Scholar
(2012). A model of therapist competencies for the empirically supported interpersonal psychotherapy for adolescent depression. Clinical Child and Family Psychology Review, 15, 93–112.Crossref, Medline, Google Scholar
(1995). Supervision for fun and profit: Or how to tell if the fun is profitable. Psychoanalytic Inquiry, 15, 190–210.Crossref, Google Scholar
(2003). Cognitive behavioral therapy: A blueprint for attaining and assessing psychiatry resident competency. Academic Psychiatry, 27, 154–159.Crossref, Medline, Google Scholar
(1949). Basic Principles of Curriculum and Instruction University of Chicago Press: Chicago, IL.Google Scholar
(2000). Journal of Clinical Psychology The Evolution of Psychotherapy Training: Reflections on Manual-Based Learning and Future Alternatives. Journal of Clinical Psychology, 56(3), 309–318.Crossref, Medline, Google Scholar
(2013). Training in and Implementation of Commitment and Acceptance Therapy for Depression in the Veterans Health Administration: Therapist and Patient Outcomes. Behaviour Research and Therapy, 51, 555–563.Crossref, Medline, Google Scholar
(1975) Evaluation of Progress in Supervision. Psychoanalytic Quarterly 44, 191–205.Crossref, Medline, Google Scholar
(1998). Understanding by design. Association for Supervision and Curriculum Design: Alexandria VA.Google Scholar
(1979). Effective supervision as perceived by beginning counselors-in-training. Journal of Counseling Psychology, 26, 64–73.Crossref, Google Scholar
(1980). The Cognitive Therapy Scale Rating Manual. University of Philadelphia: Author. http://www.academyofct.org/upload/documents/CTRS_Manual.pdfGoogle Scholar
