Using the Achievement of Therapeutic Objectives Scale to Operationalize “Unlocking” of the Unconscious

Intensive short-term dynamic therapy (ISTDP) (1–3) is an evidence-based psychotherapy founded on psychodynamic principles. Current literature indicates that ISTDP is effective in treating various psychiatric disorders, including depression, anxiety, and personality disorders (4–7). Current process-oriented ISTDP research has focused on using the clinical judgment of expert raters to identify the accurate and timely use of interventions, in accordance with Davanloo’s central dynamic sequence (1). To further elucidate the processes of ISTDP, this pilot study examined the use of a profile developed from a psychotherapy process-oriented scale to accurately identify episodes of “unlocking” of the unconscious, a central intervention in ISTDP. This study aimed to explore an alternative method for identifying unlocking.

ISTDP operates under the assumption that psychological symptoms are associated with resistance to unwanted emotions (2, 8). Patients may have childhood experiences in which certain emotions are seen as unacceptable. According to ISTDP theory, such unwanted emotions are pushed out of conscious awareness. In adulthood, patients experience anxiety when previously repressed unconscious emotions are triggered. Patients engage in defensive processes to avoid experiencing unwanted feelings. ISTDP helps patients overcome their internal resistance, fully experience their emotions, and develop greater self-insight.

Unlocking of the unconscious is considered a central driver of change in ISTDP (1–3, 8). Unlocking involves bringing unconscious feelings and experiences and the fantasies of childhood into conscious awareness. These feelings were previously unconscious because of the activation of defenses and anxiety. When anxiety and defenses are addressed, complex feelings rise to conscious awareness. The patient and therapist process newly conscious information and its links to current psychological problems by working collaboratively to overcome defenses and resistance. Depending on the extent to which a patient’s resistance is overcome, unlocking can be observed at different levels. Most commonly, unconscious feelings are first observed as complex emotions toward the therapist. Generally, these feelings comprise a combination of anger at the therapist’s relentless focus on the patient’s anxiety and defenses and gratitude toward the therapist for making the effort to help. The therapy explores emotions in order to develop insight regarding past attachment experiences. ISTDP posits that the experience of previously unconscious emotions and the development of insight regarding previously unconscious experiences result in symptom improvement (3). Insight involves the linking of cognitive elements such as fantasy and memories of early childhood with the felt experience of complex emotions.

As with all psychotherapies, with ISTDP, it is difficult to identify causal relationships among hypothesized mechanisms of change, the interventions that are central to engender change, and symptom improvement (9, 10). A small body of literature has examined the association between episodes of unlocking and therapeutic change (4, 8, 11). These studies provide initial indications that unlocking is associated with positive therapeutic outcomes. However, a significant and persistent methodological issue is the lack of independent raters who are reliably trained to judge the unlocking process. In addition, current approaches to identifying episodes of unlocking could be improved through the use of defined observational measures.

The Achievement of Therapeutic Objectives Scale (ATOS) was developed by researchers seeking to evaluate the therapeutic effects of short-term dynamic therapy (STDP) (11). STDP is an umbrella term encompassing multiple short-term dynamic approaches, including ISTDP (11–13). The ATOS was initially developed in reference to affect phobia therapy (13, 14), another STDP approach. However, the measure was intended to be transtheoretical and widely applicable across psychotherapeutic approaches (13).

The ATOS has seven subscales that relate to different aspects of the therapeutic process within a session. In the current study, we hypothesized that four of these subscales—exposure, motivation, insight, and inhibition—would be useful in the identification of unlocking events. These subscales were chosen because of their relevance to the theoretical components of unlocking episodes. The exposure subscale refers to the intensity of affective arousal. The insight subscale refers to the level of awareness regarding maladaptive patterns. The motivation subscale measures willingness to give up unhelpful maladaptive defenses. The inhibition subscale refers to inhibitory emotions, that is, the level of the patient’s (pathological) guilt, anxiety, shame, or pain. Each of these subscales relates to the components of an unlocking episode—the experience of intense emotions, a decrease in defensiveness and inhibitory emotion, and the development of new insights.

Several ISTDP studies have used the ATOS to explore the therapeutic process. Town and colleagues (15) examined process-outcome associations across 20 sessions of ISTDP. Results indicated that higher scores on the exposure subscale of the ATOS were associated with decreased distress within the following session. In another study, Town and colleagues (16) used the ATOS to examine the relationship between therapist interventions and patient affect in STDP. Therapist interventions were found to account for significant variance in the exposure subscale. Berggraf and colleagues (17) found that the ATOS was sensitive to differences between patients and, among individual patients, between individual subscales. These findings suggest that the ATOS may be a useful measure of ISTDP processes.

Methods

Design

This pilot study aimed to explore the use of an observational measure, the ATOS, to identify episodes of unlocking in ISTDP. This exploration builds on previous studies that have used expert clinician ratings of unlocking. The ATOS includes an assessment of the dominant patient affect in each identified 10-minute segment. A profile of subscale cutoff scores was developed to define the presence of an unlocking event. This study also examined the interrater reliability of raters who used clinical judgment to identify unlocking episodes.

Expert raters and the treating therapist used their clinical judgment to identify instances of unlocking across video-recorded segments of therapy sessions. Therapy segments were then rated by using the ATOS to determine whether the measure was accurate in identifying episodes of unlocking. In addition, ATOS subscale scores were compared between sessions where unlocking occurred, sessions with no unlocking, and sessions with specific unlocking segments. As part of the implementation of the ATOS, raters also identified the dominant affect present in each therapy segment.

Hypotheses

We developed three hypotheses. First, we hypothesized that there would be high interrater reliability between expert raters in their identification of unlocking episodes. Second, application of a specific profile of ATOS cutoff scores would accurately identify the unlocking episodes recognized by the expert raters. Third, unlocking episodes, as identified by the ATOS, would include complex feelings of rage and guilt about the rage. The ATOS raters’ identification of both anger and guilt about the anger due to feelings of attachment in a 10-minute segment would be classified as “complex feelings.”

Participants

The study examined up to the first 10 sessions of participants’ therapy. The study received institutional ethics approval from the Queensland University of Technology. Participants were recruited from among clients referred to the treating therapist’s private practice psychology clinic. Individuals were eligible for the study if they were ages 18–70 and had the capacity to provide written informed consent. Exclusion criteria were active suicidal or violent behaviors, substance dependence, evidence of an organic brain syndrome, bipolar or psychotic disorder, or changes in medication during the previous month. All referred clients (N=20) were screened, and three were excluded from participation because of active substance addiction, a recent history of violent behavior, or a recent change in medication. The participants took part in a clinical interview and were screened by the first author with the Structured Clinical Interview for DSM-IV (I and II). Of the 17 eligible clients, the first four to accept an offer of participation were included in the study. The therapist was an ISTDP therapist with more than 15 years of experience.

Definition of Unlocking

Unlocking was defined as the process of overcoming resistance (defenses), leading to the conscious experience of previously unconscious complex feelings. Unlocking typically follows a predictable sequence. As anxiety and resistance are overcome, the client becomes aware of complex feelings toward the therapist. The client undergoes a process of visualizing the enactment of the transference of rage onto the therapist and then immediately focuses on positive feelings toward the therapist, which produces feelings of guilt. The conscious experience of the previously unconscious guilt enables the client to enter an “unlocked” state. In this state, memories, feelings, and fantasies from early life are consciously accessible. Working with the therapist, the client develops insight regarding the links among the experience of complex feelings resulting from the transference within the therapy, previous attachment experiences, and defensive processes that had operated to repress this material (1–3).

Measures

The ATOS.

The ATOS (10) is a process measure of therapeutic change. The scale was designed to measure seven therapeutic objectives by observing in-session patient behavior (18). The ATOS comprises seven subscales, outlined in Table 1.

The ATOS manual (affectphobiatherapy.com/wp-content/uploads/2013/10/ATOS-scale-manual.pdf) indicates that video-recorded sessions should be rated in 10-minute segments. For each segment, every subscale is rated holistically on a scale from 1 to 100. Higher scores indicate higher levels of the observed construct. Raters are provided with a description of the construct being rated. For the exposure subscale, raters are also required to identify the dominant affect or affects present in the session. ATOS scores can be used for specific 10-minute segments or averaged across segments to provide an overall session score. Prior studies indicate that the ATOS has satisfactory psychometric properties (13, 16–18). Previous research has also found satisfactory interrater reliability, ranging from 0.60 to 0.87 for various subscales (17, 18).

The ATOS manual provides cutoff scores defining five levels for each subscale. The lowest subscale scores (levels 1 and 2) indicate very limited demonstration of the construct. The patient may be able to demonstrate the construct only with the therapist’s assistance. The moderate level (level 3) indicates a fair, functional demonstration of the construct being measured. The highest subscale scores (levels 4 and 5) indicate high levels of the construct, independently demonstrated by the patient. In the current study, the ATOS manual was used to inform a profile of cutoff scores to define the presence of an unlocking event (exposure, insight, and motivation >50; inhibition <50). The four subscales were chosen because of their theoretical relevance to the concept of unlocking. On the basis of the ATOS scoring system, the defined cutoff scores indicate at least a moderate level of insight and motivation. The cutoff score for the exposure subscale indicates moderate or higher levels of affect intensity. For the exposure subscale, raters identified the dominant affect present in each 10-minute segment. Each affect received a rating on the subscale, with a rating of >50 on at least one affect required to meet the cutoff. The cutoff score for the inhibition subscale indicates moderate to low levels of inhibitory affect and maps onto the core characteristics of an unlocking episode. Unlocking involves the experience of intense, complex feelings (analogous to higher scores on the exposure subscale), insight into previously unconscious issues (analogous to higher scores on the insight subscale), a reduction in defenses (analogous to lower scores on the inhibition subscale), and increased motivation to surrender defensive processes (analogous to higher scores on the motivation subscale) (1, 2). An unlocking episode was identified if all subscale cutoff scores were met within a 10-minute segment. If an unlocking episode persisted across multiple 10-minute segments, all of those segments were considered unlocking segments.

Observational rating system for unlocking events.

Expert raters were asked to make a clinical judgment after watching 10-minute video-recorded therapy segments to determine whether unlocking had occurred. A total of 192 therapy segments were rated. A binary rating system was used to identify whether an unlocking event occurred on the basis of the raters’ clinical judgment (1=unlocking of the unconscious occurred during therapy, 0=unlocking of the unconscious did not occur during therapy).

Raters

All raters in this study achieved competency on the standard online training course for the ATOS. This course involves theoretical readings in addition to extensive practice rating therapy segments and comparing results with those of expert raters. In addition, raters completed an 8-hour training session with an ATOS trainer and received specific feedback. Overall, each rater completed 30 hours of practice with the ATOS before providing ratings for this study. This level of training is in line with the 25–30 hours of training recommended by the scale developers (12). At the end of training, the two principal raters (P.H. and S.B.B.) achieved reliability coefficients of 0.73 and 0.78, respectively (compared with expert ATOS raters), placing them in the good-to-excellent range for the ATOS (12).

Therapist.

The therapist was a trained ISTDP clinician with over 15 years’ experience. After each session, the therapist identified whether an unlocking event had taken place in the session and the number of unlocking events. Any instances identified by the therapist were included in the total number of unlocking events for the analyses.

Independent raters.

Two independent raters assessed the occurrence of unlocking events and provided ATOS scores after watching recorded therapy segments. Ratings for unlocking events and ATOS scores were made 12 weeks apart to reduce the likelihood of bias in the data. The raters were both trained ISTDP therapists with a minimum of 5 years’ clinical experience and with previous experience rating unlocking by watching ISTDP videos. The raters were blind to patient outcomes. To assess the likelihood of experimenter bias, a third rater (Z.I.) also assessed the occurrence of unlocking while blind to patient outcomes, ATOS results, and unlocking results.

Procedure

Patients attended therapy weekly, with the number of total sessions dependent on patient need. The duration of the initial session was 120 minutes, and the subsequent sessions were between 50 and 60 minutes. A total of 28 sessions were provided. All sessions were video recorded, and up to the first 10 sessions for each client were assessed for unlocking events by using clinical judgment and the ATOS. The therapist was blind to outcomes for all patients until the end of treatment.

Results

Participants

The participants were three men and one woman, ranging in age from 35 to 70 years (mean±SD= 47±13 years). One patient dropped out of treatment after the sixth session and could not be contacted. One patient participated in 4 hours of treatment (two 2-hour sessions) before ceasing treatment because of financial limitations. Participants’ demographic and clinical characteristics are outlined in Table 2.

Interrater Reliability for Unlocking Episodes

Data were analyzed with SPSS, version 22.0. Interrater reliability for unlocking episodes was assessed by using intraclass correlation coefficients (ICCs). Results indicated consistency among the therapist and the three independent raters. Based on previous studies, the average reliability value is expected to be >0.70. The current study found an excellent reliability value (ICC α=0.93).

Unlocking Events Identified by Expert Raters and by Using the ATOS

Overall, the therapist and independent raters identified 11 instances of unlocking. All of these unlocking events occurred with one participant; no other participant experienced an unlocking event. To be considered valid, at least three of the four raters (including the therapist) had to agree on the rating. Of the 11 unlocking episodes identified by the raters, seven met all of the specified ATOS criteria for an unlocking episode (exposure, insight, and motivation >50; inhibition <50) (Table 3).

Comparison of ATOS Scores for Sessions With Versus Without Unlocking

ATOS ratings were completed for all 10-minute therapy segments. These segment scores were then averaged to create an overall session score. Mean ATOS subscale scores were compared for sessions and segments where unlocking occurred versus those where unlocking did not occur (Table 4).

A visual analysis of the data indicated that inhibition scores were lower overall in sessions where unlocking occurred compared with sessions where unlocking did not occur (Table 4). Inhibition was also lower during the 10-minute unlocking segments compared with the unlocking sessions as a whole. Exposure was higher during unlocking segments than during sessions or segments with no unlocking. The motivation and insight subscales showed little variation between segments and sessions.

As part of the ATOS exposure subscale, raters identified the dominant affect present in each session. On the basis of ISTDP theory, unlocking events are typically characterized by the presence of both anger and guilt about the anger due to feelings of attachment. The proportion of sessions featuring anger and guilt about the anger as dominant emotions was compared for sessions with and sessions without unlocking (Table 5). As shown in Table 5, complex feelings occurred at a higher rate in sessions where unlocking occurred, compared with sessions with no unlocking.

Discussion

The current pilot study aimed to examine the feasibility of an ATOS-derived profile as a measure of unlocking of the unconscious in a naturalistic therapy setting. This study represents an initial effort to operationalize the concept of unlocking in ISTDP. Three expert raters and the treating therapist assessed for instances of unlocking for four participants for up to their first 10 sessions of therapy. These unlocking events were then assessed with the ATOS by using a profile of predefined ATOS subscale cutoff scores to identify whether unlocking had occurred.

Results indicated strong interrater reliability with regard to the identification of unlocking events, suggesting that unlocking episodes are consistently identifiable events within the therapy. Future research should explore interrater reliability in identifying unlocking, using larger sample sizes and taking into account factors such as raters’ experience, the patient’s systems of resistance, and the level or degree of unlocking.

Results suggest that the ATOS may have the capacity to identify instances of unlocking when the defined subscale cutoff scores are used. Of the 11 unlocking events identified by expert raters, the ATOS identified seven as likely instances of unlocking, suggesting that the measure is sensitive to unlocking events. Results regarding average subscale scores also supported ISTDP theory. As expected, inhibition scores were lowest during episodes of unlocking and were lower in sessions where unlocking occurred compared with sessions where unlocking did not occur. Motivation, insight, and exposure were all highest during episodes of unlocking. This finding supports the theoretical characteristics of unlocking events, which are characterized by a greater willingness to give up defenses, less intense defensive processes, increased insight, and increased emotional arousal (3, 8).

Davanloo, the founder of ISTDP, argued that the experience of previously avoided emotions (without anxiety and defenses) accounts for the positive changes observed in ISTDP patients (3). Unconscious anxiety and defenses decrease with rising emotional awareness, allowing formerly unsettled feelings to be processed in a healthy way (3). Thus, the current findings suggest that the conceptual focus of the ATOS may be a good match for the development of a profile that can help identify unlocking events.

Raters found that complex feelings occurred more frequently in sessions where unlocking occurred compared with sessions with no unlocking. These findings suggest that this emotional profile may be an important identifier on the ATOS measure.

This study had several limitations. As a pilot study, the research was limited in scope. The study’s aim was an initial exploration of the feasibility of using a structured measure of unlocking. It thus employed a small sample of four participants within a single treatment setting, limiting the conclusions that can be drawn from the research. Because of the small sample, comparisons between average subscale scores were limited to a visual analysis of the data. In addition, only the first two to 10 sessions of therapy were examined for each participant. A patient’s experience of unlocking within therapy may change over time, particularly as the relationship with the therapist develops. In this small sample, only one participant experienced unlocking, which raises the possibility that the results may have been unique to that individual’s experience of the therapy. Future studies could increase the sample size and examine unlocking events over the course of a longer-term intervention. Two of the raters assessed episodes of unlocking and the ATOS scores across four subscales. Although this approach was necessary because of resource limitations in the study, it raised the possibility of rater bias. To address this issue, a third rater identified unlocking episodes while blind to previous unlocking and ATOS scores, and results correlated highly with those of the first two raters. However, future studies would benefit from having separate raters identify instances of unlocking and complete the ATOS.

With regard to outcomes for the ATOS measure, results indicate that the scale may be sensitive to instances of unlocking; however, the specificity of the measure has not yet been established. It is unknown whether the ATOS is effective in differentiating between instances of unlocking and other events within the therapy session. Future studies could assess this aspect of the measure by comparing ATOS ratings for unlocking and for other significant events in the therapy across a range of clients with different presentations. Future research could also examine the factor loadings of different subscales to assess whether certain subscales are particularly sensitive to identifying unlocking. The ATOS manual specifies that segments should be rated in 10-minute increments. This approach may make it difficult to assess episodes of unlocking that occur gradually over the course of a session. Future studies may focus on exploring alternative measures and examine the capacity of the ATOS to identify unlocking that occurs over time. The ATOS manual also does not provide clear definitions of dominant affects identified for each segment. Current literature supports the interrater reliability of the ATOS; however, the lack of definitions regarding dominant affects in the manual may reduce clarity in identifying unlocking episodes. A helpful step for future researchers would be to develop clearer definitions regarding the microprocesses underlying unlocking, particularly regarding the dominant affects involved.

Conclusions

Unlocking the unconscious is purported to be central to the process of change in ISTDP. The current study operationalized and provided an initial exploration of the use of an ATOS profile as a potential measure to identify unlocking of the unconscious in ISTDP. The study is the first to examine the interrater reliability of identifying episodes of unlocking using clinical judgment. Results indicate that unlocking episodes can be operationalized and reliably identified. Future research may incorporate the proposed measure to better understand the process of change in ISTDP, with a goal of enabling a better understanding of alliance-related, transtheoretical, and other factors associated with therapeutic change in specific patient populations.