Research on Social Work Practice http://rsw.sagepub.com
Contributions of Therapist Characteristics and Stability to Intensive In-home Therapy Youth Outcomes Johanna K. P. Greeson, Shenyang Guo, Richard P. Barth, Sarah Hurley and Jocelyn Sisson Research on Social Work Practice 2009; 19; 239 DOI: 10.1177/1049731508329422 The online version of this article can be found at: http://rsw.sagepub.com/cgi/content/abstract/19/2/239
Published by: http://www.sagepublications.com
Additional services and information for Research on Social Work Practice can be found at: Email Alerts: http://rsw.sagepub.com/cgi/alerts Subscriptions: http://rsw.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations http://rsw.sagepub.com/cgi/content/refs/19/2/239
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Contributions of Therapist Characteristics and Stability to Intensive In-home Therapy Youth Outcomes
Research on Social Work Practice Volume 19 Number 2 March 2009 239-250 © 2009 Sage Publications 10.1177/1049731508329422 http://rswp.sagepub.com hosted at http://online.sagepub.com
Johanna K. P. Greeson Shenyang Guo School of Social Work, University of North Carolina at Chapel Hill
Richard P. Barth School of Social Work, University of Maryland
Sarah Hurley Jocelyn Sisson Youth Villages, Memphis, Tennessee
Objective: This study examines the influence of therapist and youth characteristics on post-discharge outcomes from intensive in-home therapy. Method: Data for 1,416 youth and 412 therapists were obtained from a behavioral health services provider. The Huber–White method was used to account for nested data; ordered logistic regression was employed to assess outcomes. Results: Therapist gender and employment stability were significantly associated with youth outcomes. The likelihood of an undesirable outcome was significantly less for cases with female therapists. Conclusion: Findings underscore the need for additional study concerning the impact of therapist characteristics and stability on youth outcomes, and to improve the understanding of the relationship between the two. Future studies in these areas would advance social work practice in family-based treatment programs. Keywords: multisystemic therapy; family therapy; in-home therapy; therapist characteristics; youth characteristics; outcomes
O
ver the course of the last decade, researchers have increasingly turned their attention to the contributions that both therapist and client characteristics make to treatment outcomes of interventions used in youth and family therapy (Karver, Handelsman, Fields, & Bickman, 2006; Shirk & Karver, 2003). These recent studies have underscored the therapeutic relationship as a critical factor influencing successful outcomes (Goldfried, 1998;
Authors’ Note: This study was supported by a grant to Youth Villages of Memphis, Tennessee, from the Hyde Family Foundations. Findings, opinions, and recommendations expressed in this article are those of the authors and not necessarily those of Youth Villages or the Hyde Family Foundations. We are grateful to Diane Wyant for review of an earlier version, and to Rebecca L. Green for her data management assistance. Correspondence may be addressed to Johanna K. P. Greeson, School of Social Work, University of North Carolina at Chapel Hill, 325 Pittsboro St., CB #3550, Suite 400, Chapel Hill, NC 27599; e-mail jgreeson@email.unc.edu.
Johnson, Ketring, Rohacs, & Brewer, 2006; Kazdin, Marciano, & Whitley, 2005; Shirk & Saiz, 1992). Other characteristics including therapist skills (i.e., both interpersonal and direct influence), youth and parent willingness to participate in therapy, and youth and parent participation in treatment have also been demonstrated as predictors of youth outcomes (Karver et al., 2006). In addition, researchers have identified the type of problem manifested by the client as an important factor for treatment outcomes. That is, studies have shown that outcomes of children with externalizing disorders are more strongly associated with the therapeutic relationship than are the outcomes of children with internalizing disorders (Shirk & Karver). Furthermore, other youth risk factors have also been identified as significant predictors of family therapy outcomes, including age, multiple child maltreatment types, and past secure placement (Barth et al., 2007a).
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
239
240 Research on Social Work Practice
Among the most revealing findings in youth and family therapy studies are those from research on multisystemic therapy (MST), which is a comprehensive, short-term, manualized, home- and community-based intervention for troubled youth and their families. Although in-home therapy is a relatively new approach in youth and family therapy, a considerable body of evidence (i.e., 11 randomized controlled trials that involved 1,300 families, one meta-analysis of 11 outcome studies [Curtis, Ronan, & Borduin, 2004], and one systematic review of 27 studies [Littell, 2005]) has already established MST as one of the most examined treatment approaches for troubled youth and their families (Borduin et al., 1995; Henggeler et al., 2006; Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Henggeler, Melton, & Smith, 1992). Furthermore, several studies have demonstrated that processes within family therapy, such as treatment fidelity (Henggeler et al., 1997; Huey, Henggeler, Brondino, & Pickrel, 2000) and peer relations (Huey et al., 2000) were associated with the changes observed in youth and their families. However, with the exception of this understanding, the pathways by which family therapy works to promote positive outcomes remain under investigated. Moreover, despite growing recognition of the rapid turnover of the clinical workforce (Blankertz & Robinson, 1997), the contribution of therapist employment stability in influencing treatment outcomes remains unknown. Annual estimates of turnover among mental health and human service agencies often exceed 25% of therapeutic staff (Gallon, Gabriel, & Knudsen, 2003), and some estimates have exceeded 50% turnover of qualified therapists (Glisson, Dukes, & Green, 2006; Glisson & James, 2002). The organizational costs associated with turnover, including recruitment and training of new employees as well as the impact of turnover on the quality of services provided, have been well documented (Glisson, 2002; Knudsen, Johnson, & Roman, 2003). However, much less is known regarding the impact of therapist employment stability on client outcomes. Given the importance of the therapeutic relationship, a better understanding of the implications of turnover may inform organizational and personnel decision making regarding critical client treatment and staff retention issues.
Therapist Characteristics In the most comprehensive analysis of therapeutic constructs in the youth treatment literature to date, Karver and his colleagues (2006) examined 49 studies of
youth interventions and investigated the interaction of several variables with the therapeutic relationship to determine how well these factors explained the variance in youth treatment outcomes; their findings revealed modest to strong correlations. Moreover, the results of this meta-analysis showed that robust predictors of youth outcomes included both the therapist’s interpersonal and direct influence skills. In addition to therapist skills, the amount of time spent with families has also been examined as a predictor of youth and family therapy outcomes. In their study of child and adolescent impairments and functioning among 111 youth at a community services agency in the United States, Yorgason, McWey, and Felts (2005) found that therapists spent a wide range of hours with families (2.5 to 242 hrs) within a relatively short period, 4 months (M = 4.33, SD = 2.7). This finding suggested families who received a greater amount of the therapist’s time during the few months of service experienced better treatment outcomes.
Youth Characteristics Studies have also examined the influence of youth characteristics on treatment outcomes in child and family therapy. Yorgason and his associates (2005) identified several significant indicators of treatment outcome that included history of hospitalizations, school or work performance, community interactions, and self-harm behaviors. Client gender, race, living arrangements, county of residence, and family income were not significantly related to treatment outcome. Similarly, in a study examining outcomes at 1-year post-discharge for 862 at-risk youth who received intensive in-home family therapy, Barth and his colleagues (2007a) found that exposure to more maltreatment types, and the experience of receiving treatment in a secure setting (including past partial hospitalization, residential treatment, or inpatient treatment) predicted less likelihood that youth would remain living with their family and greater likelihood of out-of-home placements. In addition, these researchers also identified age as a significant predictor of treatment outcome; when measured at the 1-year post-discharge follow-up, the probability of youth having experienced trouble with the law increased by 1% for each month increase in age at intake. Meta-analyses have identified the influence of youth characteristics on treatment outcomes across a variety of studies. In a meta-analysis of 23 studies drawn from the youth and family therapy literature, Shirk and Karver (2003) examined interactions of therapeutic relationship
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Greeson et al. / Contributions of Therapist Characteristics and Stability 241
variables and treatment outcomes in child and adolescent therapy. Although these researchers found only a modest association between relationship variables and treatment outcomes, they also discovered greater variability among the results than might be expected based on chance alone. Such variability suggested the presence of moderator variables that were affecting the interplay of therapeutic relationships and outcomes. Further analyses revealed that the association of the therapeutic relationship and outcomes was moderated by the client’s type of problem behavior (externalizing versus internalizing); that is, the relationship and outcome association varied as a function of the type of client problem. Although other moderators between the therapeutic relationship and client outcomes were hypothesized, including patient age; type of treatment (behavioral versus nonbehavioral); mode of treatment (individual, family, and parent treatments); level of treatment structure (manualized versus nonmanualized interventions); and the context of therapy (service therapy versus research therapy), these characteristics were shown to be nonsignificant. The meta-analysis of Karver et al. (2006) also revealed that youth and parent characteristics were robust predictors of treatment outcomes. These characteristics included youths’ willingness to participate in treatment, parental willingness to participate in treatment, youth participation in treatment, and parent participation in treatment; however, the later study lacked adequate statistical power for each construct domain. To address this problem, Karver et al. aggregated all construct domains. However, their subsequent results did not support that any of the hypothesized characteristics (i.e., age of client sample, referral problem, treatment type, and time of measurement of the process variable) were significant moderators of the overall association between therapeutic process and treatment outcomes.
Therapist Stability Rates, reasons, and consequences of provider turnover in child and family services have been the focus of research for several decades (e.g., Blankhertz & Robinson, 1997; Gallon et al., 2003; Glisson & James, 2002; Jayaratne & Chess, 1984). With estimates of turnover rates ranging from 20% to 52% of treatment staff, these rates vary further by field. For example, estimates of treatment staff turnover in community mental health settings hover around 20% (Blankhertz & Robinson), whereas turnover rates in agencies providing substance abuse services are nearly 25% (Gallon et al.). With staff turnover rates nearing 50%,
public child welfare and juvenile justice agencies have some of the highest turnover estimates across the child and family services fields (Glisson & James). Just as rates of turnover vary by field, so too do the hypotheses regarding why turnover occurs, but most commonly include financial reasons, role ambiguity, level of stress, and amount of social support (e.g., Jayaratne & Chess; Nissly, Mor Barak, & Levin, 2005).
Therapist Characteristics, Therapist Stability, and Youth Age, Race, and Risk Factors As demonstrated in our review of the literature, provider turnover in the human services field has received considerable attention. Although such studies have provided a relatively comprehensive understanding of many of the factors that influence provider turnover, even the field’s best intentions to decrease these turnover rates will take time to develop, implement, and produce an effect. Therefore, increasing our understanding of the impact of provider turnover on youth and family outcomes has the potential to improve the operation and effectiveness of treatment programs in the current service context in which the therapeutic relationship is often disrupted. The present study examined the contribution of therapist characteristics and employment stability to treatment outcomes in an evidence-based family- and community-focused intervention. We developed several conceptual models based on the literature related to therapist characteristics, employment stability, and youth characteristics, which were used to guide the formulation of the following study questions regarding main effects: Are therapist characteristics, such as race, gender, age, level of education, and experience, associated with 1year post-discharge outcomes? Is therapist employment stability associated with 1year post-discharge outcomes? Are therapist characteristics associated with therapist employment stability? Are youth characteristics associated with therapist employment stability?
In addition, we examined the therapist as an independent factor, which took into account the characteristics that the therapist contributed to the therapeutic process, by asking the following questions about moderating effects:
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
242 Research on Social Work Practice
Do therapist characteristics moderate the relationship between therapist employment stability and 1-year post-discharge outcomes controlling for the effect of youth characteristics? Does therapist experience moderate the relationship between therapist level of education and therapist employment stability controlling for the effect of youth characteristics?
Method Design Beginning in 1994, the agency involved in the investigation was among the first providers in the nation to offer MST. Currently, the agency has multiple licensed MST teams providing services in Washington, D.C., Dallas, Texas, and across the state of North Carolina. In the agency’s other locations (i.e., Tennessee, Alabama, Mississippi, Massachusetts), it offers intensive inhome services through a second program called Intercept. These locations serve primarily the child welfare and mental health populations. Both of the agency’s intensive in-home programs—MST and Intercept—have characteristics similar to other intensive models including intensive case supervision, low caseloads, staff on call to families 24/7, and extensive pre-service and inservice training. This agency has an ongoing outcome evaluation system, with outcome data up to two years post-discharge for over 10,000 youth served since 1994. Standardized client information is collected by the agency at intake and records are maintained in an electronic information system. All study procedures were approved by the Institutional Review Board of the University of North Carolina at Chapel Hill.
Sample Description Youth. Data were extracted and analyzed from a sample of 1,965 closed cases. The 1,965 cases represent all youth who received only intensive in-home therapy and who were discharged between July 2000 and June 2004. Using listwise deletion, cases were deleted based on three criteria: (a) missing treatment notes, (b) length of service, and (c) diagnostic criteria. From the original sample, 145 cases (7.4%) were excluded because of missing treatment notes; of the remaining 1,820 cases, another 20.2% (n = 368) were excluded based on length of service criteria defined as less than 90 days or more than 408 days (M = 150.1, SD = 49.4). Cases with length of service of less than 90 days were considered uninformative to this investigation because the 90 days was considered the minimal
optimal “dose” of treatment. Cases with length of service in excess of 408 days were excluded based on convention that deletes all cases with a length of service that is equal to or greater than 3 times the standard deviation above the mean. Finally, of the remaining 1,452 cases, 2.5% (n = 36) were excluded based on certain mental disorders, including pervasive developmental disorders, sexual disorders, and psychotic disorders. Cases with these diagnoses were excluded because they were neither typical nor normative of the sample population. Our final analytic sample of 1,416 youth cases represented 72.1% of all the cases that received intensive in-home therapy during the study period (July 2000 to June 2004). Table 1 presents the demographic characteristics of the youth included in the study sample. The mean age of the youth at admission was 13.1 years (SD = 3.2); two thirds of the sample were males (66.7%), and about three quarters were Caucasian (76.4%). Based on the DSM-IV (American Psychiatric Association, 1994) diagnostic criteria, about 30% of the youth were identified as having behavior or conduct disorders, and another 30% were identified as having a mood disorder. In addition, youth risk factors were evaluated at intake, with delinquency (56.1%) identified as the most common risk factor. Approximately two thirds of the sample had received mental health treatment prior to their intake for the in-home treatment (63.9%); of these youth, nearly one half (43.9%) had received past day treatment, outpatient treatment, or group therapy, and almost 30% had received past secure treatment in an inpatient setting, partial hospitalization, or residential treatment setting. Therapists. Similar to the youth study sample, therapist cases were screened for missing data, and all cases missing identification information were excluded. The final study sample of therapists comprised 412 therapists who provided services to 1,373 youth in our study sample; 43 of the 1,416 total youth sample had unknown therapists. The demographic characteristics for the therapist study sample are shown in Table 2. Most therapists were Caucasian (74.5%) and female (79.1%). Almost equal proportions of therapists had master’s degrees (43.7%) as compared with those with bachelor’s degrees (48.5%). The remaining 7.8% of the therapist sample was missing data for the level of education. The most commonly reported academic majors were psychology (27.4%), social work (19.7%), and counseling (17.0%). Of the 412 therapists, 26 cases were missing a date of birth or the therapist’s hiring date. For the remaining 386 therapists, the mean age at date of hire was 29.6 years (SD = 7.9).
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Greeson et al. / Contributions of Therapist Characteristics and Stability 243
Table 2 Therapist Sample Demographic Characteristics (n = 412)
Table 1 Youth Sample Demographic Characteristics (n = 1,416) Variable
% or M (SD)
Race Caucasian African American Gender Male Female Age in years at intake Length of service in days Mental health diagnosis Behavior or conduct disorders Mood disorders Other disordersa Missing/No diagnosis/Diagnosis deferred Risk factors identified at intake Mental health problems Runaway behavior Committed a sex offense Status offense chargeb Other criminal behaviorc Simple assault Child maltreatment Siblings in out-of-home care Delinquencyd Substance abuse Gang involvement Past treatment received Mental health Inpatient, partial hospitalization, or residential treatment Day treatment, outpatient treatment, or group therapy Past foster care placement Referral source Tennessee Department of Children’s Services TennCare – Tennessee’s Medicaid Waiver Program Community Services Agency Mississippi Department of Human Services Othere Program location Urban Area (Pop. 50,000+) Non-Urban Area a
76.4% 23.6% 66.7% 33.3% 13.1 (3.2) 105.1 (49.4) 30.2% 29.4% 11.7% 28.7% 47.5% 10.7% 6.7% 14.3% 14.1% 4.9% 13.2% 2.4% 56.1% 13.3% 1.3% 63.9% 28.0% 43.9% 4.7% 3.0% 81.5% 8.7% 5.2% 2.0% 55.9% 44.1%
Includes adjustment, anxiety, impulse control, substance-related, and personality disorders, and V-codes b Includes runaway, incorrigible, truancy, violated probation, violated curfew, or alcohol related c Includes arson, auto theft, burglary, theft, aggravated assault, kidnapping, homicide, violent sex offense, robbery, drug trafficking, drug possession, DUI, nonviolent sex offense, or possession of a weapon d Includes aggression, assault, fire setting, gang problem, oppositional behavior, runaway, theft, or truancy e Includes private insurance, charity care, and unique funding agreements
Variable Race Caucasian African American Other Gender Male Female Age in Years on Hire Date Therapist Experience in Monthsa Level of Education Bachelor’s Master’s Major Counseling Criminal Justice Education Psychology Religion Social Work Sociology/Anthropology Other
% or M (SD) 74.5% 20.9% 0.5% 18.2% 79.1% 29.6 (7.9) 7.7 (9.6) 48.5% 43.7% 17.0% 3.9% 6.6% 27.4% 4.4% 19.7% 6.1% 6.6%
Note: Percentages do not total 100% due to missing data. a Group mean for number of months between date of hire and date of first therapy note written for youth in the sample population.
Measures Youth and therapist characteristics. Youth administrative data were obtained from the provider agency’s electronic case management information system, and included gender, race, age at intake, length of service, mental health diagnosis, presenting problems, past services received, referral source, and urbanization of the program location where the youth was served. Therapist administrative data were gathered by the provider agency as part of routine human resource activities for newly hired staff. These therapist data included gender, race, age at time of hire, level of education, and major. It is worth noting that months of experience, a measure derived from the electronic case management system, was calculated at the youth level to indicate the therapist’s length of experience at the beginning of each case. Therapist stability and experience. First, the primary therapist for each case was identified. This was accomplished by dividing the total number of family therapy notes, which were written following each session, by the client’s total number of notes to derive the percent of notes written by each therapist. The therapist with the highest percent of notes was identified as the primary therapist.
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
244 Research on Social Work Practice
Next, the stability variable was created by recoding the percent of notes written by the primary therapist into the following categories: high, moderate, and low. Cases for which 100% of the youth’s case notes were written by one therapist were coded as having high therapist stability; cases with 75% to 99% of the youth’s case notes written by one therapist were coded as having moderate therapist stability, and cases with 74% or less of the youth’s case notes written by one therapist were coded as having low therapist stability. Similarly, the therapist’s experience variable was calculated by subtracting the date of the primary therapist’s first therapy note for each youth case from his or her hire date; thus, therapist experience was a continuous variable. One-year post-discharge outcomes. The 1-year postdischarge data regarding the youths’ educational attainment, legal problems, living arrangements, and level of care were collected by the staff of the provider agency. Telephone interviews were conducted with the parents, guardians, youth (if they were over 15 at the time of the interview), or custodial agents. Data from the provider’s interviews regarding youths’ living arrangements at the 1-year followup were recoded for our study purposes as either “family” or “not family.” Out-of-home placements that occurred in the 6 months prior to follow-up and any trouble with the law since treatment discharge were dichotomized into a “yes/no” variable, which was based on the provider agency’s original variables. For youth in the study sample, out-of-home placements included regular foster care homes, adult jail, drug or alcohol rehabilitation centers, diagnostic centers, emergency shelter, group homes, half-way homes, juvenile corrections, psychiatric hospitals, residential treatment centers, or therapeutic foster care homes. The variable for educational achievement was derived from two categories of current school status: (a) “making/made progress” (i.e., youth attends school or high school equivalency classes (GED), or graduated high school/received GED); and (b) “dropped out/does not attend.” For the purposes of our analysis and consistent with previous research using these data (Barth et al., 2007a; 2007b), we created a composite outcome scale using three categories—desirable, mixed, and undesirable— that were derived from the original 1-year outcome variables, including living with family, educational attainment, whether the youth had experienced trouble with the law, and whether an out-of-home placement had occurred. The desirable category consisted of those cases with positive indications for four criteria: living with family, making progress in school, no incidents of trouble with the law, and no out-of-home placements. The
mixed outcome category comprised those cases with indications for living with family, and a positive indication for at least one of the following three criteria: not making progress in school, has experienced trouble with the law, or has experienced an out-of-home placement. The undesirable outcome category consisted of those cases with an indication that the youth was not living with his or her family.
Analytic Models The youth and therapist study samples differed by the variables selected for inclusion in the analytic models as well as by patterns of missing data. The final youth sample used as the reference point for each analysis consisted of 1,416 youth cases. The primary study variable— the 1-year post-discharge composite outcome—had valid data for 602 youth who received services from 265 therapists. For therapist stability, the final model included 1,238 youth who received services from 378 therapists. The 1-year post-discharge composite outcome was the study variable of primary interest; therefore, we conducted bivariate analyses using this outcome to determine if the individuals included (n = 602) were different from individuals excluded (n = 814) due to missing values. Chi-square and t-test statistics were estimated for age, race, gender, length of stay, mental health problems at intake, delinquency problems at intake, and past mental health treatment. Of the seven variables, only race emerged as significant. Of all the excluded youth, 27% were African American. Of all the African American youth, 64% were excluded and 37% were included in the analysis [χ2(1, n = 1,416) = 6.83, p < .05]. Analysis of the study samples was confounded by nested data. Because each therapist could work with one or more youth, the youth data were nested within therapists. However, our interest was in understanding the overall effects (i.e., aggregated over all groups), rather than an evaluation of effect variations across groups. Therefore, to address the dependency of the residuals, the F-statistics and standard error estimates were corrected by the Huber–White method using robust (sandwich) standard errors. All analyses were performed using Stata 9 (StataCorp LP, 2007). We constructed two ordered logistic regression models to answer the research questions. The purpose of the first model was to evaluate whether therapist characteristics were associated with therapist stability when controlling for the affect of youth characteristics. The purpose of the second model was to test the main and moderating effects of therapist characteristics on the 1-year post-discharge composite outcome when controlling for the effect of
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Greeson et al. / Contributions of Therapist Characteristics and Stability 245
youth characteristics. Because both outcome variables were ordinal (i.e., therapist stability had three levels based on percentage of case notes written by one therapist: 1 = high or 100% of notes, 2 = moderate or 75% to 99% of notes, and 3 = low or less than 75% of notes; and the 1-year post-discharge outcome had three levels: 1 = desirable, 2 = mixed, and 3 = undesirable), the ordered logistic regression was deemed appropriate. Based on prior studies, available data, and a systematic data exploration, specific youth risk factors were selected as control variables. Four logistic regressions were modeled, and each regression analyzed a binary outcome (i.e., education achievement, trouble with the law, current living arrangement, and out-of-home placements in past 6 months). This process was necessary because after listwise deletion of missing data, the sample size was reduced substantially. Because of the reduced sample size, some risk factors had only a small proportion of positive responses and, therefore, were not acceptable for logistic regression. To select meaningful independent variables, we first retained all independent variables derived from the four logistic regression models. A systematic check of model-fit indices and odds ratios resulted in the final model containing four youth risk factors: (a) maltreatment status, (b) mental health problems, (c) past receipt of mental health treatment, and (d) past day treatment, outpatient treatment, or group therapy. To answer the research questions pertaining to moderating effects of therapist characteristics, the research team systematically searched significant interactions. We tested four interactions separately to assess whether therapist’s race, age, gender, level of education, and experience moderated the relationship between therapist instability and 1-year post-discharge outcomes. Of the four interactions, only therapist stability by therapist gender was shown to have statistical significance. As such, the therapist stability by gender interaction was included in the final model.
Results Descriptive Statistics Therapist stability. As previously mentioned, youth cases with an unknown therapist (n = 43) were excluded from the analytic sample. The 1,373 remaining youth cases represented 97.0% of the analytic sample. Of these youth cases, almost half had moderate therapist stability (n = 703; 51.2%); and nearly equal proportions had either high therapist stability (n = 339; 24.7%) or low therapist stability (n = 331; 24.1%). Put differently,
almost three quarters of the youth cases had either high or moderate therapist stability. Therapist experience. Youth cases were excluded from the analytic sample (N = 1,416) if the youth’s therapist case was missing the date of hire (n = 123; 8.69%). The remaining youth cases (n = 1,293) represented 91.3% of the analysis sample. Between therapist date of hire and date of first therapy note, the mean therapist experience at the agency was 7.69 months (SD = 9.67) with a median of 4.6 months. One-year post-discharge outcomes. Based on the analytic sample of 1,416 youth cases, the following outcomes were reported to the provider agency one year following discharge from the intensive in-home therapy program: 54.9% (n = 777) of the sample was living with family, and 10.0% (n = 141) were not living with family; 498 cases were missing data on current living arrangements. At 1-year post-discharge, 54.6% (n = 773) of youth had not experienced an out-of-home placement in the preceding 6 months, whereas 10.2% (n = 145) had experienced some type of an out-of-home placement; 498 cases were missing data on out-of-home placements. At the followup, 6% (n = 85) of the sample had experienced trouble with the law since treatment discharge, and 36.9% (n = 523) had not had trouble with the law; however, a substantial portion of the analytic sample was missing data for this domain (n = 808). Positive educational progress was reported for 43.9% (n = 621) of the sample as compared with 2.5% (n = 36) who had not made educational progress or had dropped out; data for this domain was missing for 759 cases. As demonstrated, the number of missing cases varied widely by domain, and thus missing cases were excluded by outcome domain. One-year post-discharge composite outcome variable. Based on the analytic sample of 1,416 children and youth, 430 (30.4%) cases achieved a desirable outcome 1-year post-discharge from the intensive in-home therapy program. Another 6.4% (n = 91) of the children and youth achieved a mixed outcome, and 5.7% (n = 81) reported an undesirable outcome. Cases with missing data on any of the outcome variables were excluded, and these deleted cases comprised 57.5% (n = 814) of the analytic sample.
Main Effects Therapist characteristics and composite outcome. Of the therapist demographic characteristics modeled, only therapist gender was significantly associated with 1-year post-discharge outcomes. This main effect of therapist gender indicated that the likelihood of having an undesirable
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
246 Research on Social Work Practice
Table 3 Estimated Ordered Logistic Regression Model Predicting Therapist Stability (n = 1,238 youth; n = 378 therapists) Variable
Coefficient
Therapist Characteristics Therapist's education - Master (No Master) Therapist gender Female (Male) Therapist race African American (Other) Therapist aged 30 or older (Below 30) Youth Characteristics Gender Male (Female) Race African American (Other) Age at adimission (1 = 0−11, 2 = 12−15, 3 = 16+) Child maltreatment - Yes (No) Metal health problems - Yes (No) Past mental health treatment - Yes (No) Past day treatment, outpatient treatment, or group therapy - Yes (No) Intercept 1 Intercept 2 Sample Size (Number of Youth, Number of Therapists) Pseudo R2
−0.272 0.238 0.045 0.096 0.119 0.097 0.020 −0.020 −0.016 0.005 0.227 −0.800 1.469 (1238, 378) .01
Odds Ratio
Robust S.E.
0.762 1.269 1.046 1.101
0.133 * 0.154 0.178 0.133
1.126 1.102 1.020 0.980 0.984 1.005 1.254
0.116 0.141 0.090 0.159 0.166 0.195 0.159 0.270 0.280
Note: Reference group for dichotomous variables is shown in parentheses. Robust standard error (S.E.) is estimated by Huber-White method. *p < .05
youth outcome for a female therapist was 87% lower than that for a male therapist (p < .01), after controlling for the youth’s background characteristics. Therapist demographic variables that were tested but that were shown to be nonsignificant included race, age, level of education, and experience. Therapist characteristics and stability. Of the demographic characteristics modeled for therapists, only the level of education was significantly associated with therapist stability. This main effect of therapist stability indicated that the likelihood of high stability for therapists with a master’s degree was 5% greater than that for therapists without such a degree, and similarly, the likelihood of low stability for therapists with a master’s degree was 5% less (p < .05) than that for therapists without such a degree. Table 3 presents the final model of the ordered logistic regression analyzing the threelevel therapist stability variable. The model explained about 1% of the variance, which is equivalent to a “small” effect size (0.0 to 0.2; Cohen, 1992). Nonsignificant demographic variables included race, gender, and age. Therapist stability and composite outcome. Therapist stability was also significantly associated with 1-year post-discharge outcomes for youths. The main effect of
therapist stability indicated that with each one-level change in stability (i.e., a change from high stability to moderate stability, or a change from moderate stability to low stability), the likelihood of having an undesirable 1-year post-discharge youth outcome decreased by 56% (p < .05). Table 4 presents the final model of the ordered logistic regression analyzing the three-level 1-year postdischarge outcomes. The model explained about 7% of the variance in the outcome variable, which is equivalent to a “medium” to “large” effect size (0.5 to 0.8; Cohen, 1992). Youth characteristics and stability. We also modeled the relationship between the youth demographic characteristics and therapist stability. The youth demographic characteristics included race, gender, age, and risk factors; however, none of the demographic characteristics emerged as statistically significant predictors of therapist stability.
Moderating Effects In addition to main effects, our analyses also investigated moderating effects. The goal was to examine the therapist as an independent factor by considering the characteristics that therapists introduce into the therapeutic process. The following demographic characteristics
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Greeson et al. / Contributions of Therapist Characteristics and Stability 247
Table 4 Estimated Ordered Logistic Regression Model Predicting 1-Year Post-discharge Composite Outcome (n = 602 youth; n = 265 therapists) Variable
Coefficient
Theraphist Characteristics Therapist stability (1 = high, 2 = moderate, 3 = low) Therapist's education - Master (No Master) Therapist gender Female (Male) Therapist race African American (Other) Therapist aged 30 or older (Below 30) Youth Characteristics Gender Male (Female) Race African American (Other) Age at adimission (1 = 0-11, 2 = 12-15, 3 = 16+) Child maltreatment - Yes (No) Metal health problems - Yes (No) Past mental health treatment - Yes (No) Past day treatment, outpatient treatment, or group therapy - Yes (No) Interaction Therapist stability by therapist gender Intercept 1 Intercept 2 Sample Size (Number of Children, Number of Therapists) Pseudo R2
Odds Ratio
Robust S.E.
−0.829 0.219 −2.020 0.102 −0.216
0.437 1.245 0.133 1.107 0.806
0.343 * 0.220 0.739 ** 0.312 0.220
0.288 −0.118 0.708 0.765 −0.600 0.410 −0.589
1.334 0.888 2.030 2.149 0.549 1.507 0.555
0.214 0.247 0.139 *** 0.310 * 0.275 * 0.345 0.280 *
2.876
0.375 ** 0.766 0.761
1.056 0.756 1.815 (602, 265) .07
Note: Reference group for dichotomous variables is shown in parenthesis. Robust standard error (S.E.) is estimated by Huber-White method. *p < .05, ** p < .01, *** p < .001
were investigated as moderators: race, gender, age, and level of education.
Discussion and Applications to Social Work
Therapist gender as moderator. Our results showed that only therapist gender exerted a significant moderating effect on the relationship between therapist stability and 1-year post-discharge outcomes of youth (p < .05). That is, when therapist stability decreased, the predicted probability of having a desirable youth outcome at 1-year post-discharge for a female therapist decreased, whereas the same probability for a male therapist increased. In other words, when therapist stability decreased, the predicted probability of having an undesirable youth outcome at 1-year post-discharge for a female therapist increased, whereas the same probability for a male therapist decreased (Figure 1).
This study investigated the contribution of therapist characteristics and stability over service duration to treatment outcomes in the implementation of an intensive in-home family- and community-focused intervention. This study is one of the few known to the authors that includes individual therapist characteristics and therapist stability as predictors of youth outcomes, and goes beyond rates and predictors of turnover to consider the potential impact on treatment outcomes. Results revealed that therapist gender and therapist stability were significantly associated with 1-year post-discharge youth outcomes, and that therapist level of education was significantly associated with therapist stability. In addition, we explored the moderating effects of therapist characteristics on the relationship between therapist stability and 1-year post-discharge outcome. This approach permitted the examination of the independent influence of the therapist on treatment outcome. In so doing, this study endeavored to take into account the therapist’s contribution to the therapeutic process of the intervention. Thus, this study represents a first step in
Therapist experience as moderator. Because we found that the therapist level of education was significantly associated with therapist stability, we investigated therapists’ experience levels as a potential moderator between therapist level of education and therapist stability. However, no moderating effect of experience on the relationship between level of education and stability was found.
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
248 Research on Social Work Practice
Figure 1 Interactive Effects of Therapist Stability and Therapist’s Gender on 1-Year Post-Discharge Outcome Desirable Outcome 0.9 Predicted Probability
0.8 Therapist's Gender
0.7 0.6 0.5
Male
0.4
Female
0.3 0.2 0.1 0 High
Moderate Therapist Stability
Low
Undesirable Outcome 0.9
Predicted Probability
0.8 0.7
Therapist's Gender
0.6 0.5
Male
0.4
Female
0.3 0.2 0.1 0 High
Moderate Therapist Stability
Low
Mixed Outcome 0.9 Predicted Probability
0.8 Therapist's Gender
0.7 0.6 0.5
Male
0.4
Female
0.3 0.2 0.1 0 High
Moderate
Low
Therapist Stability
developing a comprehensive view of the therapist within a larger change model. Results provide insight that may advance our understanding of both the moderating and mediating processes at work in family therapy, and subsequently improve social work practice in family-based treatment programs.
Several limitations must be considered to adequately interpret our study results. This study was nonexperimental. Consequently, this limitation restricts our ability to (a) conclude causal relationships between therapist characteristics, instability, and the 1-year post-discharge outcomes; and (b) generalize our findings to settings and populations beyond those of the provider agency from which we obtained the study data. The low level of therapist experience on average in our sample also has implications for generalization. We do not know if this low level of experience is typical of intensive in-home therapy programs. Therefore, our findings may not generalize to studies in which there is a much higher mean level of therapist experience. Listwise deletion of missing data reduced the sample size substantially, which, in turn, eliminated several potentially important youth risk factors from the logistic regressions because of empty cell problems. Additionally, a measure of reason for therapist instability was not available. Child outcomes may be differentially affected in cases where therapist instability is due to turnover versus cases where a more experienced therapist was brought in to help a family that was in need of additional expertise. The dependent variable—the 1-year postdischarge outcome—was also not highly sensitive or well distributed; most youth did well, and the model was not able to explain a substantial amount of variation. Although expected given the purpose of the model, other unavailable variables could account for more of the explained variance. Relatedly, this study used administrative data. Such databases are designed primarily for purposes other than research. Therefore, certain limitations are unavoidable, including imprecise measurement of constructs of interest (Sorensen, Sabroe, & Olsen, 1996). Finally, in light of the counterintuitive gender interaction effect for male therapists, interpretations of this finding are speculative. This effect may be a true finding, may be due to a model specification error, or may be due to some undetermined measurement error. Subsequent research will be needed to clarify the significance of this finding. An additional issue that emerged from this study concerns available statistical methodologies. One important question left unanswered by the present investigation is whether therapist stability mediates the effect of therapist characteristics on 1-year post-discharge outcomes. Testing mediation would allow for the simultaneous detection of both the direct effect of therapist characteristics on youth outcomes and the indirect effect of therapist stability. Potentially, results might indicate whether therapist stability accounts for, either fully or partially, the relationship between therapist characteristics and youth outcomes. Such findings would permit conclusions
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
Greeson et al. / Contributions of Therapist Characteristics and Stability 249
regarding one potential mechanism for understanding how therapist characteristics are associated with 1-year postdischarge youth outcomes through therapist stability. However, two separate aspects of the data used in the present investigation made testing for mediation infeasible. First, because of the data clustering (i.e., youth data nested within therapists), the effects required correction by the Huber–White method of using robust standard errors. The disadvantage of this requisite step was that the calculation of direct and indirect effects was not applicable. Second, the dependent variable (1-year postdischarge outcome) was a three-level ordinal outcome variable. A method for combining the slope and standard error of the regression coefficient in a logistic regression equation to test mediational effects has yet to be developed. Despite these limitations, the present study has strengths that allow it to make an important contribution to research in the area of youth and family therapy treatment outcomes. The study offers findings in some areas that have rarely—or in some cases never—been examined among therapists and the youth and families they serve. These under-investigated areas include the impact of therapist demographic characteristics and therapist stability on youth outcomes. The lower likelihood of having an undesirable youth outcome for a female therapist is worth further consideration, observation, and study as is the finding that for female therapists, higher levels of stability were significantly associated with the increased probability of a desirable outcome. We observed the opposite finding for the males. When therapist stability decreased, the likelihood of a desirable youth outcome increased. We consider this gender interaction counterintuitive because one would hypothesize that the relationship between stability and youth outcome is positively associated regardless of therapists’ gender. One possible explanation for this finding may relate to level of engagement, especially between caregiver and therapist. Some research has hinted that ethnic match between therapist and caregiver is related to treatment outcomes (Halliday-Boykins, Schoenwald, & Letourneau, 2005). Our findings suggest that gender match may also influence the caregiver's level of engagement with the therapeutic process. Given the preponderance of female primary caregivers, it is possible that male therapists face additional challenges in establishing an adequate level of engagement with caregivers to secure desirable outcomes. Further exploration of this issue is required before recommendations can be made regarding training and/or skill development for therapists. In addition, two other potential explanations for the counterintuitive stability finding for male therapists warrant further exploration. These explanations include the
potential clinical or administrative mechanisms that may account for therapist turnover, such as cases that are reassigned to a different therapist for clinical reasons. For example, after an adequate length of service when low engagement levels persist or there is a lack of progress on treatment goals, the case may be reassigned to a more experienced therapist who may be able to advance the progress with the client or family. In contrast, cases may also be reassigned for administrative reasons, such as when a therapist resigns or when a therapist accepts a promotion to a supervisory role, which necessitates a caseload reduction. Whereas most agencies assign clients to therapists based on an open slot in a caseload rather than other characteristics, administrative reassignment typically occurs on a random basis instead of a systematic basis. This study also brings to the forefront of discussion the important practice implications related to youth demographic characteristics. Although previous studies have confirmed the contribution of youth demographic characteristics and risk factors to treatment outcome (Barth et al., 2007a; Karver et al., 2006; Shirk & Karver, 2003; Yorgason et al., 2005), the present investigation does not support these variables as predictors of therapist stability. That is, the lack or presence of therapist stability across treatment duration is independent of the youth served. It appears that more troubled youth do not drive their therapists away or cause employment instability. Furthermore, this study demonstrated that a therapist’s level of education was significantly associated with employment stability. That is, therapists who held a master’s degree had a 5% greater likelihood of high stability than therapists who did not have this education level. Although this finding implied that a master’s degree predicts greater stability (i.e., less turnover), further investigation is needed before making prescriptive statements regarding hiring practices. Furthermore, because this model explained just 1% of the variance in the stability variable, including additional predictors in the model might provide a better understanding of individual therapists’ differences in stability. Usually, staff turnover is interpreted as a negative occurrence. Yet, under certain circumstances, including the introduction of a second therapist when there is a lack of progress or when a therapist is promoted, this phenomenon can be considered a positive development in relation to later treatment outcomes. Goerge (1994) also found a counterintuitive result when he showed that turnover rates among child welfare workers led to greater rates of family reunification, and concluded that apparently workers endeavored to resolve their cases before they moved on to new positions or roles. Future investigations could make an additional and important contribution to the study of
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009
250 Research on Social Work Practice
turnover in the children and youth services field by closely examining therapist stability using an approach that takes into account the level of experience of the therapist on a case-by-case basis to determine both the mechanisms of instability and its consequences to client outcomes. Such information holds the promise of advancing our understanding of the complexities associated with turnover in the helping professions, and subsequently improving social work practice in family-based treatment settings.
References American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. Barth, R. P., Greeson, J. K. P., Guo, S., Green, R. L., Hurley, S., & Sisson, J. (2007a). Changes in family functioning and child behavior following intensive in-home therapy. Children and Youth Services Review, 29, 988-1009. Barth, R. P., Greeson, J. K. P., Guo, S., Green, R. L., Hurley, S., & Sisson, J. (2007b). Outcomes of youth receiving intensive inhome therapy or residential care: A comparison using propensity scores. American Journal of Orthopsychiatry, 77, 497-505. Blankertz, L. E., & Robinson, S. E. (1997). Turnover intentions of community mental health service workers in psychosocial rehabilitation services. Community Mental Health Journal, 33, 517-529. Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M., et al. (1995). Multisystemic treatment of juvenile sexual offenders: A progress report. Journal of Consulting and Clinical Psychology, 63, 569-578. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155159. Curtis, N. M., Ronan, K. R., & Borduin, C. M. (2004). Multisystemic treatment: A meta-analysis of outcome studies. Journal of Family Psychology, 18, 411-419. Gallon, S. L., Gabriel, R. M., & Knudsen, J. R. W. (2003). The toughest job you’ll ever love: A Pacific Northwest Treatment Workforce Survey. Journal of Substance Abuse Treatment, 24, 183-196. Glisson, C. (2002). The organizational context of children’s mental health services. Clinical Child and Family Psychology Review, 5, 233-253. Glisson, C., Dukes, S., & Green, P. (2006). The effects of the ARC organizational intervention on caseworker turnover, climate, and culture in children’s service systems. Child Abuse & Neglect, 30, 855-880. Glisson, C., & James, L. R. (2002). The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior, 23, 767-794. Goerge, R. M. (1994). The effect of public child welfare workforce characteristics and turnover on discharge from foster care. In R. P. Barth, J. Duerr Berrick, & N. Gilbert (Eds.), Child welfare research review, Vol. 1. (pp. 205-217). New York: Columbia University Press. Goldfried, M. (1998). A comment on psychotherapy integration in the treatment of children. Journal of Clinical Child Psychology, 28, 49-53. Halliday-Boykins, C. A., Schoenwald, S. K., & Letourneau, E. J. (2005). Caregiver-therapist ethnic similarity predicts youth out-
comes from an empirically based treatment. Journal of Consulting and Clinical Psychology, 73, 808-818. Henggeler, S. W., Halliday-Boykins, C. A., Cunningham, P. B., Randall, J., Shapiro, S. B., & Chapman, J. E. (2006). Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments. Journal of Consulting and Clinical Psychology, 74, 42-54. Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821-833. Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using multisystemic therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60, 953-961. Huey, S. J., Jr., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in multisystemic therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68, 451-467. Jayaratne, S., & Chess, W. A. (1984). Job satisfaction, burnout, and turnover: A national study. Social Work, 10, 448-453. Johnson, L. N., Ketring, S. A., Rohacs, J., & Brewer, A. L. (2006). Attachment and the therapeutic alliance in family therapy. American Journal of Family Therapy, 34, 205-218. Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment literature. Clinical Psychology Review, 26, 50-65. Kazdin, A. E., Marciano, P. L., & Whitley, M. K. (2005). The therapeutic alliance in cognitive-behavioral treatment of children referred for oppositional, aggressive, and antisocial behavior. Journal of Consulting and Clinical Psychology, 73, 726-730. Knudsen, H. K., Johnson, J. A., & Roman, P. M. (2003). Retaining counseling staff at substance abuse treatment centers: Effects of management practices. Journal of Substance Abuse Treatment, 24, 129-135. Littell, J. H. (2005). Lessons from a systematic review of the effects of multisystemic therapy. Children and Youth Services Review, 27, 445-463. Nissly, J. A., Mor Barak, M. E., & Levin, A. (2005). Stress, social, support, and workers' intentions to leave their job in public child welfare. Administration in Social Work, 29(1), 79-100. Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71, 452-464. Shirk, S. R., & Saiz, C. (1992). The therapeutic alliance in child therapy: Clinical, empirical, and developmental perspectives. Development and Psychopathology, 4, 713-728. Sorensen, H. T., Sabroe, S., & Olsen, J. (1996). A framework for evaluation of secondary data sources for epidemiological research. International Journal of Epidemiology, 25, 435-442. StataCorp LP. (2007). Stata 9 Data Analysis and Statistical Software. College Station, TX: Author. Yorgason, J. B., McWey, L. M., & Felts, L. (2005). In-home family therapy: Indicators of success. Journal of Marital and Family Therapy, 31, 301-312.
Downloaded from http://rsw.sagepub.com at INDIANA UNIV KOKOMO LIBRARY on June 2, 2009