Program Evaluation

The Early Risers “Skills for Success” program has evolved over a period of eight years, informed by results from efficacy and small-scale effectiveness trials.  The intervention was initially validated in rural school settings with high-risk Caucasian children.  More recently, Early Risers has been studied in effectiveness trials conducted in collaboration with a network of neighborhood centers serving low-income inner-city families.  An earlier and less structured version of the PEER component, referred to as the “buddy system”, has been studied in a naturalistic context.  Results from the efficacy and effectiveness trials are summarized below.

Efficacy Study [Phase 1]

Validation of the Early Risers program has been ongoing in a longitudinal, randomized, controlled trial that began in 1997.  Twenty rural schools were matched on sociodemographic variables and randomized to intervention (n=10) and natural progression control (n=10) conditions.  All kindergarten children (n=1840) enrolled in these 20 schools were screened by teachers for early aggression.  Children who scored in the upper 15% on the Aggression scale of the CBCL-TRS (Achenbach, 1991) were enrolled in the study (124 intervention, 121 control).  The intervention was initiated in the summer following the kindergarten year and ran continuously for 5 years.  A panel of proximal outcome measures assessing child and parent competencies were administered annually.  An evaluation conducted following the first two years of the trial indicated that program children made significantly greater gains in academic achievement and classroom behaviors than did controls.  Only the most severely aggressive children showed improvement in self-regulated behaviors (e.g., reduced aggressiveness, hyperactivity, and impulsivity (August, Realmuto, Hektner, & Bloomquist, 2001).  These effects were maintained following a third year of intensive intervention and complemented by gains in various dimensions of social competence (August, Hektner, Egan, Realmuto, & Bloomquist, 2002).  In addition, parents of program children reported more improvement in parenting practices than their control counterparts.  Dosage analyses replicated the result of intent-to-intervene analyses.  Larger effect sizes were found for participants who achieved higher levels of exposure to the intervention as measured in terms of rates of attendance. 

Efficacy Study [Phase 2]

At the 4-year evaluation, child participants had completed 3 years of intensive intervention and one “booster” year.  At this time point, we sought to find evidence for generalization of program effects by administering peer assessment instruments to classmates of Early Risers’ participants in the natural classroom setting.  More specifically, sociometric assessment was used to evaluate whether the program had modified the reputations, acceptance/rejection, and quality of friendship selection of program participants control to controls.  Relative to controls, program children were viewed by their classmates as higher in leadership and social etiquette and selected friends who were lower in aggression (Augst, Egan, Realmuto, & Hektner, 2003).  Self-reported quality of friendship also differed between groups, with program children reporting more companionship and recreation, program girls reporting more validation and caring, and severely aggressive program children reporting less aggression toward others than their control counterparts. 

Efficacy Study [Phase 3]

One year following the completion of the 5-year intervention trial, a 6-year evaluation was conducted to assess program effects on to-be-prevented outcomes.  At this time, participants were completing their first year in middle school (6th grade).  Follow-up data were collected for 73 program youth and 78 control youth.  No differential attrition effects were observed.  Structured mental health interviews (DISC-IV) were administered to both parent and youth.  Group comparisons were performed on number of oppositional and defiant disorder (ODD) problems endorsed and number of conduct disorder (CD) problems endorsed.  Based on parent report, program youth had significantly less ODD problems (5.72 out of possible 12) than did control youth (6.85 out of 12; p<.05).  No difference was observed for number of conduct problems.  Similarly for youth report, program youth reported less ODD symptoms (4.07 out of 12) than did control youth (8.40 out 5.60; p<.05).  Once again rates of conduct problems were not different.  Drug use outcomes, including tobacco, alcohol, marijuana, and other drugs, were also assessed.  As might be expected in this 6th grade sample, rates of past year use and frequency of use were very low and not significantly different between groups.

Effectiveness Study

Beyond efficacy, the next step was to determine whether program effects would be replicated when the program was implemented by community providers under real-world options for implementation.  A local community agency serving inner-city, low-income families was interested in adopting the program and served as the host site.  Characteristics common to both the efficacy study and this effectiveness study were (a) enrollment of children screened for aggressive behavior, (b) random assignment of recruited participants to program or control conditions, (c) provision of training, program manuals, and technical assistance to the implementers, (d) provision of coaching, supervision, and fidelity monitoring by the program developers, and (e) outcome assessments conducted by trained technicians.  This trial differed from the efficacy study in serving African American families, utilizing a community service setting as the provider system, employing community practitioners rather than experienced professionals, and permitting intervention adaptations (with approval from program developers) to accommodate the needs of the community agency.  Program effects were initially evaluated following a two-year intervention trial with intent-to-intervene analyses.  Results replicated those of the efficacy trial.  Program children made significant gains in social competence and school adjustment, and only the most severely aggressive children showed reductions in externalizing behavior problems (August, Lee, Bloomquist, Realmuto, & Hektner, 2003).