October/November 2007
National Coalition Institute's Research into Action

Evidence-Based Program Effects May Be Delayed

Implementation of evidence-based programs by community coalitions help support the reduction of youth substance abuse and the effects of those programs may be delayed, according to a study published recently in Substance Use & Misuse.

The study examined the effects of community coalitions that implemented evidence-based substance abuse prevention programs as part of a State Incentive Grant (SIG) in Kentucky. Nineteen coalitions were studied in the Kentucky Incentives for Prevention Project (KIP). Each coalition implemented one to four universal programs.

The researchers compared 8th and 10th graders in schools in communities implementing KIP and in non-KIP schools. These schools were matched based on school size and percentage of students living in urban areas. Student survey data were analyzed in 1999 and 2002 in KIP schools and 2000 and 2002 in non-KIP schools.

Results and Discussion
The study examined 3 key questions:
1. Do coalitions that implement evidence-based programs reduce substance abuse among adolescents? Researchers looked at short-term effects on 8th graders that would have more recently received the prevention program and more sustained effects on 10th graders that would have received the program in the past. Results indicate that the 8th graders did not experience decreases in substance use.

However, the picture is different when looking at sustained effects on 10th graders who experienced slightly lower use of cigarettes and alcohol and rates of binge drinking. 10th graders in non-KIP schools also experienced decreases in cigarette and alcohol use but these decreases were smaller than KIP-schools. Additionally, non-KIP 10th graders experienced an increase in binge drinking. The more positive effects on 10th graders suggest that effects of evidence-based prevention programs may be delayed.

2. What risk and protective factors explain the relationship between implementing evidence-based prevention programs and youth substance use? The researchers also examined the role risk and protective factors play in explaining why the implementation of evidence-based prevention program(s) impacts youth substance use. They found that two factors—friends’ drug use and perceived availability—seemed to play a role in explaining the relationship

between use of evidence-based programs and reductions in 10th graders’ binge drinking and alcohol and cigarette use. Much more work needs to be done to explain the relationship between risk and protective factors and evidence-based prevention programs in youth substance use outcomes.

3. Does implementing more than one evidence-based program have a greater impact in reducing youth substance use? Schools were divided into those that implemented less than two evidence-based programs and two or more evidence-based programs. It appears that the number of evidence-based programs implemented does not have an impact on youth substance use. However, the researchers do state that there is evidence that KIP communities did not implement programs with the highest levels of fidelity; so that might explain why the number did not matter. Also, researchers did not compare the different types of combinations of programs implemented in each community and whether this made a difference.

What does this all mean?
The prevention and coalition field is in somewhat of a debate between the use of individually focused prevention-programs versus broad, environmental approaches such as changing policies, systems and community practices. This article provides some support for the use of evidence-based programs. However, the study has limitations—it only includes school survey data and does not report on other local data such as emergency room, arrest and school conduct/behavior records; does not describe what coalition-related activities the non-KIP communities were engaged in and does not describe which combination of evidence-based programs resulted in greater or lesser success, etc. It does not appear that only implementing evidence-based programs will result in communitywide changes in substance abuse outcomes. Perhaps the communities would have found stronger reductions in youth substance abuse rates if they had implemented additional interventions that targeted environment and community systems. While more research is needed, the study suggests that evidence-based prevention programs are one tool of many that coalitions should have in their intervention arsenals.

Source: Collins, D., Johnson, K., & Becker, B.J. (2007). A meta-analysis of direct and mediating effects of community coalitions that implemented science-based substance abuse prevention interventions. Substance Use & Misuse, 42, 985-1007.
What Coalitions Can Do www.coalitioninstitute.org
" Use local data to understand your community. Risk and protective factors differ by community, so it's important to know the key factors in your community contributing to your substance abuse problems. You need to get down to local conditions that explain what is going on in your community.
" Implement evidence-based programs, policies and practices as part of your comprehensive community strategy. While this study indicates that implementing at least one evidence-based program can have a positive impact in your community; keep in mind that the reductions were only experienced by 10th graders, sustained only a couple years after participating in a prevention program, and very small in size. Effective coalitions implement a wide array of programs, policies and practices to bring about communitywide reductions in substance abuse outcomes.
" Implement broad, comprehensive approaches instead of one-time interventions. Prevention should be more than a one-shot intervention. This study suggests a delayed effect of the impact of prevention programs; however, more research is needed to explain why this may have occurred. Coalitions should not assume that the effects of discreet prevention programs will be sustained over the next few years or even the lifetime of the individual. Prevention should be broad, comprehensive, age appropriate and also focus on changing the environment. Changing the environment and community systems supports prevention lessons/messages received in individual, school and family-focused interventions. Coalitions should think how various prevention interventions (programs, policies and practices) complement and reinforce each other to collectively contribute toward communitywide decreases in substance use.
CADCA's National Coalition Institute, 625 Slaters Lane, #300, Alexandria, VA 22314, 1-800-54-CADCA
Research into Action may be copied without permission. Please cite CADCA's National Coalition Institute as the source.
CADCA's National Coalition Institute is federally-funded through a contract with the Office of National Drug Control Policy. The Institute's mission is to increase the effectiveness of community anti-drug coalitions throughout the nation.