Feb. 1 , 2007
National Coalition Institute's Research into Action

Going Beyond Program Implementation–Issues to Consider When Institutionalizing an Evidence-Based Prevention Program

Is it possible for a community to implement an evidence-based program with the same outcomes? That is the question researchers asked in a study published recently in Prevention Science.

The study looked at Early Risers “Skills for Success,” an evidence-based prevention program targeting conduct problems. In previous studies, program developers had demonstrated the program’s ability to improve academic achievement and decrease conduct problem behaviors when implemented in a real world setting. However, these results were achieved in settings in which the program developers provided considerable levels of support, guidance and supervision to the nonprofit community agency running the prevention program.

The recent study examined how well the community agency could replicate the outcomes from the previous study but this time without as much direct support from the program developers. While the study demonstrated that the community could implement the program with fidelity, there was less participant attendance in the program and only one program outcome was replicated.

The primary reason that the community agency could not replicate previously achieved results is that the agency struggled with engaging families at recommended levels. This is a dosage issue—in order for families to benefit from the Early Risers program, they need to participate at

certain levels. This was achieved when the program developers were involved in the implementation of the program; however, when the community agency took ownership of the program, participation levels decreased.

Several factors explain the reduced levels of program participation. One critical reason was transportation. In the first study, the school district contracted out the service to provide children transportation to the Early Risers’ Summer and After-School Programs. Due to budget cuts, this service was suspended, and parents cited the lack of reliable transportation as the main reason for sporadic attendance in the program.

Another reason was problems in collaboration among the project’s primary stakeholders. In the first study, the program developers facilitated the collaboration for the project but this collaboration was not sustained in the current study.

A third factor reason involved staff turnover. Many families experienced changes with the family advocates assigned to them, likely affecting their participation in the program.

Source: August, G.J., Bloomquist, M.L., Lee, S.S., Realmuto, G.M., & Hektner, J.M. (2006). Can evidence-based prevention programs be sustained in community practice settings? The Early Risers’ advanced-stage effectiveness trial. Prevention Science, 7, 151-165.
What Coalitions Can Do www.coalitioninstitute.org
Create a context for change. The coalition is the vehicle for collaboration and effective prevention programming in the community. A coalition can address most of the issues that impacted program outcomes and participation rates in the above study. For example, if transportation is a real issue for a target population being able to access the program, then the coalition can work with the community and service providing agency on how to address gaps in transportation.
Plan for program sustainability at the organizational level. Sustainability at the organizational level refers to the degree to which an intervention becomes institutionalized in organization practice. The coalition needs to work with service agencies/organizations to develop a plan that addresses the factors that influence program implementation and sustainability.
Attune to collaboration among program stakeholders. As coalitions collaborate at the community level, a degree of collaboration needs to occur among stakeholders at the programmatic level. This includes agency administrators, the program staff, and individuals receiving the service.
Be an informed consumer. When determining the appropriateness of an evidence-based program, policy or practice, ask the program developers questions about their study findings. Did the developer attune to the sustainability of the program, policy or practice in a community health setting? How involved were the program developers in supervising, supporting, guiding the community in studies of program effectiveness? Ask other coalitions about their experience using a program and use their lessons learned to tailor your program.
CADCA's National Coalition Institute, 625 Slaters Lane, #300, Alexandria, VA 22314, 1-800-54-CADCA
Research into Action may be copied without permission. Please cite CADCA's National Coalition Institute as the source.
CADCA's National Coalition Institute is federally-funded through a grant from the Office of National Drug Control Policy that is administered by SAMHSA's Center for Substance Abuse Prevention. The Institute's mission is to increase the effectiveness of community anti-drug coalitions throughout the nation.