Decision Making
Decision making in research is more authoritative than in evaluation, usually driven by the researcher. In coalition evaluation the decision making process is more collaborative, involving all stakeholders.

Setting Standards
Traditional research sets standards called internal and external validity. Internal validity means changes occurred due to a specific intervention, that there is a cause and effect relationship between these two factors and no other factor can account for this relationship. External validity means that the results will be seen outside of the research setting. The goal of research is to repeat the intervention in various settings with the same results. The standards for coalition evaluation are on its usefulness, feasibility, appropriateness and accuracy. Coalitions are typically most concerned about what is going on in their community – does it work in our community context. For this reason, evaluation standards are set on usefulness of evaluation information, the feasibility to evaluate certain efforts, the appropriateness of evaluation questions and the accuracy of evaluation results.

Framing Questions
Research seeks to discover facts, so the types of questions asked in research are often about describing situations, finding associations between variables and determining cause and effect. In contract, coalition evaluations focus on values, including the quality, worth and importance of a given coalition initiative or the coalition as a whole.

Constructing Knowledge
Knowledge construction in conventional research focuses on isolating changes and controlling circumstances. This usually means that control/comparison groups are needed to determine if there are differences between a group that receives an intervention and a group that does not receive the intervention. Additionally, context is something that researchers need to control for---researchers seek to prove the effectiveness of an intervention regardless of context. Evaluation, on the other hand, values the context. This provides valuable information about the community and the circumstances in which coalition efforts can be successful. Additionally, coalition evaluation value flexibility and improvement so that the greatest understanding of the community context is captured.

Collecting Evidence
Research uses a limited number of sources of information, concentrating on using sources that are extremely accurate. On the other hand, coalition evaluation usually uses multiple sources of information to triangulate findings. Since one measure is not enough to capture all that a coalition does, several sources of information are used to fully tell a coalition story. For example, student surveys, fatal alcohol crashes and emergency room admissions, parent focus groups, and key stakeholder interviews can all be used to see where data line up and to explain what is going on in the community.

Analyzing & Synthesizing
Analysis and synthesis of data usually occurs once at the end of the research project. Evaluation of coalition efforts requires more ongoing analysis and data synthesis. As taught in many Coalition Institute trainings by Paul Evensen, evaluation should be “less of an autopsy and more of a regular doctor’s visit.”

Making Judgments
Judgment making in research is based on the data. This process should be logical and free of values. However, in coalition evaluation, this process relies on judging information according to community values. What is critical is that all values are understood and acknowledged.

Justifying Conclusions
Research’s objective is to conclude that one factor can be attributed to another---a direct cause and effect relationship. Research has various methods to make causal attributions, such as establishing a time sequence, controlling for other variables that might explain the change, and replicating findings. Trying to prove causation or attribution in a community context with an infinite number of factors present is impossible. Luckily, coalitions are not held to the standard of attribution. Coalitions must demonstrate that their efforts have CONTRIBUTED to changes in substance abuse. It’s unlikely that the coalition is the only factor impacting substance abuse. For example, the smuggling of illegal drugs into the country, the development of a new federal drug policy and the development of a new designer drug are out of the direct control of coalitions. However, since these are factors that can also affect changes substance abuse rates, the coalition should take care to acknowledge that these outside conditions can be alternative explanations for increases and decreases in substance abuse rates.

Using New Knowledge
In both research and evaluation, information from the study is disseminated to interested audiences. However, evaluation also focuses on how information is to be used and the users of the information so that evaluation findings can be tailored specifically. Additionally, one of the major intents of the knowledge generated by coalition evaluation is to build capacity by improving and strengthening coalition efforts and functioning.

This article is based on information created by Bobby Milstein of the Syndemics Network at the Centers for Disease Control and Prevention (CDC). Additional information is available in the chart below or on the Syndemics Prevention Network section of the CDC's Web site. Please contact Evelyn Yang, Evaluation and Research Manager, at eyang@cadca.org. with your questions.