Before you develop evaluation questions, set up a data collection system, or decide who will conduct the evaluation, determine the purpose of your evaluation. Every other decision is tied to this one. To start, your team should consider what you want to learn about the program and participants. Determine whether your goal is to gather data that will help you improve your program or to gather data so that you can report on the effectiveness of your program to funders, families, partners, schools, and other key stakeholders (or a combination of stakeholders). Also bear in mind what your funders require you to report or if you are obligated to conduct specific types of evaluation.

An understanding of the purpose of your evaluation will help you determine whether you want to do a formative or a summative evaluation—or a combination of the two. If you are early in your program development and have minimal requirements from your funder, you may want to focus on doing a formative evaluation to make sure you are offering the quality of programming you had intended. For instance, you might gather data about how much young people like your program, how often they are coming, whether families are satisfied, and how you can improve what you are offering. As your program develops and you begin to have higher levels of funding with higher stakes, you may be asked to report on whether youth in your program are achieving certain outcomes. At that point, you may want to focus on conducting a more rigorous summative evaluation that explores the impact your program may be having. An impact evaluation is a form of summative evaluation that uses an experimental or quasi-experimental design (i.e., compares participants to nonparticipants) to show that a program caused the outcomes of interest. Impact evaluations are very expensive and should be conducted by trained external evaluators. We recommend careful consideration and extensive formative evaluation focused on program improvement before your program invests in an impact evaluation. It is important to recognize that programs cannot “prove” that they benefit the young people they serve unless they have designed their evaluation using impact evaluation methods. Newer programs, or those with limited time and resources, may want to stick to a more formative evaluation focused on program improvement or look at outcomes to assess trends and begin to explore whether the program may be having an effect on certain outcomes. Use Tool 83: Determining Your Evaluation Purpose to help you determine what is best for your program.

In the section on program design, we discuss how to develop your program’s logic model and theory of change. The logic model is your roadmap for how you and your planning team think your program will reach its ultimate goals. The theory of change is the process you go through to say, “We think that if we do X, then Y will happen.” For evaluation planning, the logic model is the place to start. So, if you haven’t done this yet, visit the program design section and work through it now.