Many sources exist to help programs that are new to evaluation. If you plan to conduct your own evaluation, break it into doable parts: Start with what is possible now and, when you are ready, gradually increase your evaluation activities. It is better to evaluate a few things well - such as attendance or the number of teachers reporting better homework from program participants - rather than lots of things poorly. Conducting an internal program evaluation will enable your program staff to witness the benefits of engaging in evaluation and prepare them to participate fully should you choose or need to expand your evaluation or work with an external evaluator later on.

The key things to keep in mind if you are doing your own evaluation are the expertise of your staff and the rigor of your results. It is helpful to have someone on staff who understands evaluation methods and can make sense of the data you collect. This doesn't have to be someone with a doctoral degree in statistics, but it should be someone with more than a basic understanding of the purpose and methods (i.e., the why and the how) of evaluating programs. In addition, when conducting an evaluation internally, it is important to interpret your results cautiously. Remember, internal evaluation is best suited to formative evaluation and program improvement or, at most, an examination of trends and potential outcomes. All program leaders want to be able to announce definitively the impact of their programs on young people, but an internal evaluation (and even most external evaluations!) won't be able to do this. It is only at the most rigorous levels of external evaluation that you can confidently say that X caused Y to happen at your program.

A good way to start an internal evaluation is to focus not on impact but on program quality. Numerous self-assessment tools (see Tool 85: Program Observation Tools) are available online and free of charge. They are designed to help afterschool and expanded learning programs assess their strengths and weaknesses. Conducting a self-assessment with your staff can be a great way to identify strengths and determine where the program may need to focus additional professional development or program redesign efforts. Program quality assessment (observations using the tools noted earlier), youth satisfaction surveys, and participation data also are great internal evaluation activities that are relatively inexpensive and require little formal evaluation expertise but yield data that will help inform and improve your program.

Many different types of external evaluators exist, including individuals based at a university, private evaluation consultants, and evaluation firms or organizations. When considering an external or third-party evaluator, two of the most important things to consider are their expertise and familiarity with your type of program and your budget for evaluation.

When possible, it is best to work with an evaluator who is familiar with evaluating programs of your size and scope. Too often, programs hire an outside evaluator who has never worked with staff members or participants of an afterschool and expanded learning program; without the appropriate experience, an evaluator may design an evaluation that is not suited to the nature and goals of your program.

Fortunately, a variety of existing tools and protocols have been designed and validated for use in afterschool and expanded learning settings. Ask potential external evaluators what tools they would recommend and why. Become familiar with some of the existing tools so that you can recommend them to your potential evaluator or identify an evaluator with experience using them. You do not want to spend valuable resources paying evaluators to create new tools when very good ones exist at little or no cost. Finally, check references and request work samples to help you figure out how well the evaluator’s work fits with your program’s philosophy and evaluation needs.

Budget is another important factor when considering an outside evaluator. The cost of an evaluation can range quite dramatically, depending on a variety of factors, including whether you use an individual or a firm or organization, the complexity of your evaluation design, and how you plan to use your findings. If you determine that it is essential to have unbiased, outside evaluation results but have a small budget, you may want to consider cost-saving strategies, such as finding a graduate student or other individual rather than hiring a larger evaluation firm or doing your own data collection (e.g., distributing surveys and gathering youth-level data from the district) rather than having the evaluator do the legwork. Perhaps you can phase in your evaluation across multiple years to ensure you have formative data at multiple points and one final summative report at the end. Another strategy is to partner with other programs in your area or district to participate in a multisite evaluation because this allows programs to share the costs of the evaluation effort. If possible, work with school or district staff members (if you are working with or in a district) because you might have access to people who are experienced in conducting evaluations. Such people are familiar with school data sources and may know some or all the staff members working in your program. If an individual is already working with the school or district, he or she may be less expensive than hiring someone without those connections.

When working with an external evaluator, plan on allocating at least 10 percent of your budget to the evaluator. Begin by interviewing the evaluator (see Tool 86: External Evaluator Questions and Worksheet for guiding questions you can ask during the interview and selection process), select the evaluator early (ideally before programming begins), and work to build an ongoing relationship. The better the evaluator knows you and your program, the more the evaluator can help you answer the questions you want answered. Likewise, you should spend considerable time, very early and often, working with the evaluator and the rest of your planning team to develop an evaluation plan. Provide the evaluator with copies of grant proposals, your theory of change or logic model, your goals, and other important planning documents. Spell out clearly and in writing the tasks the evaluator will do and the tasks that will be done by program staff members, including reporting requirements and data collection. Require that the evaluator spend some time on-site attending meetings, observing programs, and becoming familiar with your program’s staff. Ultimately, you should view the evaluator as your ally. His or her input can help your program operate more effectively and demonstrate the value of your efforts to others.