No matter how well you plan your activities, your evaluation questions, and your evaluation plan, it will all be for naught if you don’t have a useful way to collect the data you need to answer your evaluation questions. Some indicators are easier to measure than others. For example, as long as you can get access to academic records from the school or district, you will be able to easily measure grades, attendance in school, and test scores. Other indicators are more cumbersome and labor intensive to measure. For example, determining whether a young person has increased motivation, is more engaged in school, or is better able to solve problems requires a youth survey of some kind, which can be difficult to implement.
Keep the relative complexity of data collection in mind as you finalize your indicators, select data sources, and develop your data collection plan. In most afterschool and expanded learning programs, staff members will have to participate in data collection. As a result, selecting strategies that do not overburden your staff is a good idea. If you know in advance that you need to collect hard-to-find data, it is good to make those arrangements early. Each evaluation question and associated indicator for which you intend to track data should be matched to a good data source.
Many strategies, tips, and tricks can be used to ensure that data collection activities are successful. When participants enroll in your program, for example, make sure you get signed consent from family members to collect information about their children, particularly to conduct surveys and access school records. Having permission from the outset helps simplify data collection requests you may need to make to the school or other personnel in the future. Emphasize that the information will be aggregated and used only for program improvement, not to identify individual participants. Furthermore, collect only the data you will need. For example, if you only want to know age and grade, do not ask other demographic questions. Likewise, ask only for data you do not already have. For example, if your program is based at a school, you may be able to get all the demographic information you need from the main office. Ask your school about its policies on sharing certain kinds of data.
It also is important to know the languages spoken in the homes of your participants and consider translating permission and consent forms, family surveys, and other evaluation materials into the languages most commonly spoken by participants’ families. Make sure that you are aware of cultural norms for the population you are serving and be respectful of roles or positions, particularly with youth and family surveys and focus groups. Try not to use youth as interpreters for their families because this puts them in a potentially uncomfortable role. A bilingual staff member or translator can be more effective.
Assign each participant an identification number or use a school-assigned identification number (if one exists) and keep data filed by that number. This practice reduces the chance of mixing up two youth with similar names or birthdays, allows you to make surveys and other documents anonymous, and is useful when it comes to data analysis. Be sure that you have all the data protections in place. See the management section for guidance on policies that support the data rights of participants. Finally, be sure to collect all your information at one time. For example, the easiest time to collect demographic data is when young people enroll.
You can collect data in many different ways. Whichever way you choose, it is important that you collect data in a systematic and reliable manner—that is, collect data the same way each time; use existing, tested surveys and tools when possible, rather than creating your own; and create appropriate conditions for data collection. Some of the more common forms of data collection include attendance or participation tracking; surveys (see Tool 87: Tips for Developing and Administering Surveys); focus groups (see Tool 88: Tips for Running Good Focus Groups); program observations (see Tool 89: Tips for Conducting Program Observations); interviews (see Tool 90: Tips for Conducting Good Interviews); collecting demographic information about participants in the program; collecting information reports from schools, the district, the police department, or other entities that collect relevant information (e.g., grades, test scores, comparisons of crime statistics, or detention reports); and other documents (newsletters, meeting minutes, and other sources of information).