Measuring Outcomes & Evaluation

Performance Level 1

The program goals and intended outcomes are unclear, or the relationship between the program goals and intended outcomes and the organizational mission, vision, and identified needs is unclear. As a result, the goals are not measurable. Staff members are unaware of the mission and the program goals.

Performance Level 2

Broad program goals and intended outcomes generally relate to the organization’s mission. Goals are vague and difficult to measure. The relationship of the goals and intended outcomes to identified needs is vague or unknown. Staff members are aware of the mission, vision, goals, and intended outcomes of program, but are not clear how their work contributes to meeting them.

Performance Level 3

Program goals and intended outcomes are developed based on the needs of participants. Goals and intended outcomes are specific, measurable, and aligned with and support the organization’s mission. Staff members meet to discuss the goals and intended outcomes of the program and to ensure the program activities work towards meeting the goals and intended outcomes.

Performance Level 4

Program goals and intended outcomes are developed based on the identified strengths and needs of program participants. Goals and intended outcomes are specific, measurable, achievable, relevant, and time-bound (SMART), and are clearly aligned with the organization’s mission. Activities are designed to support both short- and long-term goals. Staff members, participants, families, and other stakeholders are actively engaged in developing, assessing, and evaluating goals and intended outcomes.

Performance Level 1

Program evaluation may occur but is not a planned process. Instead, evaluations are informal and occur irregularly. Evaluations are not consistent over time. Therefore, there is no comparable data for program stakeholders to review.

Performance Level 2

The site director develops a plan for program evaluation. The plan includes collecting only qualitative or quantitative data. The collection methods used are informal, and the site director is solely responsible for gathering data.

Performance Level 3

The site director develops a plan for program evaluation with input from staff and stakeholders. The plan includes collecting both qualitative and quantitative data, and includes all data necessary to report to funders, parents, and other stakeholders. The site director and other staff members use surveys and observations to gather data. The evaluation plan includes a system for using evaluation results, which includes reviewing results prior to and during program planning and while shaping management and operational practices.

Performance Level 4

The site director partners with staff members and stakeholders to develop a plan for ongoing program evaluation. The plan includes collecting both qualitative and quantitative data, and includes all data necessary to report to funders, parents, and other stakeholders, as well as youth development outcomes, academic and cognitive 1development outcomes, and observable and non-observable aspects of program management and operations. The site director and other staff members use surveys, observations, self-assessment, and other means to gather data, which is then stored in electronic and paper files. The evaluation plan includes a system for using evaluation results to improve the program and inform program decisions.

Performance Level 1

Participants’ progress is assessed informally through anecdotal information from participants and/or program staff. Data is rarely recorded.

Performance Level 2

The program has a participant self-report method for measuring participants’ progress. Data is captured,but accuracy is unknown. Staff members receive anecdotal information verbally from participants, but it is not always recorded.

Performance Level 3

The program measures participants’ progress in a few ways, which may include participant self-report, staff observation, pre-year and post-year surveys, parent surveys, and other tools. Assessment is integrated into the program and informs the development of future activities. Participants and families are informed regularly of their progress. Staff members record anecdotal information received verbally.

Performance Level 4

The program measures each participant’s progress in a variety of ways, including participant self-report, staff observation, pre-year and post-year surveys, and parent surveys. The program is in regular communication with the school and families about the participant’s progress, in accordance with FERPA. Assessment is integrated into the program and informs the development of future activities. Participants and families are informed regularly of their progress. Staff members also ask families and other stakeholders to submit written anecdotal information, which is kept in participants’ files.

Performance Level 1

The site director and staff members do not meet to discuss their work and do not participate in professional development events,so they are unaware of what practices are effective.

Performance Level 2

The site director and staff members meet occasionally to plan and discuss the program activities. Promising practices and related information are shared informally and irregularly. The site director and staff members occasionally participate in professional development events.

Performance Level 3

The site director and staff members meet regularly to discuss their program activities and track promising practices through writing successful curricula and activity guides so that the promising practices can be replicated. Staff members share these practices among themselves and occasionally with colleagues from other sites. The site director and staff members regularly participate in professional development events.

Performance Level 4

The site director and staff members monitor and track promising practices through writing up curricula and activity guides. Staff members regularly share these practices at staff meetings. Staff members also share their promising practices with colleagues from other sites through meetings, listservs, and at conferences. The site director and staff members regularly and frequently participate in professional development events

Performance Level 1

The site director does not alert program stakeholders when an evaluation is conducted. The evaluation summary and related data are not made available.

Performance Level 2

The site director alerts some program stakeholders when an evaluation is conducted through informal conversations. The evaluation summary only is available upon request.

Performance Level 3

The site director alerts all program stakeholders when an evaluation is conducted through a formal method of communication, such as an e-mail or newsletter. The evaluation summary and related data are posted,and copies are available upon request.

Performance Level 4

The site director alerts all program stakeholders when an evaluation is conducted through multiple formal methods of communication, such as e-mail, meeting minutes, and newsletters. The evaluation summary findings and related data, including both strengths and challenges, are communicated. The entire evaluation or an executive summary is clearly posted and copies are distributed to all participants, families, partner organizations, members of the Board of Directors, local principals, and other stakeholders.

Performance Level 1

Occasional feedback is received through informal conversations with participants, families, and other stakeholders to assess program activities.

Performance Level 2

The site director has created or located an internal method for assessing program activities. The method uses one type of assessment (i.e. surveys) and is implemented irregularly. Only the site director reviews the information collected. Sometimes the information is used to inform modifications in program design.

Performance Level 3

The site director, with input from staff members, has created or located an internal method for assessing program activities. The method uses several types of assessment (i.e. surveys, focus groups, verbal feedback, etc.) and is implemented regularly. The site director always reviews the information collected, and staff members are encouraged to review the information as well. The information is used to inform modifications in program design.

Performance Level 4

The site director, in collaboration with other staff members, participants, and other program stakeholders, has created or collaboratively decided upon an internal method for regularly assessing program activities. The method uses several types of assessment (i.e. surveys, focus groups, verbal feedback, etc.) and is implemented regularly. The site director, staff members, and participants always review the information collected. The information is used to inform regular modifications in program design and delivery. All information collected is stored in paper and electronic files to enable the site director, staff members, and participants to review program progress over time.

Performance Level 1

The site director occasionally observes staff members’ performance and gives them verbal feedback.

Performance Level 2

The site director has a simple internal method for assessing staff performance. The method uses a one-way assessment (i.e. observation) and is implemented irregularly. The site director only shares the information collected verbally with staff members. Sometimes the information is used to inform modifications in program management and operations.

Performance Level 3

The site director, with input from staff members, has created an internal method for assessing staff performance. The method uses both one-way assessment (i.e. external observation) and two-way assessment (i.e. self-assessment) and is implemented regularly. The site director shares the information collected with staff members. The information is used to inform staff members’ goals for the coming year and to collect suggestions to determine professional development opportunities.

Performance Level 4

The site director, in collaboration with other staff and program stakeholders, has created an internal method for assessing staff performance. The method uses several types of assessment (i.e. observation, self-assessment, etc.) and is implemented regularly. The site director shares the information collected with staff members, and asks them to reflect on their own performance. The information is used to inform staff members’ goals for the coming year and to collect suggestions to determine professional development opportunities. If a staff member receives a negative review, a corrective action plan is developed. All information collected is stored in paper and electronic files to enable the site director to review program progress over time.

Performance Level 1
The program requires medical forms. No tracking is done to ensure completed records of participants are received. Forms that are submitted are kept on file but rarely used. Therefore, staff members are not always aware of the special health needs of participants.

Performance Level 2
The program requires medical forms and tracking is done to ensure all forms are received. Forms are kept on file and are reviewed if there is a medical concern or emergency. No review of forms is done to make the staff aware of special needs. Staff members may only become aware of the issue during an emergency that prompts them to review a participant’s form.

Performance Level 3
The program requires medical forms and receives them from each participant. Forms are reviewed by staff members and special health needs are flagged; forms are then kept on file. Staff members are informed of relevant special health needs of participants, such as food allergies, at the beginning of each year. Adjustments are made to the program design as necessary based on participants’ health needs. Any information shared with staff members is done so in consideration of confidentiality rules.

Performance Level 4
The program requires medical forms and receives them from each participant. Forms are reviewed by staff members and by a nurse or health specialist and special health needs are flagged; forms are then kept on file. Staff members are informed of relevant special health needs of participants, such as food allergies, at the beginning of each year, and again in the middle of the year. Adjustments are made to the program design as necessary based on participants’ health needs. The site director or other staff members maintains relationships with school nurses to receive updates on participants’ health needs as they change. Any information shared with staff members is done so in consideration of confidentiality rules.

Performance Level 1

Feedback from program stakeholders, such as participants, staff members, families, and community leaders, is not included in program evaluation. They are not involved in the evaluation process.

Performance Level 2

Feedback from a few program stakeholders, such as participants and staff members, is included in program evaluation on an ad hoc basis, if they volunteer to speak with the site director or evaluator.

Performance Level 3

Feedback from several program stakeholders, such as participants, staff members, families, and community leaders, is included in program evaluation. Stakeholders are invited to be involved in the evaluation process, and are given the opportunity to speak with the staff leading the evaluation or the evaluator. There is a section in the evaluation dedicated to stakeholder feedback.

Performance Level 4

Feedback from all program stakeholders, including participants, staff members, families, and community leaders, is a critical component in program evaluation and is collected on an ongoing basis. Multiple ways to include stakeholder feedback are included as a part of the evaluation design. Stakeholders have the opportunity to communicate directly with the staff members leading the evaluation and/or the evaluator, and their feedback is embedded throughout the evaluation. Stakeholders also have multiple opportunities throughout the year to review and provide feedback on progress evaluations.

Performance Level 1

Evaluation is conducted on an infrequent basis or not at all. When evaluation is conducted, the site director doesn’t share the findings with staff members. The site director rarely considers the evaluation findings when designing program activities and policies.

Performance Level 2

Evaluation is conducted on an infrequent and/or irregular basis. The site director shares findings with staff members who ask to see them. The site director sometimes reviews the evaluation findings before designing program activities and policies.

Performance Level 3

Evaluation is conducted regularly. The site director shares findings with staff members and program stakeholders. The site director always reviews evaluation findings before designing program activities and policies. The evaluation findings are reflected in changes made to the program design.

Performance Level 4

Evaluation is ongoing and evaluative feedback is collected throughout the year. The staff and program stakeholders are involved in all stages of the process. The site director shares findings and feedback with other staff members and program stakeholders. They discuss and brainstorm ways in which to make improvements to the program throughout the year. The site director and staff members always use the evaluation findings to design program activities and policies. The evaluation findings drive the changes made to the program design.