Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Impact Evaluation Seminar
Transcript of Impact Evaluation Seminar
Evaluation What is Impact Evaluation? A form of evaluation that
judges the worth of a program based on cumulative findings
can be used to make decisions about the future - programs should be tested before being expanded or spread to new locations
refers to both immediate and long term improvements
focuses on delivery and/or outcomes of a program
More than any other Form of evaluation we have studied, the findings from Impact Evaluation can be used more broadly to contribute to providing a generalized 'funded knowledge' to be used towards multiple programs, rather than one specific program Timing of Impact Evaluation Usually occurs when a program is in a mature or settled stage and has had sufficient time to have an effect
Some program evaluators use Impact Evaluation prior to having time to become an established program. This can create negative consequences as the program may be terminated, rather than being improved due to the evaluation displaying poor results of the worth of the program Impact Evaluation and Accountability Impact Evaluation and accountability are connected
Public has the right to expect funds to be effectively allocated and translated into efficient social or educational interventions
Citizens at large should know whether the program, either a program funded by government or one they have interest in, is making a difference - meaning are the goals and needs being met that stakeholders expect Major Concern of Impact Evaluation Outcomes are a major concern of Impact Evaluation Definition -
"An outcome is the effect [and product] of program activities on target audiences or populations, such as a change in knowledge, attitudes, beliefs, skills, access to service, policies, and environmental conditions" (Owen, p. 253) Typical Issues of Impact Evaluation Has the program been implemented as planned?
Have the stated goals of the program been achieved?
Have the needs of those served by the program been met?
What are the unintended outcomes?
Does the implementation strategy lead to intended outcomes?
How do the differences in implementation affect program outcomes?
What are the benefits of the program given the costs? Trends of Impact Evaluation How do we determine a program's worth? "The more a program is efficient, the more worth it will have" There are various levels of outcomes, which can be developed into a hierarchy of outcomes where one level leads to a higher level.
Example of a hierarchy of outcomes for a youth mentoring program (Owen, p. 253)
a. attending school more regularly
b. improved reading skills
c. getting higher grades
d. getting a job Key Trends Benefit to cost ratio (BCR) - ratio of program benefits to program costs
Return on investment (ROI) - ratio of net program benefits to program costs Case Study The High School Education Board of Richmond Hill decided they wanted to educate students on sexual health. Although statistics for teen pregnancy are not high in this area, the education system believes students are not as educated as they should be. In the schools, students were given several health classes that focused on primary sexual health throughout grades 6-9. These classes only taught basic sexual health education. The school board then got together with nursing students from a near by university, to be the program leaders and teach the courses for extra credit. This course is offered to students after school as extra credit as well. Students from grades 10-12 were offered the program. The program was funded by the Education Board so they decided to complete an evaluation to see if the program has an effect on students. Key Approaches Objectives-based
Performance Audit This specific approach is used for management in public and private sectors. The objectives-based approach basically determines whether goals or objectives of a program have been achieved or not. Needs-based Objectives-based Goal-free The needs-based approach is used for judging program worth both internally and externally. This approach determines the worth of the program on the basis of whether a program meets an identifiable need or not. The goal-free approach intentionally ignores all the goals that are set or stated for the given program. The purpose of this is so that goals don't limit the program, but it allows full examination for all areas and aspects of the program since goals narrow things down and only focus on some particular areas and not the overall program itself Process-outcome Studies This approach examines programs to determine outcomes. Owen explains how the examination of a program is an integral part of an impact evaluation Realistic Evaluation This approach is actually based on two propositions as Owen explains in his text. The first proposition that Owen discusses is that evaluation studies should be adopting a "realist epistemology." This means that "findings about the impact of a program are generated through inquiry" (Owen, p. 260). The second proposition that Owen explains is the fact that some programs are more effective under certain circumstances, in certain groups and in certain concepts Performance Audit This approach is designed to provide management with recommendations for action within a program, and also to provide management with opinions for all the activities and ideas within the program. This approach deals with a combination of financial and non-financial measures Owen, J. M. (2007). Program Evaluation: Forms and Approaches. New York: The Guildford Press.