Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

The Road to Results

Theory of Change
by

Anna Hubáčková

on 20 January 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of The Road to Results

Understanding the Evaluation Context and Program Theory of Change The Road to Results 1. Front-end Analysis it is what we do to figure out what to do 3. Constructing, Using, and Assessing a Theory of Change an innovative tool to design and evaluate social change initiatives
a representation of how the organization or initiative is expected to lead to the results and an identification of underlying assumptions being made Constructing a Theory of Change Example of Constructing a Theory of Change for Teacher Visits to Student Homes goal: better academic performance of students
intervention: teachers visiting the homes of students
The logic of the situations follows in this way:

if teachers (input) visit the homes of their students
(activity) and talk to parents (output) they will be more empathetic to the child (outcome), the parents will understand better the school’s need for homework to be completed on time and for the child to attend school every day, and make sure both happen (outcomes). Because the child does homework, attends school regularly, and has an empathetic teacher, the student’s achievement will increase (impact). Anna Hubáčková The evaluator investigates: Who is the main client for the evaluation? Who are other important stakeholders?
How will the timing of the evaluation in relation to project, program, or policy implementation affect the evaluation?
How much time is available to complete the evaluation? What is the nature and extent of available resources?
Does social science theory have relevance for the evaluation? completing a good front-end analysis - critical to learning about an intervention
save time and money on the evaluation
ensure the evaluation meets client needs, and sustain or build relationships, not only with the client, but also key stakeholders
make sure the evaluation is addressing the right questions to get information that is needed rather than collecting various data that may never be used Balancing Potential Costs and Benefits of the Evaluation: Expected costs and benefits of the evaluation and how to balance them should be on the agenda during the front end planning
Think of: cost of evaluation in relation to the cost of the program. They should be in line; spending US $50,000 to evaluate a US $25,000 program does not make sense Pitfalls Involved in the Front-End Planning Process: The belief that everything can be covered up front, including the belief that if front-end planning has been taken place, everything will be okay.
The to-do-ism fixation: this is the danger that the evaluator fixed on the original planning document of planning and can only do what has been planned already. There is resistance to modifying the plan 2. Identifying the Main Client and Key Stakeholders An important part of the front-end analysis is identifying the main client and key stakeholders of the project, program, or policy The Main Client: Often there is one key stakeholder sponsoring or requesting the evaluation
The main client may be:
authorizing and funding the program
authorizing and funding the evaluation
accountable to the public for the intervention
to whom the evaluators are accountable Stakeholders: Stakeholders are all those people or representatives of organizations that have a “stake” in the intervention.
Typically, they are those who are affected by an intervention either during its lifetime or in subsequent years.
Stakeholders can include:
- Participants: those people who participate or have participated in the intervention
- Direct Beneficiaries: those people who directly and currently benefit from the intervention
- Indirect Beneficiaries: those people who are not recipients of the intervention but who benefit from others who are beneficiaries. For example, employers benefit from educational programs since they are able to hire better-trained people
- Development organizations who provide funding
- Program directors, staff, board members, managers, and volunteers Identifying and Involving Key Stakeholders: to identify the key stakeholders can be done by looking at documents about the intervention and talking with the main evaluation client, program sponsors, program staff, local officials, and program participants.
by engaging key stakeholders early on, the evaluator will have a better understanding of the intervention, what it was to accomplish, and the issues and challenges it faced in doing so.
the evaluation team will be better informed as to possible key issues for the evaluation and what information will be needed, when, and from whom Stakeholder Analysis: stakeholder analysis as a technique to identify and assess the importance of key people, groups of people, or institutions that may significantly influence the success of an evaluation.
They also suggest the following reasons for doing a stakeholder analysis:
- identify people, groups, and institutions that can influence the evaluation (either positively or negatively)
- anticipate the kind of influence, positive or negative, these groups will have on the evaluation
- develop strategies to get the most effective support possible for the initiative and reduce obstacles to successful implementation of the evaluation. depict a sequence of the inputs the project, program, or policy will use, the activities that the inputs will support, the outputs to which the project, program, or policy is budgeting either from a single activity or a combination, and the results anticipated in terms of outcomes and impacts
depict other events or conditions in the project, program, or policy context that might affect obtainment of the outcomes
identify the assumptions the program is making about how things will work in terms of cause and effect
identify critical assumptions that based on the policy and environmental context, and a review of the literature, need to be examined by the evaluation a theory of change must: Theories of change open the “black box” or how an intervention expects to take inputs, conduct activities, and produce outputs, and then convert them to results as show in a simple model It is important to identify what is happening in the broader context. This is the environment in which the program operates leading to the result. It can influence the inputs, activities, and the black box Why Use a Theory of Change: A theory of change is valuable to both evaluators and stakeholders because they work together to build “a commonly understood vision of the long-term goals, how they will be reached, and what will be used to measure progress along the way”
A theory of change has many possible uses for an evaluator
-as a framework to check milestones and stay on course
-to document lessons learned about what really happens
-to keep the process of implementation and evaluation transparent, so everyone knows what is happening and why
- as a basis for reports to funders, policymakers, boards a theory of change helps with the strategic marketing efforts in: describing programs in language clear and specific enough to be understood and evaluated
focusing attention and resources on priority program operations and key results for the purposes of learning and program improvement
developing targeted communication and marketing strategies for a project, program, or policy has several benefits: It helps identify elements of programs that are critical to success.
It helps build a common understanding of the program and expectations among stakeholders based on a common language.
It provides a foundation for the evaluation
It identifies assumptions that may become the basis of evaluation questions Before beginning to review or construct a theory of change, evaluators must have a clear understanding of the purpose and goals of the project, program, or policy

The critical questions are:
- Is there research underlying the program?
- What is the logic of the program?
-What are the key assumptions being made? The logic of a program identifies probable occurrences surrounding the goal of a project, program, or policy and the intervention
They may begin by being thought of as “If… then…” situations
identifies “If X happens, then Y should happen” This is the heart of the template and will be the core of the theory of change. In this space, the evaluator writes a clear statement of the problem or issue. It is helpful to know what other successes have done and to learn from best practices from others. The research done before beginning this step can help at this step. . In this box, the evaluator specifies the needs and/or assets of the community or organization so that they can address the problems or issues. If a needs assessment has been conducted or if the needs of the community or organization have been prioritized, these are the kinds of information to include here The evaluator identifies what the project,
program, or policy is expected to achieve
in both the near-term and the long-term.
This might be a vision of the future.
These will become the outputs, outcomes,
and impact. the evaluator lists the potential barriers
and/or supports that might impact for
the change that is hoped. For example,
these may be risk factors, existing policy
environment, or other factors. Here the evaluator lists general, successful strategies
the research has identified that helped similar communities
or organizations to achieve the kinds of results the project,
program, or policy is attempting to elicit.
These may be called “best practices” in the research. the evaluator does is to state the assumptions of how and why the identified change strategies will work in the community or organization. These may be principles, beliefs, and/or ideas The evaluator began with the intended result “”Achievement in Reading” and placed it at the bottom of the diagram
Next, the evaluator identified the intervention at the top by writing “Visits by teachers to student’s home”. From there, the evaluator identified three possible results from home visits:
-teachers gaining an understanding of the home culture of the student
- parents gaining information about what the school expects of the students
- both teachers and parents able to special problems that keep the student from doing well at school. From each of these three possible the evaluator identified other results for each of the three results, creating chains of results and interacting results. For example, from the result “Teachers understanding of the home culture” the evaluator identified a chain of results that included:
- teacher gaining sympathy with children and their view of the world
- teaching in ways that are more comfortable to the student (words and environment)
-student morale improves
- students improve achievement in reading The theory of change also identified several assumptions. The key assumptions the evaluator identified were:
- parents will welcome teachers into their homes
- parents will feel comfortable discussing their views of
educating their child
- teachers will better understand the home culture and learn to adjust their teaching to what they learned
-parents want to be involved in student learning and want their children to attend school Example: Micro-Lending Program Many micro-lending programs intend to promote new livelihoods and improve household well-being by helping women enter the labor force and build entrepreneurial skills thereby increasing household income. Key Assumptions:
Women will create business. They will have time and family support to do so.
The profits generated will not be diverted (e.g. used to pay for dowry).
The business will succeed because there is a strong demand for the product at the price to be charged and there is short supply.The business will succeed despite possible constraints on women’s time or social pressures.The business will succeed as women are provided financial management skills and peer support. If a logical framework is developed for a well-baby clinic, it might include immunizations as one of its activities, with a target result of immunizing 50% of all children under age 6 in a particular district. If this target is achieved, then the incidence of preventable childhood diseases should decrease.
This ultimately should achieve the overall objective of reducing the number of deaths of children under age six.
The second column identifies the indicators that verify the extent to which each objective has been achieved.
The third and fourth columns specify where the data will be obtained in order to assess performance against the indicators, and any assumptions made about the nature and accessibility of those data.
Logic models are extremely useful in showing how a program is supposed to work and achieve its intended outcomes and impacts. They are also useful in identifying through assumptions the threats to the program working as it supposed to and achieving the desired outcomes and impacts. But when conducting an evaluation based on logic models, the evaluator must also look for unintended outcomes and impacts, both positive and negative The logframe is a specific type of logic model or approach. It helps to clarify the objectives of a given project, program, or policy, and to identify the causal links between inputs, processes, outputs, outcomes, and impact.
Performance indicators are drawn up for each stage of the intervention. Key assumptions are articulated, and the manner in which evaluation and supervision will be undertaken is explained. A logical framework links up the activities, results, purpose, and objectives of a program, policy, or project in a hierarchy.
For each of the components, the evaluator identifies the indicators that are needed, the sources, and the assumptions. Logical Framework (Logframe) Example: Logframe for Childcare Summary - Evaluators need to resist the urge to rush into the tasks of evaluating without a plan.
- A front-end analysis is a valuable part of planning that helps evaluators get a larger picture of the project, program, or policy.
- The front-end analysis can answer important questions about the evaluation, including timing, time to complete, who will be involved, resources, design, program theory and logic, and what already exists.

- One part of the front-end analysis is identifying who else is involved in the project, program, or policy.
-A stakeholder analysis is one way to identify the key evaluation stakeholders and determine what they know, what they can contribute, and what they expect.
- Constructing a theory of change underlying the program helps evaluators and stakeholders visualize the program and what evaluation questions could be addressed.
- The process of constructing a theory of change begins with research. What have others learned about similar situations? What is learned from the research can help evaluators make more informed decisions for evaluating the project, program, or policy.
The next step in the process involves three activities: constructing the logic of the program, identifying the key assumptions being made when constructing the logic, and mapping or drawing out the chain of events and their relationships. Than you for your attention!
Full transcript