Sarah Earl Evaluation Unit
Sarah Earl
October 2011
Objectives
1) Review how our paradigms
influence our approach to
evaluation
2) Explore the purpose behind IDRC`s
support of evaluation capacity
development with project partners
3) Share the main lessons
from DECI
Nurturing an Evaluation Culture with Partners
The Goal
DECI
engage in self-reflection & self-examination
engage in evidence-based learning
encourage experimentation & change
Weaker Evaluation Cultures
gather information on results BUT limit its use mainly to external reporting
acknowledge the need to learn BUT provide no time or structured occastions to do so
claim to be evidence-seeking BUT discourages challenging and questioning
talk about the importance of achieving results BUT value following the rules and frown on risk taking
John Mayne, 2008
Higher-Impact
Higher Performing
Manage Multiple Accountabilities
John Mayne, 2008
Leadership
Support structures
Learning focus
Evaluation Paradigms
qualitative data
naturalistic inquiry
case studies
inductive analysis
subjective perspective
close to the program
holistic contextual portrayal
systems perspective
dynamic, ongoing view of change
purposeful sampling of relevant cases
focus on uniqueness and diversity
emergent, flexible designs
extrapolations
Constructivist
Positivist
quantitative data
experimental designs
treatment and control groups
deductive hypothesis testing
objective perspective
aloof from the program
independent & dependent variables
linear, sequential modeling
pre-post focus on change
probabilistic, random sampling
standardized, uniform procedures
fixed, controlled designs
statistical analysis
generalizations
Adaptive
Pluralists
Neo-
Newtonians
Robert Chambers, 2008
Stronger Evaluation Cultures
John Mayne, 2008
Use-Focussed Evaluation
by Michael Quinn Patton
?
What are the main messages from the video?
What experiences can you share where evaluation was/wasn`t useful to you and/or partners? What contributed?
...
Background
?
Purpose: to build partners' capacity
& use of high-quality evaluation
Structure: 5 large projects, 3 Asian
UFE mentors, 2 facilitators
Results: primer; partners' evaluations & use of it;
capacities of evaluators & researchers;
Lessons
What resonates for you?
How is this different from what you`ve experienced?
What can you learn from DECI & use?
...
What roles have you played?
what`s been your experience?
what other roles might you try?
How do we do it well?
Evaluation Studies &
Evaluation Thinking
Bring:
a crown corporation created by Parliament in 1970
215$ million budget in 2009-2010
mission = Empowerment through Knowledge
Endogeneous researchers take the lead
Developing research capacity
Building networks
Influencing policy and practices
overview
our approach
our regional presence
our evaluation system
our evaluation principles
The decision to evaluate should be strategic not routine
Evaluative thinking adds value to a project or program from the outset
Evaluation should be an asset to those being evaluated
Evaluation should enlist the participation of relevant users
Evaluation processes should develop capacity in evaluative thinking and evaluation use
Evaluation should meet quality and ethical standards
Learning about the theory, practice, and findings of evaluation should be documented and shared.
How to do it
Video 24:09-29:25
Thank you!
searl@idrc.ca
http://prezi.com/iodcisqrt3pv/edit/#0_54656
Power plays itself out through evaluation
It isn`t always A primary INTENDED user
UFE doesn`t happen naturally & competencies are required
UFE mentor model can be effective
4:05-13:53
