Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Complexity and the Art of the Possible

Presented at the International Program for Development Evaluation Training on 10 June 2016 in Ottawa
by

Jeneen Garcia

on 31 October 2017

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Complexity and the Art of the Possible

Adapted from OECD DAC 2002

AND
What is
Inputs and Activities
Correct assumptions
An intervention’s typical theory of change
IMPACT?
Positive + Negative
Intended + Unintended
Primary + Secondary
Direct + Indirect
Changes and trends that are
Caused by the intervention
IMPACT
NOTHING
IT'S
International Program for Development Evaluation Training
10 June 2016, Ottawa
Jeneen R. Garcia
The larger system
a.k.a. the real world
How to know what to include
What elements does it intend to use to achieve impact?
What does it intend to influence to achieve impact?
What impact does it intend to achieve?
What else can it potentially influence?
What other elements can potentially influence the intended impacts?
What can potentially influence its implementation?
What can potentially influence the elements that are supposed to help achieve impact?
INTERVENTION
IMPACT
LARGER SYSTEM
Read! Talk to experts! Think creatively!
INTERVENTION TOC
and the
COMMUNICATION'S
NEXT
BIG
Art of the
Practical Approaches to Dealing with Complexity in Evaluation
MITIGATE. ADAPT.
WHAT IS
COMPLEXITY
?
Multiple INTERDEPENDENT components
UNPREDICTABLE outcomes
NON-LINEAR causal pathways
No consensus on definition!
WHAT IS THE IMPACT OF GEF SUPPORT?
US$ 3.4 billion
in grants to
137 countries
US$ 12.0 billion
in cofinancing for
2,785,350
sq km
1,292
protected areas
618 projects
over the last
25 years
COMPLEXITY
Possible
THINK OUTSIDE THE BOX!
SCOPE
DESIGN
REPORT
CONDUCT
Dealing with complexity at
DIFFERENT STAGES OF THE EVALUATION
HOW DO YOU ASK THE RIGHT QUESTIONS?
Reflect complexity in Theory of Change
your intervention
the system that your intervention is PART OF
HOW DO YOU ANSWER THE QUESTIONS RIGHT?
Use mixed methods AND mixed approaches
Identify
data and methods available and feasible
QUALITATIVE
QUANTITATIVE
SPATIAL
NATURAL SCIENCES
SOCIAL SCIENCES
>30000
35351
Mixed and fixed effects modeling
2440
Regression trees and random forest modeling
Principal Components Analysis
Qualitative Comparative Analysis
Generalized Additive Models
33
Portfolio analysis
Propensity score matching
Remote sensing analysis
191
TERMINAL EVALUATIONS
PAs
METTs
CONTEXTUAL VARIABLES
PIXELS
Peer-reviewed literature
Global databases
7
28
COUNTRIES
SITES
Case study analysis & synthesis
Linear regression
Supercomputers
High-resolution satellites
communication and trust
REGION
COUNTRY
SITE
SCS Network with GEF
SCS Network without GEF
Different Units of Analysis
VERSUS
indicators you need to measure
what
when
where
complementarity to other donors
participation in regional initiatives
mainstreaming of management approaches
functionality of systems
Different Sources of Evidence
environmental trends
community engagement in environmental activities
Interviews w/ different countries
Historical documents
Global databases
Scientific literature
Social network analysis
Interviews w/ different ministries
Financial documents
Qualitative comparative analysis
Propensity score matching
Interviews w/ different villages
Field visits
Local monitoring data
Remote sensing analysis
management capacities
HOW DO YOU DEAL WITH SURPRISES?
Adapt to emerging findings
how
Match methods to units of analysis
HOW DO YOU DEAL WITH MULTIPLE FINDINGS?
Triangulate!
Allows you to identify what needs to be measured, when and where
Linear Theory of Change
Non-Linear Theory of Change
Theory of Change
Units of Analysis
Methods
Difference-in-difference at global scale
Propensity score matching at country scale
(ecological)
South China Sea
East Asia Region
(geopolitical)
Less forest cover loss in GEF PAs
Positive wildlife abundance trends
0.9%
3.4%
GEF PA
2.3%
4.5%
non-GEF PA
in the same biomes and countries from 2001 to 2012
23%
less forest loss than non-GEF PAs across all Mexico PAs
20%
more forest in non-GEF PAs in mangrove biome
up to
28%
avoided forest loss in tropical and subtropical coniferous forest biome
Positive environmental outcomes at project end
68%
12%
Outcomes at risk due to continuing threats
Weaker link between objectives and species
= less positive population trend over time
Elephant population trend at Queen Elizabeth National Park, Uganda
(n=88)
Link to project objectives
Protected Area System
Protected Area
Stronger management capacities
Increased community engagement
Ecosystem rehabilitation
Fire control
Joint law enforcement
Human-wildlife conflict prevention
Professional and dedicated staff
Improved infrastructure & equipment
Sources of revenue
Expanded areas
Policy development
Science-based
Broad stakeholder consultation
Benefits to communities
Financial & HR systems
Transparency
Sustainability
Management Approaches
Increased connectivity
Ecosystem representativeness
BROADER
ADOPTION
in large part due to socioeconomic benefits
Wildlife Conservation Department, Ministry of Tourism, Wildlife and Antiquities, UGANDA
James Lutalo
,
Commissioner
Evaluation Officer
THEN
AND
Understand
THEN
Interventions
Evaluation Arrangements
Evolution of Complexity Theory
http://www.art-sciencefactory.com/complexity-map_feb09.html
under what conditions
Not just YES or NO, but
how
why
Access live Pezi here: https://prezi.com/hbqv1xjluujc/complexity-and-the-art-of-the-possible/
Full transcript