Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Evaluation Challenge December 2012

No description

Barbara Gallo

on 5 December 2012

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Evaluation Challenge December 2012

preventive strategies to reduce violence (physical, sexual, emotional) occurring in FAMILY SETTING and affecting children, including – but not limited to – the youngest children (0-8 years) ONLY external EVALUATIONS
(not implementation) 3 CATEGORIES OF INTERVENTIONS:
large scale and well established interventions;
small scale innovative interventions;
interventions expected to have an INDIRECT impact in terms of violence reduction. GEOGRAPHICAL FOCUS: low and middle-income country - DAC list TYPE OF APPLICANTS:
- local or international NGOs,
- university/research institutes
- exceptionally, for-profit organizations. Partnerships between researchers and practitioners encouraged. Selection process 20 APPLICATIONS SHORTLISTED What did we learn
from the 1st call? CHALLENGES AHEAD 9 SUCCESSFUL GRANTEES Thank you! For more information
www.evaluationchallenge.org WHO World Report on Violence and Health, 2002 UN Study on Violence 2006 Report of the UN SRSG
on VAC, 2011 hosted by Rationale SMALL CASH GRANT MECHANISM Rationale and potential for impact
Focus of the evaluation (scope, objectives)
Quality and rigour of the methodology
Evaluation capacity and feasibility
Partnership and opportunity for knowledge transfer
Dissemination and utilization of findings
Cost-effectiveness Steering Committee of the Fund
+ Ad-hoc Assessment Team of 8 experts in violence prevention, child protection, research and evaluations Selected by Selection Criteria Total amount
€EUR 758,884 Type of Partnerships:
Local NGO + consultants
Local NGO + local research institute
Local NGO + for-profit org
Local NGO + for-profit org & consultant

Intl. NGO + foreign university
Intl. NGO + foreign University & consultant
Intl. NGO & Govt + foreign university & for-profit org

Government + Foreign University
Government + Local University Programme Manager: Barbara Gallo
barbara.gallo@nef-europe.org EXPERTS' CONSULTATION
(October 2012) 6 global experts Main experts'
recommendations THEMATIC FOCUS AREA
PREVENTION of ANY form of VIOLENCE against CHILDREN (0-18 years) Various nature and scale of intervention
large scale and well established interventions;
small scale innovative interventions;
not designed to prevent violence but expected to have an impact in terms of violence reduction. GEOGRAPHICAL FOCUS: low and middle-income country - DAC list TYPE OF MAIN APPLICANTS:
- local or international NGOs,
- university/research institutes
- exceptionally, for-profit organizations. CALL FOR PROPOSALS 2012 TECHNICAL SUPPORT: How to set up the mentorship mechanism in practice? Dilemma: quality/rigor vs. capacity building? How to know if a rigorously designed evaluation is also rigorously conducted? QUALITY REVIEW MECHANISM? COMMUNICATION - what to do once we get the evaluation reports? META- EVALUATION - how do we assess the impact of the Evaluation Challenge Fund? Child Protection Funders Group Questions for the experts Who? What? Why? The Fund in a nutshell Key elements TYPE OF INTERVENTIONS Parenting, teachers training and child empowerment at school, ALBANIA
Parenting education through self-administered manuals, CHINA
Parenting (Violence Prevention Curriculum) in an ECD programme, COLOMBIA
Community ECD centres & Child Protection Committees, DOMINICAN REPUBLIC
Boys/men and community engagement, ETHIOPIA
Girls and women empowerment against sexual abuse, INDIA
Child empowerment and community awareness to reduce sexual abuse against children with disabilities, BURUNDI
Child sexual offenders rehabilitation to avoid recidivism, SOUTH AFRICA
Housing programme to reduce risk of violence against children at home, KENYA OPPORTUNITIES FOR COLLABORATION:
How to bring in other critical stakeholders?
How to foster inter-sectoral approaches to VAC? Violence prevention: primary, secondary + tertiary Interest in CLUSTERS of interventions - list not exhaustive
parenting education
interventions addressing cultural and social norms
community-based interventions - focus on the most vulnerable RIGOROUS EVALUATION - use of recognized social research or evaluation methods that help assess the changes that can attribuetd to a specific intervention or to which the interevention contributed to.
Wide range of evaluation methods form experimental and quasi-experimental to mixed methods and qualitative. EVALUABILITY! TYPE OF EVALUATIONS 3 experimental - RCTs
4 quasi-experimental (mixed quant-qual)
1 mainly qualitative
1 pre-post test design (baseline in 2008) Main Applicants:
4 local NGOs
2 international NGOs
1 local university
2 foreign universities 479 CONCEPT NOTES from 74 low- and middle income countries! Success rate in the first phase:
only 1,9% for local NGOs (7 out of 367) vs. 40% for foreign universities (4 out of 10) 76% of applications from local NGOs – not only researchers and academics are interested in evaluations - there is demand! ‘There is a lot going on’ about violence prevention and child protection in the field There is lack of common understanding about ‘rigorous impact/outcome evaluation’ There is still limited evaluation capacity – esp. in local NGOs There is huge need for/interest in the initiative and evaluations Only 38% of concept reflected a partnership practitioners & researchers --> but it was a key factor for success (85% of the shortlisted) issue of UNBALANCE in the partnership: either clearly academic-driven or practitioners - driven! Joost Kooijmans Chris Mikton
Sam Bickel
Bill Bell
Charlotte Watts Xiangming Fang
Irene Cheah
Syed Masud Ahmed Rachel Jewkes
Patrick Onyango
Mokhantso Makoae
Lori Michau
Deograsias Mushi Tatiana Moura
Silvia Raquel Franco 1. Strategic positioning of the FUnd
2. Thematic focus of the 2nd call:
wide vs. narrow formulation?
3. What type of intervention and what type of evaluation?
4. How to foster partnerships between researchers and practitioners? How to involve local researchers?
5. How to ensure the dissemination and uptake of the evaluation findings? 10 regional experts Wider thematic focus + sub-clusters of interventions
Importance of the quality and evaluability of the intervention --> evaluation must be justified, feasible and likely to provide useful information
Quality and rigour of the evaluation are essential (if possible, cost-effectiveness considerations should be included)
PARTNERSHIP implementers/evaluators must be a requirement!
Issue of EVALUATION CAPACITY --> MENTORSHIP MECHANISM as on-going technical assistance (from proposal development to the end of the evaluation project) to be offered by the Fund to the applicants in need There is a disconnection practitioners/ researchers --> PARTNERSHIP ISSUE PARTNERSHIP IS A MUST! CV or organizational profile must be attached to the concept note NEW!! Governments or international organizations can be PARTNERS. Involvement of local researchers strongly encouraged. DEADLINE:
17th December TIMELINE 1 November 2012: Launch
17 December 2012: deadline concept notes
By March 2013: 1st phase of selection
April- May 2013: Full Evaluation Proposals
By July 2013: Successful grantees announced
By October 2015: EVALUATION RESULTS! SELECTION CRITERIA Relevance and potential for impact
Quality and evaluability of the intervention
Quality and rigor of the evaluation
Feasibility and capacity to deliver
Partenrship, opportunity for knowledge transfer and involvement of local researchers
Stakeholders engagement and utilization of findings
Efficiency/cost-effectiveness of the evaluation
Research ethics SO FAR... 9,221 visits to the website 2,187 downloads of the pdf Call for proposals 2012 111 new subscribers to the Mailing List -->total of 752 subscribers to be continued... COMMUNICATION STRATEGY Key Target audiences Key Messages Strategies and means Communication plans with budget
Stakeholders involvement since the beginning
Clarity on the messages
Identify target audiences
Identify upcoming opportunities (post MDGs?)
Set standards for reports (Rule 1/4/40)
Learning process - assess and adjust NGOs - local and international
Universities/research centres - North and South
Key child protection stakeholders (as channels)
Donors (Foundations and beyond)
Policy makers in LMICs (indirect targets) EVALUATION FINDINGS - what works/does not work, why, how
need to invest in evaluations
need for practitioners and evaluators to partner
need to use evaluation findings
Identity of the Fund and funding opportunities On-line communication tools: website, social media, mailing list
Publications: reports, journals (The Lancet...), policy briefs
Synthesis of the findings (?)
'Good stories to tell' - Most Significant Change
International/regional conference/symposium
Grantees' workshops
Media involvement (?) Annual General Meeting
6-7 December 2012 Save the Children - ALBANIA Evaluation of Violence against Children in Schools and Families in Durres , Elbasan and Berat Districts IMPLEMENTERS:
Save the Children Albania & Regional Education Authorities EVALUATORS:
National consultant (Buka) &
International consultant (Di Maio) Duration: June- July 2012 (extension to Dec 2012 for communication activities) INTERVENTION:
Implemented in 4 schools and 3 kindergarten of the 3 districts
Strategy: actively involving children, families and teachers to reduce violence in schools and at home. Components:
- complaints procedures for children to report abuse
- violence prevention modules in school optional curricula
- teachers capacity building on positive discipline
- parents training on positive parenting EVALUATION:
Pre/Post-test design (baseline 2008)
Mixed quant-qual: surveys with children (338), parents (308) and teachers (81)+ focus group discussions (13) and interviews (23)

Budget for evaluation: EUR 15,501.92 Key findings Remarkable reduction in physical violence by teachers in schools
No decrease in pshycological violence (even increased)
Slight reduction of violence within family
Positive effect on reduction of peer-violence (unintended)
Higher impact in the project sites of 3 years of implementation (Durres, Elbasan) vs. 1 year (Berat) Opportunities for impact At programme level: "The evaluation findings will inform SC strategy 2012-15": more focus on parent/community involvement (beyond schools) and more behavioural changes/social mobilization approaches . Findings to be shared with SC neighbouring country offices in the Balkans At policy level: "The National Inspectorate has picked up the system of reporting and resolving compliants against teachers' inappropriate behaviour - it is now likely to propose this system (...) into the bylaws expected to be revised in the near future

Findings to be presented in a National Education Conference - "SC will continue to work with the MoE and REA. The evaluation has brought credibility to our work and will help us work more closely on this issue at national and sub-national level". Recommendations * More teachers training on how to handle students without using psychological violence
* More parents' engagement on positive discipline
* Peer-to-peer violence as a structural element of the project design
* Reflect on a more gradual phasing-out
* More advocacy to have reporting procedures embedded in the existing legal framework and training programmes into standard curricula
* Stronger child safeguarding policies
* More attention to discrimination suffered by children on geographical and/or ethnic basis * Evaluation report + abridged version
* Child-friendly report
* Meetings with beneficiaries and stakeholders (children, teachers, communities, child protection workers, Regional Educational Authorities, University)
* National Education Conference Budget for implementation: EUR 288,000 US Evidence Summit on Protecting Children Outside of Family (Public Law 109-95), Dec 2011 US Government Plan of Action on Children in Adversity
Full transcript