Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Utilization-Focused Evaluation

No description
by

JING JING Xu

on 22 November 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Utilization-Focused Evaluation

The founder of U-FE is Michael Quinn Patton, former President of the American Evaluation Association. He developed this evaluation approach in his book
Utilization-Focused Evaluation,
which

was published in 1978, the latest (the 4th) edition was published in 2008.
Description of UFE
Premises of U-FE
No evaluation should go forward unless, and until, there are primary intended users who will use the information that can be produced
Primary intended users are involved in the process
Evaluation is part of initial program design - The primary intended users want information to help answer a question or questions.
Evaluator’s role is to help intended users clarify their purpose and objectives.
Make implications for use part of every decision throughout the evaluation – the driving force of the evaluation process
The 17 STEP U-FE Framework
What is Utilization-focused evaluation (U-FE)?
Rationale of U-FE
Utilization-Focused Evaluation
by Jessica, JingJing, Razan and Susana
Michael Quinn Patton
Utilization-focused evaluation does not depend on any particular model, method, theory, or use
This type of evaluation “answers the question of whose values will frame the evaluation” (Patton, 2010a, p. 252)
Since it is focused on the users, they will be responsible of determining the content, model, method and theory for their own use
Evaluator must develop relationship with users
Evaluator facilitates judgement and decision-making rather than being a distant judge

U-FE is "evaluation done for and with
specific intended primary users
for
specific, intended uses
",
it is
“concerned with how real people, in the real world apply evaluation findings and experience the evaluation process” ----
Michael Quinn Patton
Definition of U-FE
Pros and Cons of U-FE
Pros of U-FE
Creates buy-in because intended users make decisions about evaluation; therefore, the evaluation is also more likely to be used.
Trains users in use of evaluation by actually involving them.
Evaluation purpose, data, design and focus chosen because they are the most appropriate.
One of nine types of evaluations that meet the standards for evaluation.
Cons of U-FE
Users may renege on promise once high cost and time commitment is understood.
Does not include the audience on the task force therefore, can be viewed as missing major stakeholder.
Can create bias if relationships get too close.
Assumes evaluators are well versed in multitude of hard and soft skills. (Donaldson, Patton, Fetterman & Scriven, 2010)
May not meet expectations of clients who view RTC as the "gold standard".
U-FE framework:
The original 12 steps
UFE Framework
Patton (2008) summarized U-FE into 12 steps. It should be noted that while UFE is summarized into a series of steps,
the process
itself is
not linear
(Patton, 2012). The following diagram was produced by Romirez and Brodhead (2013) and
clearly shows the 12 steps U-FE framework, as well as its non-linear characteristic.
U-FE Application:
An Example
If you are interested in additional information on U-FE and how it is applied, the link below leads to a case study that describes how the Information Society Innovation Fund (ISIF) used UFE for the Developing Evaluation Capacity in ICTD (Information and Communication Technology for Development) project.

http://evaluationinpractice.files.wordpress.com/2012/05/case-study_isif_final.pdf
Advice for using U-FE
UFE requires active and skilled guidance from, and facilitation by, an evaluation facilitator.
Time resources available for the evaluation must be clearly negotiated, built in from the beginning, and stated in the agreement.
The essentially collaborative nature of UFE demands
time and active participation, at every step of the entire process
, from those who will use the evaluation results. Additional resources may be needed if new uses or users are added after the evaluation has begun.
Financial resources available for the evaluation must be clearly stated. They must include financial resources beyond mere analysis and reporting. Resources that facilitate use must be available.
In conducting a UFE, the evaluator must give careful consideration to how everything that is done, from beginning to end, will affect use.
Case Study Take-Aways
The organization needs to be open to the evaluation.
Additional resources (financial, human) may be required as evaluation progresses.
U-FE requires expertize.
The research area and the potential users and uses need to be clearly identified.
Personal relationships are critical.
References

Better evaluation. (n.d.).
Utilization-focused evaluation.
Retrieved from http://betterevaluation.org/plan/approach/utilization_focused_evaluation

Noakes, Lindsay A. (2009).
Adapting the Utilization-Focused Approach for Teacher Evaluation.
Journal of MultiDiciplinary Evaluation, 6 (11), 83-88.

Patton, M. (2012).
Utilization-focused evaluation checklist
. Retrieved from http://www.wmich.edu/evalctr/wp-content/uploads/2010/05/UFE.pdf

Patton, M. (2010a). Utilization-Focused Evaluation. In
From policies to results: Developing capacities for country monitoring and evaluation systems.
New York: UNICEF.

Patton, M. (2010b).
Utilization-focused evaluation workshop
. Retrieved from http://ccer.org/service/documents/Michael%20Patton/U-FE%20workshop.pdf

Patton, M. (1984).
An alternative evaluation approach for the problem-solving training program: A utilization-focused evaluation process
. Evaluation and Program Planning, 7(2), 189-192.

Ramirez,R. & Brodhead, D. ( 2013).
Utilization focused evaluation: A primer for evaluators
. George Town, Malaysia: Southbound

Stewart I. Donaldson, Michael Q. Patton, David M. Fetterman Fetterman Associates, & Michael Scriven. (2010).
The 2009 Claremont Debates: The Promise and Pitfalls of Utilization-Focused and Empowerment Evaluation
. Journal of MultiDisciplinary Evaluation, 6(13), 15.
History of U-FE
In 2012, Patton added 5 additional steps (highlighted in red below) into his original framework to provide guidance for possible variations.
(Patton,M. 2010, p.29)
Step 1: Assess program and organizational readiness
Step 2: Assess evaluator readiness and competence
Step 3: Identify, organize, and engage primary intended users.
Step 4: Conduct situation analysis with primary intended users
Step 5: Identify primary intended uses by establishing the evaluation’s priority purposes.
Step 6: Consider and build in process uses if appropriate.
Step 7: Focus priority evaluation questions.
Step 8: Check that fundamental areas for evaluation inquiry are being adequately addressed.
Step 9: Determine what intervention model or theory of change is being evaluated.
Step 10: Negotiate appropriate methods to generate credible findings and support intended
use by intended users.
Step 11: Make sure intended users understand potential controversies about methods and
their implications.
Step 12: Simulate use of findings.
Step 13: Gather data with ongoing attention to use.
Step 14: Organize and present the data for use by primary intended users.
Step 15: Prepare an evaluation report to facilitate use and disseminate significant findings to
expand influence.
Step 16: Follow up with primary intended users to facilitate and enhance use.
Step 17: Metaevaluation of use: Be accountable, learn, and improve
The 17 step UFE framework
(Better evaluation, n.d.)
U-FE Quiz
1. Define U-FE.

2. What is the role of the evaluator in U-FE?

3. What type of design is best for U-FE?

4. How does U-FE facilitate use of the
evaluation results?

5. T/F: The U-FE framework is a linear
process.

6. Name 2 pros and 2 cons of U-FE.


Answer Key
1. Utilization-focused evaluation, U-FE, is a type of evaluation that engages primary intended users in the evaluation process. U-FE is done
for and with
primary intended users with the aim of creating buy-in so that the results of the evaluations will be used.

2. The evaluators role is to facilitate judgement and decision-making by helping primary intended users to clarify their purposes and objectives.

3. U-FE doesn't promote any particular design, model, method, theory or use. The primary intended users are responsible for determining the content, model, method and .theory for their own use
Answer key
4. U-FE facilitates the use of the evaluation results by creating buy-in with the primary intended users. These users are involved in entire process and make decisions about the design, method, and focus of the evaluation. By engaging the primary users so that they understand and feel ownership over the evaluation process and findings, the results are more likely to be used.

5. False. The 17-step U-FE framework, and its 12-step predecessor, are not linear processes. There are interconnections and feedback loops that need to be considered throughout the process.

6. Pros: creates buy-in with intended users; trains users in use of evaluation; most appropriate evaluation purpose, design and methods selected; one of nine recognized evaluation models.
Cons: requires high cost and commitment from users; can create bias; potentially excludes a major stakeholder; assumes evaluators have multitude of hard and soft skills.
Donaldson, Patton, Fetterman & Scriven, 2010
Check your learning
Learning Outcomes
By the end of this presentation, students should be able to:

1. Define utilization-focused evaluation.

2. Identify the roles of the primary intended users and the evaluator in UFE.

3. Recognize components of the UFE framework.

4. Identify pros and cons of the UFE approach.
U-FE starts with the premise that evaluations should be judged by their utility and actual use
U-FE can be used for different types of evaluation as well as different research designs and types of data.:
Purpose
: formative, summative, or
developmental
Data
: quantitative, qualitative or mixed
Design
: e.g. naturalistic, experimental
Focus
: e.g. processes, outcomes,
impacts, costs, and cost-benefit
When and In What Context
to use U-FE?
Full transcript