Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Theory-Based Evaluation

No description
by

on 14 February 2015

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Theory-Based Evaluation

What is Theory-Based Evaluation (TBE)?
What is Theory-Based Evaluation (TBE)?
3.) It is an approach to evaluation that requires surfacing the assumptions on which the program is based in considerable detail: what activities are being conducted, what effect each particular activity will have, what the program does next, what the expected response is, what happens next, and so on, to the expected outcomes (Birckmayer and Weiss, 2000)
Theory-Based Evaluation

By: Lisa, Colin and Eric
What is Theory-Based Evaluation (TBE)?
Theory-based evaluation attempts to use a well-developed theory related to the program that can provide guidance for the evaluation.
Theory driven evaluations should explain a program theory or model
Seeks to investigate how programs cause intended or observed outcomes

Who's Who
Chen and Rossi

Black box = the space between inputs and outputs (The "how" something occurs)
Theory driven evaluations will:
Examine both the stakeholder and evaluators views on outcomes
Examine
why
and
how
program fails

(Chen & Rossi, 1992)
Who's Who

References
1970's
Suchman believed program failure came from 2 items
Failure in Implementation- Not delivered as planned
Failure of the Theory
Weiss believed TBE was the link process between inputs and outputs
1980s
Chen and Rossi
Thought evaluation needed to be connected with social science research
Bickman
Need to know how the program was implemented, but we need measurable outcomes (Quantitative)
1990s-present
Quantitative measures popular, died off again, and are resurfacing now.
"Overall, there really is not much to recommend theory-based program evaluation, since doing it right is usually not feasible and since failed or misrepresented attempts can be highly counterproductive."
Favorite Stufflebeam Quote:
Disadvantages of TBE
Program Theory may be unclear
Approaches may lack convincing evidence
The outcomes may happen but not by the stated theory
May not be generalizable
Some believe the approaches are difficult to understand or ambiguous
The evaluators may not feel comfortable using this method
Advantages of TBE
Who's Who
Carol Weiss

Better programs will result from a better understanding of evaluation and how it is used
A good program theory will take into consideration how politics work
Acknowledges that multiple relationships exist between politics and research
TBE should influence decision making indirectly
What is Theory-Based Evaluation (TBE)?
Let's listen to some amazing robots...
1.) “Any evaluation strategy or approach that explicitly integrates and uses stakeholder, social science, some combination of, or other types of theories in conceptualizing, designing, conducting, interpreting, and applying an evaluation.” (Coryn, Noakes, Westin, & Schroter, 2011)
History of TBE
Bickman
Believes Program theory can improve our ability to generalize and achieve consensus in evaluation planning.

Specifying the underlying theory of a program allows that theory to be tested and reveals if the program is a result of implementation failure or theory failure
(Bickman, 1987)

Causes program designers to think about assumptions
Better understanding of planning
Causal links may be revealed
Increased stakeholder engagement
Allows evaluators to provide information throughout the process
Builds capacity
Does Stufflebeam like TBE?
What is Theory-Based Evaluation (TBE)?
2.) Theory acts as an advance organizer, providing the framework for the questions, methods, and instruments that will guide the evaluation. (Stufflebeam, 2001)
Conclusion
Consider TBE when:
Concerned with the how/why your program works.
You have the time and resources to conduct this.
When you have a solid, relevant theory.

Avoid TBE when:
Theory is undeveloped or may not be relevant.
Resources and time dictate a more practical approach.
Stakeholders are not concerned with the how/why.
Core Principles of TBE
2. Theory-driven evaluations/evaluators should formulate and prioritize evaluation questions around a program theory
Core Principles of TBE
3. Program theory should be used to guide planning, design, and execution of the evaluation under consideration of relevant contingencies
Core Principles of TBE
1. Theory-driven evaluations/evaluators should formulate a plausible program theory
Core Principles of TBE
4. Theory-driven evaluations/evaluators should measure constructs postulated in program theory
Core Principles of TBE
5. Theory-driven evaluations/evaluators should
identify breakdowns, side effects, determine program
effectiveness (or efficacy), and explain
cause-and-effect associations between
theoretical constructs
(Coryn,Noakes,Westine, & Schroter, 2011)
(Coryn,Noakes,Westine, & Schroter, 2011)
(Coryn,Noakes, Westine, Schroter, 2011)
(Coryn,Noakes, Westine, Schroter, 2011)
(Coryn,Noakes, Westine, Schroter, 2011)
Other TBE Researchers
Edward Suchman
Anthony Rogers
Timothy Hasci
Tracy Huebner
Ray Pawson
Nick Tilley
Bickman, L. (1987). The functions of program theory. New directions for program evaluation, 1987(33), 5-18.

Birckmayer, J. D., & Weiss, C. H. (2000). Theory-Based Evaluation in Practice What Do We Learn?. Evaluation review, 24(4), 407-431.

Chen, H. T. (2012). Theory-driven evaluation: Conceptual framework, application and advancement. In Evaluation von Programmen und Projekten für eine demokratische Kultur (pp. 17-40). Springer Fachmedien Wiesbaden.

Chen, H. T., & Rossi, P. H. (Eds.). (1992). Using theory to improve program and policy evaluations (No. 290). Greenwood Publishing Group.

Coryn, C. L., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009.American Journal of Evaluation, 32(2), 199-226.

Donaldson, S.I. (2007).
Program theory-driven evaluation science: Strategies and applications
. New York: Erlbaum.

Fitzpatrick, J., Sanders, J., & Worthen, B. (2010). Program Evaluation: Alternative Approaches and Practical Guidelines (4th Ed.) Boston, MA: Pearson Education, Inc.

Stufflebeam, D. (2001). Evaluation models. New directions for evaluation. San Franciso, CA: Jossey-Bass.

Weiss, C. (1998). Evaluation: Methods for Studying Programs and Policies, Prentice Hall, New Jersey.
Who's Who
Donaldson
Developed seven steps of theory development:
1) Engage relevant stakeholders
2) Develop a first draft of program theory
3) Present the draft to stakeholders for further discussion
4) Conduct a plausibility check
5) Communicate these findings to key stakeholders and revise as needed
6) Probe arrows for model specificity
7) Finalize program impact theory

(Donaldson, 2007)
Full transcript