An Overview of Summative Evaluation
By: Heather Steinman
Approach #1: Smith and Ragan
- Occurs in the Evaluation Phase
- Eight Step Process
- Goals of Evaluation
- Select Indicators of Success
- Select the Orientation of Evaluation
- Select the Design of the Evaluation
- Design or Select Evaluation Measures
- Collect Data
- Analyze Data
- Report Results
Approach #2: Morris, Ross, and Kemp
Proposed Revisions and Criticisms of Kirkpatrick's Four Levels of Evaluation
Criticisms
- The levels are:
- Not in the correct order
- Not informally linked
- Certainly inter-connected
Revisions
- Flipping the model so that Level 4 comes first followed by Level 3, then 2, and finally Level 1
- Changing Level 1 to Motivation from Reaction
- Measuring how learners see and react to their learning and the performance process
- Using attitude questionnaires
- Assessing task engagement
- Major steps within summative evaluation are:
- Specifying Program Objectives
- Determining the Evaluation Design for Each Objective
- Developing Data Collection Instruments and Procedures for Each Objective
- Carrying Out the Evaluation
- Analyzing the Results from Each Instrument
- Explaining the Results
- Circulating the Results and Conclusions
What is Summative Evaluation?
Approach #4: Kirkpatrick's Four Level Evaluation Model (Level Four: Results)
Approach #3: Dick and Carey
- Two Main Phases Within the Summative Evaluation Process:
- Expert Judgement
- Field Trial
- Proves the worth or quality of instruction at the end of a program's activities
- External
- The focus is the outcome
- Answers the following questions:
- Did the object being evaluated meet the intended outcome or goals?
- Was the client's goals met?
- Measuring the return an organization receives
- Evaluates a programs effectiveness in business terms
- Seeing how well training affected different areas
- Most important level
Approach #4: Kirkpatrick's Four Level Evaluation Model (Level One: Reaction)
Approach #4: Kirkpatrick's Four Level Evaluation Model (Level Three: Behavior)
- Providing data on how participants responded to training
- Specific Reactions are needed
- Asking yourself, "How well the learners liked the learning process?"
- Transfer of knowledge of skills and attitudes from the training to the actual performance setting
- Important since the main purpose of learning is to improve results by applying newly acquired knowledge and skills to the performance setting
- Asking yourself: "Do people use their newly acquired skills on the job?"
Approach #4: Kirkpatrick's Four Level Evaluation Model (Level Two: Learning)
- Improving knowledge, increasing skills, and changing the attitudes of learners
- Asking yourself: "What did the learners learn?"
- Requires some form of a post-test