Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Using Post-Then-Pre Evaluation to Determine the LDP Seminar Impact

No description

Mohd Hazwan Mohd Puad

on 10 October 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Using Post-Then-Pre Evaluation to Determine the LDP Seminar Impact

Population and Sample
What are the changes in perceptions of knowledge, skills, and attitudes toward leadership development in CTE of the LDP seminar participants by using the post-then-pre evaluation method?
Using Post-Then-Pre Evaluation to Determine
the Leadership Development Program (LDP) Seminar Impact

Evaluating program impact is important in the world of education and training today.

Why evaluating program impact?
assess the effectiveness of the program.
helps in making decisions.
assists in planning the next program.
appraises the strengths and the weaknesses of the program.
leaves a positive impression with participants.
maintains the credibility of the program.
helps in upgrading program personnel.
Statement of the Problem
Leadership Development Program (LDP) seminar is a regular activity in the Career and Technical Education (CTE) program.

The LDP seminar offers participants up-to-date knowledge and information on Career and Technical Education (CTE), including current skills related to CTE areas, the capabilities, the competencies, the professionalism, and the credibility in CTE.

Typical evaluation method is lacking information on the program’s impact in terms of changes prior to and after the program.

Little research has been explored about the use of the post-then-pre evaluation method in measuring the program impacts of CTE.

The post-then-pre evaluation method has several strengths and advantages in evaluating the program impact.
Research Question
Literature Review
There are many ways to measure the impacts of a program. The post-then-pre evaluation, or retrospective pretest evaluation, has several strengths and benefits that can produces a better results for program evaluation.

The post-then-pre evaluation is administered once, which is after the program. Participants are asked to rate their understandings after the program, then asked to think about their understandings prior to the program (Lamb, 2005)

The post-then-pre evaluation provides more accurate and congruent results compared to other evaluation designs. Participants may have better knowledge at the end of the program that allows them to rate or assess their baseline perceptions (Rockwell and Kohn, 1989).

The post-then-pre evaluation also controls for the error of the response shift bias. Respondents are required to make assessment only after the program is over and all concepts have been sufficiently explained (Nowack, 1986; Klatt & Taylor-Powell, 2005; Rohs, 2002).

The method measures learners’ perceptions of change. Rockwell and Kohn (1989) and Allen and Nimon (2007) pointed out in their study that this method is specifically designed and effective for evaluating the program impact by asking participants to report their actual changes in behavior.

The method is convenience because it needs to be administered once, which is at the end of the program. Responding to both before and after measurement data at the same time is less burdensome as well (Lamb, 2005; Klatt & Taylor-Powell, 2005).

There are limitations as well, such as bias from recall period, bias from self-evaluation, and difficulties in certain cultural contexts (Klatt & Taylor-Powell, 2005;Colosi & Dunifon, 2006). However, the benefits and advantages of the post-then-pre evaluation design still outweigh the disadvantages.
Data Collection
The questionnaire survey will be included in participants’ seminar files.

The LDP seminar participants will be asked to complete their questionnaire survey at the conclusion of the program.
Data Analysis
Questionnaire data will be analyzed by using SAS 9.3b statistical software in order to determine perceptions of knowledge, skills, and attitudes toward leadership development in CTE.

The range of responses will be expressed and summarized by using descriptive statistics; mean and standard deviation, and inferential statistical; normal and paired t-test, in which to show differences in perceptions.
Objective of the Study
To determine the LDP seminar impact by using the post-then-pre evaluation method.
The study is a quantitative study.

The authors will use a survey design to investigate the change in perceptions of knowledge, skills, and attitudes toward leadership development in CTE.
Respondents in this survey study will be limited to all LDP seminar participants in the CTE program.

The results of this study can only be generalized and interpolated into the LDP seminar participants.
The researchers will utilize the questionnaire as an instrument.

The questionnaire will be modified from the existing evaluation survey form for the LDP seminar.

The post-then-pre evaluation questionnaire will assess changes in perceptions of knowledge, skills, and attitudes toward leadership development in CTE of the LDP seminar participants.
Expected Findings
The post-then-pre evaluation survey should reveal the changes in participants’ perceptions of knowledge, skills, and attitudes toward leadership development in CTE.

The comparison of the mean for each item in the questionnaire will delineate whether the changes are positive or negative.

The p-value from the paired t-test will indicate the significance of the changes.

The results will be discussed in order to decide the success of the program in terms of whether the LDP seminar’s objectives are achieved or not.
prepared by
Mohd Hazwan Mohd Puad & Norliza Ghazali
Full transcript