Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

Good Evaluation Data =

Good Tools + Good Evaluators

Evaluation

Evaluating Students

Tool

Evaluator

A Presentation from the

Allied Health Education Committee Learning Lunch Series

Halo Effect

Rebecca Blanchard, PhD.

Reliability

"He's such a nice boy; he must be competent."

"He measures

110cm every

hour!"

Same score consistently

Academic Week 2013

Range Restriction

Validity

(Select one)

Measuring what we think we're measuring

"Everything that can be counted does not necessarily count;

Direct Observation Evaluation

everything that counts cannot necessarily be counted." - Einstein

"This thing is not very good at measuring knowledge."

Miller (1990)

Assessment of Clinical Competence

Hawk/Dove Effect

=

Define behavior(s) of interest

5

3

2

4

1

Agree on importance of behaviors

Label response anchors

Ultimately,

it's all about helping him grow.

Discuss evaluations with colleagues

Use comments

References

> Bandiera, Glen MD. How do I improve the quality of in-training assessment of learners. CJEM. 2011 Jul;13(4):267-72.

> Canavan C, Hotlman MC, Richmond M, Katsufrakis PJ. The quality of written comments on professional behaviors in a developmental multisource feedback program. Acad Med. 2010 Oct; 85(10 Suppl):S106-9.

> Carmines EG, Zeller RA. Reliability and Validity Assessment. Sage University Paper Series on Quantitative Applications in the Social Sciences, 07-017. 1979. Newbury Park, CA: Sage.

> Harden RM. Assessment, feedback and learning. In: Approaches to the Assessment of Clinical Competence. International Conference Proceedings. Norwich: Page Brothers; 1992. 9-15.

> Jackson N, Jamieson A, Khan A. (ed) Assessment in medical education and training. 2007. Abingdon, UK: Radcliffe Publishing.

> Kogan JR, Hess BJ, Conforti LN, Holmboe ES. What drives faculty ratings of residents' clinical skills? The impact of faculty's own clinical skills. Acad Med. 2010 Oct;85(10 Suppl):S25-8.

> Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990; 65: S63-7.

> Norcini, John J. ABC of learning and teaching in medicine: Work based assessment. BMJ 2003;326:75.

Evaluator

Evaluation Tool

"Attentiveness"

"Professionalism"

Why do we

evaluate learners?

Evaluation Tool

5

3

2

4

1

Harden (1992)

3!

5

3

2

4

1

Stay

away!

Cases, MCQ, Essays

Knows How

5!

"...or even

defined!"

5

3

2

4

1

Evaluators

Direct Observation

Does

Objectives

1. Identify common threats to student evaluations

2. Discuss opportunities to improve the quality of direct observation assessments

MCQ

Knows

Teaching

& Learning

Height

Assessment

5

3

2

4

1

Simulation

Shows How

Learn more about creating dynamic, engaging presentations with Prezi