Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Constructing Meaningful Muliple Choice and Constructed Response Questions: Part II

This is a presentation created for professional development at Jefferson County Public Schools, Ky. In this part, we examine the qualities of multiple choice questions.
by

Sylvia Ives

on 1 November 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Constructing Meaningful Muliple Choice and Constructed Response Questions: Part II

?
?
?
?
CONSTRUCTING QUALITY MULTIPLE CHOICE AND CONSTRUCTED RESPONSE QUESTIONS TO SUPPORT PLC WORK
Grades 3-5
Part 2
?
?
?
?
Qualities of MCQs and CRQs
MCQs and CRQs are scaffolded to draw students into a deeper understanding of the text, even for an assessment.
Therefore, the students’ minds must flow through the entire multiple choice question and not be interrupted by distractors that make no sense.
You also don’t want to interrupt the flow with sentence constructions that are difficult to decipher, such as those using
except
and
not
, or those that use
passive construction
.
An effective set of text dependent questions delves systematically into a text to guide students in extracting the key meanings or ideas found there. As much as possible, do this in an assessment as well.
A well-constructed sequence of questions typically begins by exploring specific
words
,
details
, and
arguments
and then moves on to examine the impact of those specifics on the
text as a whole
.
Along the way they target
academic vocabulary
, some Tier 3 words but mostly
Tier 2
words that our students will see across the curriculum.
Step One: Identify the Core Understandings and Key Ideas of the Text

As in any good reverse engineering or “backwards design” process, teachers should start by identifying the key insights they want students to understand from the text. Keeping one eye on the major points being made is crucial for fashioning a set of successful questions.

Step Two: Start Small to Build Confidence

The opening questions should be ones that help orient students to the text and be sufficiently specific enough for them to answer so that they gain confidence to tackle more difficult questions later on. Fortunately, the order of the standards already does this for us: Key Ideas and Details, Craft and Structure [which includes vocabulary], and Integration of Knowledge and Ideas.

Step Three: Target Vocabulary and Text Structure

Locate key text structures and the most powerful words in the text that are connected to the key ideas and understandings, and craft questions that draw students’ attention to these specifics so they can become aware of these connections. Vocabulary selected for focus should be academic words (Tier 2) that are abstract and likely to be encountered in future reading and studies.

Step Four: Move into the higher DOK levels
with integrating knowledge and ideas.


The Nuts and Bolts of Construction
For your distractors, you can use details from the passage to assess how accurate a student's understanding of the passage is, or how deep his or her understanding goes.
The distractors can be true or at least logical possibilites, but not the answer to the question! This is key. Only one answer can be the answer to the question!! We aren't trying to trick the students when we assess them.
Now that you understand some of the principles of question construction, let's get deeper into the nuts and bolts.
As much as possible, minimize the amount of reading the student must do in a multiple choice question.
Here is an example from KDE of a poorly written stem:

The mean
A. is the most frequently occurring score in a distribution.
B. corresponds to the 50th percentile in the distribution.
C. is the arithmetic average of the scores.
D. is the difference between the highest and lowest scores.

This question requires too much reading time for the student. Reword the stem to present a more meaningful chunk and save the student reading time.

Well-written Distractors:

The mean of a distribution of tests scores is the
A. most frequently occurring score.
B. arithmetic average.
C. 50th percentile.
D. range.

More Poorly Designed Response Choices:

Contrast is a principle of design that refers to major differences between things. Which pair of colors below has the most contrast?
A. red and blue
B. red and orange
C. red and brown
D. black and yellow*.

There are many ways that a response option may stand out from the other options. One way is if one of the options begins with a different word than the other three options.

Well-designed Response Choices:

Contrast is a principle of design that refers to major differences between things. Which pair of colors below has the most contrast?

A. red and blue
B. red and orange
C. black and brown
D. black and yellow*

Additional Design Considerations

•The item stems should be stated in positive terms as much as possible. Item stems such as “Which is not…” should be used sparingly.

•The use of negatives (e.g., “not”) in both the item stem and the answer choices is very confusing.

•Avoid the use of absolute terms (e.g., always, never, all, none, only) in the distractors as much as possible.

Final Words of Advice:

See this as the task that it is, a technical, tedious, time-consuming task . If you know that from the outset, you will avoid frustration. If you train yourself to be a skillful question constructor, you will achieve greater heights in assessing your students' skills and needs. You will have constructed something meaningful...deeply meaningful.
•Whenever possible, avoid answer choices that are mutually exclusive opposites (e.g., living/nonliving, fiction/nonfiction). When such opposites are used, a student’s chance of getting the item correct becomes 1 in 2 versus 1 in 4.

•Avoid “what do you think . . .” because any answer will have to be considered correct.

•Try to keep the stem as short as possible. Statistics show that long question stems tend to cause greater numbers of incorrect responses.

•Number the questions (stems), and use letters for the responses. If there are students who have difficulty distinguishing between lowercase “b” and “d” it is preferable to use capital letters.

•Distribute correct answers so they do not form a recognizable pattern.

•Avoid using language in the question that might accidentally lead students to favor an answer based on language alone.

•When referring to a map, table or figure, label it for easy reference and develop an introductory sentence about the graphic that precedes it on the page.

Follow the graphic with the question stem/responses (e.g., “Use the map of Kentucky to answer the question below.”).

•One question should not give a clue to the answer of another question.

•Avoid questions which have only microscopically fine distinctions between the answers, unless the ability to make these distinctions is the target being assessed.
REVIEW:

Design the
key
and
distractors
.

Provide ONE and only ONE correct answer (key).
Include plausible options that demonstrate a student’s level of understanding.
Consider using common misconceptions or errors as response options.
Avoid humorous or nonsensical response options.
C. Use clear wording/vocabulary that is both age and grade-level appropriate (see #4 above).

D. Maintain a consistent or ‘parallel’ style, length, and visual display.

E. Logically order the response options that include numbers, dates, etc. Numbers should be listed in ascending or descending order. Unless testing the sequence of events from a passage, list options in the order in which they appear in the passage.

F. Avoid using the options “all of the above” and “none of the above.”

G. Have a colleague review them.
If you want to find physical features on a map, use a
A. physical map.

B. products map.

C. political map.

D. population map.

The End of Part II
Full transcript