Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Stages of Test Development

No description
by

on 26 September 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Stages of Test Development

STEP 1: Stating the problem
Answer these questions:
STEP 2: Writing specifications
Content
STEP 3: Writing and moderating items
Sampling
Choose what to include
Writing Items
Imagine misinterpretations/other correct answers
Key
Moderating Items
Scrutiny of proposed items by 2 collegues

STEP 4: Informal trialling of items on native speakers
"Items that have been through the process of moderation should be presented in the form of a test(s) to a number of
NATIVE SPEAKERS
" (p. 63).
STEP 5: Trialling of the test on non-native speakers
Items that have survived moderation and informal trialing on NS should be put together into a test to now trial with
NNS
.
STEP 6: Analysis of results of the trial; making of any necessary changes
2 types of analysis:
statistical
and
qualitative
STEP 7: Calibration of scales
What is calibration?
The term calibration means just the act of comparison.
What is the purpose of calibration?
The purpose of calibration is to ensure that a measuring device is accurate, reliable and consistent

*To test the test that been developed and also to ensure that tools is running efficiently and perfected.

STEP 8: Validation
STEP 10: Training staff
GOAL!
Stages of Test Development (ch. 7)
By Akim S., Caitlin S.
& Ruth A.

STEP 9: Writing handbooks for test takers, test users and staff
Description of Today’s Presentation
Warm-up
Detail the 10 steps
Activity: Moderate an Item
Assessment: Fill in the Stages
Warm-up
Can you put the stages of test development in order?

What do you want to know and what is the purpose of this test?
What kind of test is this? (i.e. placement, achievement, etc.)
What abilities are being tested?
How detailed/accurate must the results be?
How important is backwash?
What constraints are there? (i.e. time, money, etc.)
Gather similar tests given in the past
Operations
Structural Range
Types of text
Vocabulary Range
Adressees of text

Dialect, Accent, Style
Length of text
Speed of Processing
Topics
Readability
STEP 2: Writing Specifications for the Test (Continued)
Structure, Timing, Medium/Channel and Techniques
Test structure
Number of items
Number of passages
Medium/channel
Timing
Techniques

Criterial Levels of Performance:
E.g. 80% = mastery
Scoring Procedures:
Rating scale
Raters
Reliability/Validity

ACTIVITY
Moderate this item. Would you keep it, reject it, or remedy it? How and why?
• _____Writing specifications for the test.
• _____Writing and moderating items.
• _____Stating the problem.
• _____Informal trialling of items on native speakers.
• _____Trialling of the test on a group of non-native speakers similar to those for whom the test is intended.
• _____Training staff.
• _____Validation.
• _____Analysis of results of the trial; making of any necessary changes.
• _____Calibration of scales.
• _____Writing handbooks for test takers, test users and staff.
Number them from 1 to 10 according to their order in the chapter.
Correct order:
1. Stating the problem
2. Writing specifications for the test
3. Writing and moderating items
4. Informal trialling of items on native speakers
5. Trialling of the test on a group of non-native speakers similar to those for whom the test is intended
6. Analysis of results of the trial; making of any necessary changes
7. Calibration of scales
8. Validation
9. Writing handbooks for test takers, test users and staff
10. Training staff

Do you make your own tests or do you take them from somewhere/someone else?
If you make your own tests, do you ever follow any of these steps?
Is there any other order in which you follow these steps?
What other steps would you include? and why?
A little bit of discussion before going into the chapter...
*20 or more.
*similar age, education, general background.
*No experts.

Statistical:
reveal qualities (reliability) of the test.
Qualitative:
analysis of the responses in order to discover misinterpretations and other indicators of faulty items.
What is validation?
To validate is to prove that something is based on truth or fact, or is acceptable
Why validation is important?
1. The greater a test’s content validity, the more likely it is to be an accurate measure of what it is supposed to measure.
2. Such a test is likely to have a harmful backwash effect. Areas that are not tested are likely to become areas ignored in teaching and learning (Hughes, p.27).

Two kinds of validation
A. Concurrent validation ( achievement test)

B. Predictive validation (compering the current result with the future result).

1. The rationale for the test ( reason)
2. Description of the stages of the test
3. Description of the test
4. Sample items or sample tests
5. Advice ( how to prepare)
6. An explanation of test score
7. Training materials ( website, interviewer, etc.)
8. Information of test administration

What happens when test developers write items with no specifications?
"Item writers may produce good times in the technical sense even without the heel of specifications, but they will produce different kinds of items in terms of what is asked in each question and what kind of cognitive and textual processing each question requires." (p. 71)

"With adequate training and experience in test-development under the condition of being given specifications and adequate moderation, different item writers can construct test items that are more homogenous in terms of which sub skills they measure." (p. 71)

Jafarpur, A. (2003) Is the test constructor a facet?
Language Testing
, 20(1), 57-87. doi: 10.1191/0265532203lt244oa
1. Idaho is widely known as:
a. The largest producer of potatoes in the us
b. The location of the tallest mountain
c. The state with a beaver on its flag
d. The treasure state
2. Which of the following artists is known for painting the ceiling of the sistine chapel?
a. Warhol
b. Flinstone
c. Michaelangelo
d. Santa Clause
All test designer must be train ( what is the purpose of the test and what are the target skills)
They should know how long the test should be ( 50 minutes or 60 minutes and how long to answer question)
Full transcript