Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Designing Usable Measurements
Transcript of Designing Usable Measurements
Qualitative Research Specialist Designing Usable Measurements Program started without a strategic plan.
Grantor requires proof of improvement.
Community Capacity to engage is stifled because we lack proof the program works.
We want to grow but don't know the best way to move forward.
We don't know who we are reaching. Determine the Problem Reframe the Problem as a Question How do you keep the lights on?
Bring in Funding to Keep the Lights on.
Why do the lights need to be on?
We want to help people.
Why/how do you want to help people?
They lack resources to help selves.
They need "X" to change their circumstances. What are the GOALS? Write your own Questions:
Remember...Who, What, When, Why and How Identifying the Problem
Determining the Question
Solidifying the Goal
Clarifying the Outcome
Determine the Measurements Order of Today's Operations What makes our program work?
What are we trying to improve/change?
What does the community need to know?
Where has growth occurred in our organization?
Where can we expect growth to occur in the future?
Who we are reaching?
How are we reaching them? Keep the Lights On! We must FIRST know
what we want to know Measures without questions are a
waste of everyone's time. FIGURE OUT WHAT WE ARE DOING "If you can't describe what you are doing as a process, you don't know what you are doing."
- W. Edwards Deming So... is the real QUESTION...
"Will increases in X improve circumstances?" Revisit the Question...WHO, WHAT, WHY, HOW? SUCH AS:
Increase the number of people receiving "X" service by 10% each year.
90% of participants will be employed within 3 months of program completion.
80% of attendees will be satisfied or somewhat satisfied with the program.
All participants will feel prepared to seek employment upon completion of the program. Turn Goals into Measurable Outcomes "Measurement is the first step that leads to control and eventually to improvement. If you can't measure something, you can't understand it. If you can't understand it, you can't control it. If you can't control it, you can't improve it."
- H. James Harrington Determining The Measurements Secondary Data: Those that already exist, such as the Census, or other government collected data.
Primary Data: Those you collect yourself, using tools like an interview, focus group, or survey. Data Sources for your Measurements The Goal of Measurement is Improvement Goal: Increase the number of people receiving "X"
service by 10% each year.
Measurement Question: How many people received
services last year, and how many this year?
Post Survey Question: Did you receive services in
the past 12 months?
Goal: 90% of participants will be employed within 3
months of program completion.
Measurement Question: How many people were
employed three months after finishing program?
Survey Question: Were you employed 3 months
after program completion? Measuring YOUR Outcomes Who are you surveying? (Age, Gender, Length of time in program)
When are you surveying? (Entrance to program, exit from program, 3 months post completion)
How are you surveying? (email, telephone, classroom, USPS)
What will be done with the results? (policy change, program improvement, seek additional funding) WHAT DO YOU KNOW? Write down everything you know about the project, the program and the population to be surveyed. THEN...begin to figure out the problem you need to address. Are the people tired when taking this?
What time of day will they be taking it?
What state of mind will they be?
Will any questions offend them?
If offensive, can it be at the end to avoid influencing other responses?
Are all the questions giving you important information?
Are you using consistent language?
Is reading level no greater than 8th grade?
Will participant speak and read English?
Will some questions influence other questions?
If you ask them to score something, is the scale relative to something they have experience with?
Is there an easier way to say something?
What can you leave out?
How long will it take? 10 questions is about max for volunteer survey with no benefit to the participant. Keep it under 15 minutes. Think About What No One is Thinking About Closed Ended: Only one answer
Open Ended: A variety of responses not foreseen by the researcher (paragraphs or fill in the blank)
Rating: Respondent gives a score or associates a value as a response. (eg. Likert scales, or assign a # 1 thru 10)
Ranking: Respondents puts a list in order of preference, each item receiving a different number. TYPES OF QUESTIONS Forced Selection: No middle number (reduces ambivalent responses)
Response Options: 5 to 7 is common, 6 or 8 is only realistic for an educated audience, 4 is good for a younger audience, 9 or 10 are too many to process.
NOTE: 7 is the most numbers people can keep in their head at one time, so more than 7 could cause response variance. Writing Likert Scale Questions IMPORTANT: Make both sides of center comparable....this is a common mistake. strongly agree agree neither disagree strongly disagree YES
favor like unknown icky really dislike NO Examples for a 5 point scale.... Likert: pronounced Lick-ert
Named for the agronomist who created it as a crop management tool. He resided in Cheyenne Wyoming. Likert Trivia Introductory Lecture on Writing Likert Scale Questions