Social Research as Part of Strategic Planning

Recommendations for Layout and Design Pt. 2

Other Factors Affecting Validity

Testing

• Keep instructions and questions stylistically consistent and distinguishable

• For open-ended questions, the size of the response box should correspond to the desired response length

• If you only ask one concluding demographic question, ask for their zip code; you can then ignore responses from outside your service area

• Questions and instructions must be worded carefully
• If not carefully crafted, a survey is at best uninformative and at worst misleading
• Wide distribution is necessary to avoid sampling bias
• Sample size is key - you can't generalize accurately if too few people participate

"The possibilities for misunderstanding are endless, and no researcher is immune." (Earl Babbie, The Practice of Social Research)

Before deploying your survey in the wild, test it!

Have several people from outside the library world and from different demographic strata (age, education level, etc.) take your survey.

Ask about the experience. Find out what was confusing or incomprehensible.

Look at the results and see if they correlate to your expectations.

If everything's jake, your survey is ready!

Getting Community Input Through Online Surveys - Methodology, Implementation, and Analysis

What Are Surveys Made of?

Open-Ended Questions are those in which the respondent provides their own answer.

Example: "What kinds of programming would you like the library to offer?"

Advantages: Can bring to light information you had not considered. Can garner anecdotal evidence.

Challenges: Data provided is non-uniform, open to interpretation, difficult to quantify, and potentially irrelevant.

Recommendations for Layout and Design

• Open-Ended Questions

• Closed-Ended Questions (choose one OR choose any)

• Opinion Statements (Likert Scale, Yes/No, True/False)

Closed-ended questions are those where respondents choose from answers you provide.

Variants: Choose one of the following (multiple choice); Choose any of the following (check box).

Example: Which best describes how frequently you visit the library?Choose one: Daily, Weekly, Every other week, Monthly, Less often

Advantages: Responses are standardized and easy to quantify.

Challenges: Options must be mutually exclusive and exhaustive. There can be a bias towards normalcy (picking the middle choice). You may overlook important responses.

• Black text on light/white background
• Minimal is optimal - no graphics or distractions
• Easy to read large point fonts are best
• Minimize scrolling and clutter - one question per page (except for opinion statements)
• Question order can influence responses - ask open-ended questions first
• Ask demographic questions last - they diminish interest in participation

Recommendations for Question Formulation

Selected Online Resources

Likert Scale Statements: Statements of opinion which you ask respondents if they agree or disagree with. Named for the social researcher Rensis Likert.

Example: The library's service hours meet my needs.

Strongly Disagree Disagree Agree Strongly Agree N/A

Advantages: Responses are standardized and easy to quantify. Likert Scales are a tried and true method of gauging popular opinion.

Challenges: Respondents can be tempted to run a column. The way you word the statement can influence the outcome.

Distributing and Promoting

Your completed community vision and assessment tool reveals potential partners and other opportunities to meet community needs. It may also reveal threats (declining population, competing services, etc.)

Your survey results will indicate present strengths and weaknesses. Threats to your existing service models may also be revealed.

Your space needs assessment will indicate one very significant strength or weakness.

And now to contemplate the quality of some sample survey questions...

Opinion Statements (Likert Scale):

• Use the same scale throughout, if possible

• Never reverse the sequence of the scale

• You can group Likert statements together on the same page

• If you have a lot, separate them into groups and break them up with other question types to alleviate survey fatigue

• Don't use more than a five-point scale; research indicates that providing more options tends to bewilder respondents

Other Sources for SWOT

Advantages of Online Surveys

Look to the NDLCC's recently published standards for ND Public Libraries to identify weaknesses:

http://library.nd.gov/publications/NDLCCstandards.pdf

You can further identify concerns in the area of technology services that you might plan to address by looking at the Edge Benchmarks:

http://www.libraryedge.org/benchmarksv1

Once you have your SWOT mapped, you can start planning to maintain your strengths, take advantage of opportunities, remedy weaknesses, and address threats.

Space Needs Assessment

Closed-Ended Questions:

• Options provided must be exhaustive - no one should want to give an answer that's not listed

• Options must be mutually exclusive - for single response questions, no one should want to provide more than one answer; for multiple response questions, options should never overlap

Source: http://commons.wikimedia.org/wiki/File:SWOT_pt.svg

In general:

Keep instructions and questions clear and brief.

Your respondents should be able to read each item quickly, understand its intent, and provide an answer without struggling.

Conducting a space needs assessment will also inform your strategic planning efforts.

A space needs assessment estimates if your library's current space is sufficient to meet foreseen future needs (like those you identified in your survey).

The following guidelines are from the International Federation of Library Associations and Institutions...

To avoid sampling bias, you want your survey distributed as widely as possible throughout your service area.

• Host it PROMINENTLY on your website
• Promote it through social media channels
• Advertise it in your paper and on their website
• Place links and information on your city or county websites, as applicable
• Work local fairs and events - if your survey is too long for people to tap through on a tablet, use a separate abbreviated tool for field interviews
• Consider making paper forms available at a few choice locations throughout the community

• Done properly, surveys are the best method of collecting original data describing a population you can't observe directly

• Surveys are excellent tools for measuring the attitudes and orientations of large populations

• Online surveys are cost effective and automatically populate spreadsheets with their data

• Participants perceive online surveys to be more brief, enjoyable, interesting, and confidential than other survey methods

• Increasingly, people are already online, so you're targeting them where they're at

Open-Ended Questions:

• If you're going to ask open-ended questions, ask them early

• Don't ask many of them - they're too much work for both you and your respondents

• Express what you want clearly, but provide examples only when absolutely necessary - they are often parroted, and can subsequently skew your results

Add together the following:

• Number of volumes /100 (or /140 for high-density storage)
• Number of magazine titles to be displayed /10
• Number of magazines titles to be held in back issues /20 then multiply by the number of years to be retained
• Number of A/V recordings /100 for low-density storage; /120 for medium-density storage; or /150 for high-density storage
• Number of public use computer terminals x4 in a small installation or x3.25 in a larger installation
• Number of reader seats x3
• Number of staff work stations x12.5 in a smaller institution or x 10 in a larger institution
• Number of seats in a lecture hall
• Number of seats at a conference table x3
• Number of seats in an instruction room x3
• Add an allowance for any additional types of meeting rooms needed
• Add an allowance for special use or miscellaneous functions (storytime room, photocopier, FOL gift shop, etc.) - the book Building Blocks for Planning Functional Library Space can help with this
• Estimate an allowance for mechanical and support services by adding all of the above allowances and dividing that total by 4 for a minimal allowance (divide by 2 for a generous allowance)

In sum, these allowances provide an estimate of your library's space needs in square meters. To convert to square feet, multiply the total by 10.764.

DATA ANALYSIS

Sample Size and Confidence

Confidence Interval: Essentially a margin of error (+/- x%). This means that if you'd asked everyone in the population area, instead of merely your sample, the actual result would be within this percentage of what you arrived at.

Confidence Level: If you have a 95% confidence level, this means that if you randomly asked anyone in the population area, 95% of the time, their response would fall within the confidence interval of your result.

Finding Your Desired Sample Size: A good target to shoot for is a 95% confidence level with a 4% confidence interval. If you're in a large service area, you could easily attain a smaller confidence interval; if you're in a small one, you may have to settle for a larger one.

Notes and Caveats

The ways of categorizing data are limitless. Focus on those applicable to planning: things you can take action to address.

One response may fall under more than one category. "I wish you offered a mystery book club that started after 8pm" could be categorized as both More Adult Programming Needed and Extend Evening Hours.

Your categories should be mutually exclusive. You don't want to have both More Programming Needed and More Teen Programming Needed as categories. You can always clump like items when summarizing your results, but it makes precious little sense to redundantly codify data.

Once your results are in, you can use the calculator here to determine your actual confidence interval at either a 95 or 99% confidence level: http://www.surveysystem.com/sscalc.htm

Miss the mark? Don't despair. Your survey likely still provides you with useful data.

Confidence intervals are most significant for coin-flip responses. Mercifully, you're not trying to predict an election outcome. You're working to identifying trends and commonly held opinions.

Here's a broad example: If the following service were offered by the library, someone in my family would make use of it. Check all that apply:

Extended weekday evening hours 10%

Extended weekend hours 12%

Additional adult book club 1%

Additional computer classes 22%

Additional storytimes 2%

Additional teen game events 20%

Music production space 1%

Video production space 8%

Downloadable e-books 24%

Even if your confidence interval winds up a relatively abysmal +/-8% for the above results, you've still clearly identified three service needs, ruled out three possibilities you were considering, and found three strong contenders.

While you may not be able to say with certainty which is the most popular response for your community, you do have a pretty clear top three.

Printing Your Survey Form?!

Here is the sample size you'll need to have a 95% confidence level with a 4% confidence interval based on the population of your service area:

Population Sample Size

1,000 375

1,500 429

2,000 462

2,500 484

3,000 500

4,000 522

5,000 536

7,500 556

10,000 566

12,500 573

15,000 577

20,000 583

25,000 586

30,000 588

40,000 591

55,000 594

65,000 595

110,000 597

If you need to calculate a sample size for a different confidence level, interval, and/or population you can do so here: http://www.raosoft.com/samplesize.html

Codifying and Evaluating Open-Ended Responses (quantitative analysis of qualitative data)

Wait, isn't this an online survey? Yes. But one way to avoid sampling bias is to provide multiple ways to participate.

This step is totally optional, but some may wish to distribute print surveys, as well.

If you do, make them available at a few publicized locations throughout the community alongside drop boxes for completed forms.

Paper responses will still have to be entered into your spreadsheet from the completed forms. You can do this directly or through your online form.

First: examine your compiled responses for thematic trends; note that there will almost always be outliers

Second: Devise categories corresponding to identified trends

Third: Quantify by assigning each response to one or more categories

Fourth: Enter categories and their tallies into your spreadsheet for easy visualization and reference

Fifth: When publishing results, include both actual responses and how you interpreted/quantified them

Error of Ambiguity

Tally numerical data with:

=SUM('SHEET NAME'! X2:X)

Tally text responses from a multiple choice or checkbox question with: =arrayformula(count(iferror(find("RESPONSE CHOICE";'SHEET NAME'! X2:X))))

Creating Your Survey Form

Get started here: https://drive.google.com/

Creating an Online Survey Using Google Docs

Explanation: this counts each occurrence of RESPONSE CHOICE in SHEET NAME's column X from row 2 to the end, evaluating each cell as an array of responses; non-matching terms (errors) are counted as zeroes and matching terms are counted as ones.

You will have to use a separate formula to tally each RESPONSE CHOICE (mostly just cutting and pasting).

SHEET NAME is almost always Form Responses.

Each column corresponds to a different question.

A common thing you may want to tabulate is the % of respondents who chose a given RESPONSE CHOICE. Simply divide the figure you tabulated above by one less than the terminal row number of your Form Responses sheet (you subtract one for the header row).

Things to remember when creating a form:

• Give it a title

• Choose the response destination ("New spreadsheet")

• Don't check "Show link to submit another response"

• Customize the confirmation message! Including a link back to the survey's page on your website (or to the survey itself). Encourage participants to share the link with their friends, family members, and colleagues. Thank them for their time.

You need a Google account to create survey forms and manage the data collected with them

Google accounts are free and easily acquired

If you don't have one, create a generic library or library director account

Those filling out the forms DO NOT need Google accounts; they don't

have to sign in or otherwise identify themselves in any way

Before You Create Your Survey Form

Data Processing for Analysis and Visualization

Why Google Docs?

Plan your survey instrument out. Have a clear idea what you want more information about, how you want to solicit it, and what order you want to ask your questions in.

Write out the purpose of the survey. Post this either on the first page of the survey or on the web page you'll be directing your participants to.

The time to revise and perfect is before you go live. Never change your survey instrument once you've started collecting data.

Once you're done collecting data (and you've closed your survey - to do this, open the form, click on Responses, and uncheck Accepting Responses), you're ready to make sense of the results!

Google's automated Summary of Responses may actually be sufficiently detailed for your purposes.

Free and easy to create

Allows a virtually unlimited number of respondents

Google's forms are mobile-friendly

Easy to share via social networks and e-mail, and embed on your site

Boast an ample array of question formats and allow logic branching

Facilitate fast and flexible analysis of responses

You can export data to Excel (.xlsx), OpenDocument format (.ods), or .csv

Not convinced? Compare to other survey apps here: http://goo.gl/KsRNN

If not, here are some Google Spreadsheets formulas to get you started:

A statement is ambiguous if it has two incompatible and unrelated meanings. Ambiguity can be lexical (single word with more than one meaning) or syntactic (part of the statement has more than one potential referent).

"Child's Stool Great for Use in Garden" (Lexical)

"I once shot an elephant in my pyjamas. How he got in my pyjamas, I'll never know." (Syntactic)

Ambiguity is the nemesis of accuracy. Context can eliminate it.

Common Errors in Question Formulation

However, certain gaps still exist which can introduce a sampling bias that you should be aware of. Here's the latest Pew Research data:

Error of Using Inscrutable Jargon

Error One: Ambiguity

Error Two: Jargon

Error Three: Double-Barreled Questions

Error Four: Double Negatives

Error Five: Inherent Desirability or Social Stigma

Don't use library jargon, acronyms, or techie terms!

If you must use a field-specific term, explain it clearly and concisely immediately before asking about it.

Library jargon: ILL, ILS, OPAC, PAC, ODIN, NDLCC, CIPA, filter, periodical, serial, special collection, archive, card catalog, bib. record, automation, database...

Error of the Double-Barreled Question

Be wary of interrogators bearing conjunctions

Example: The library has convenient weekend and evening hours. Choose one: Agree Disagree Don't know

If a significant percentage of respondents disagree, does this indicate a need to increase your weekend hours, your evening hours, or both?

If a question on your survey contains the word AND, you should almost certainly rewrite it as two or more distinct questions.

Error of Double Negatives

Error of Loaded Language

The inherent desirability and social stigma of terms significantly influences responses.

Studies have shown that people respond differently to alternate phrasings of the same question.

Avoid terms that trigger knee-jerk responses.

Negative terms usher in confusion, especially for yes/no or Likert scale questions.

Double negatives confuse respondents and the interpretation of their responses.

Be wary of any questions with the words "not" or "no" or any words with the prefixes un-, in-, a-, or non-.

Avoid negation whenever possible!

designed by Péter Puklus for Prezi

Survey Methodology:

• Hunter, L. (2012). Challenging the reported disadvantages of e-questionnaires and addressing methodological issues of online data collection. Nurse Researcher, 20(1), 11-20: http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=80440695&site=ehost-live
• Kupersmith, John. "Library Terms that Users Understand": http://www.jkup.net/terms.html
• Okazaki, S. (2007). Assessing mobile-based online surveys. International Journal Of Market Research, 49(5), 651-675: http://search.ebscohost.com/login.aspx?direct=true&db=buh&AN=26611408&site=ehost-live
• Puleston, J. (2011). Improving online surveys. International Journal Of Market Research, 53(4), 557-560. doi:10.2501/IJMR-53-4-557-562: http://search.ebscohost.com/login.aspx?direct=true&db=buh&AN=63169450&site=ehost-live
• Wood, E., Nosko, A., Desmarais, S., Ross, C., & Irvine, C. (2006). Online and traditional paper-and-pencil survey administration: Examining experimenter presence, sensitive material and long surveys. Canadian Journal Of Human Sexuality, 15(3/4), 147-155: http://search.ebscohost.com/login.aspx?direct=true&db=aph&AN=25455891&site=ehost-live

Sample Surveys:

• http://www.lrs.org/library-user-surveys-on-the-web/#Public_Library_
• http://www.eric.ed.gov/PDFS/ED543074.pdf (go to p. 148)
• http://nmstatelibrary.org/index.php?option=com_content&view=article&id=206&Itemid=104 (scroll to General Questions)
• http://goo.gl/cL9NCY

Space Needs Assessment:

• Designing Libraries: http://www.designinglibraries.org.uk/
• Key Issues in Building Design by the Library Buildings and Equipment Section of IFLA: http://www.eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED510051

Trends:

• Pew Internet & American Life Project's studies related to libraries and library use: http://libraries.pewinternet.org/

North Dakota State Library - Summer Breeze 2013

Social Research as Part of Strategic Planning

Selected Print Resources

http://goo.gl/pWcVUB

• Babbie, Earl R. The Practice of Social Research (13th ed.). Belmont, California: Cengage Learning, 2012.
• Library Leadership and Management Association, American Library Association. Building Blocks for Planning Functional Library Space (3rd ed.). Lanham, Maryland: Scarecrow Press, 2011.