Question

What about your concepts?

Americans for Democratic Action

Kucinich: 95%

Serrano: 100%

Gutierrez: 90%

Shuler: 45%

Alexander: 0%

Mulvaney 25%

We're somewhere in between...

There are standards of measurement in certain subfields, but in others, it is a bit squishier

Question

Lit Review

Theory &

Hypothesis(es)

• Judicial activism
• Congressional roll-call liberalism/conservatism
• Political legitimacy
• Political sophistication
• Social capital

Experimental or

Observational

How do we measure???

Very rarely a consensus on how to measure

• Economics

• Psychology

Consequences of Poor Measurement

Problems

If the variables we observe in the real world do not mirror the abstract concepts, then that affects our ability to evaluate our theory’s empirical support.

Conceptual clarity

Reliability

Between bias and reliability, which is the lesser of the evils?

Bias

If how we operationalize does not reflect what we actually observe, we have little confidence in our results.

Validity

Internal

• Consistent measures of your independent and dependent variables AND the degree of confidence that your independent variable causes movement in your dependent variable

Final Result:

Obama 51, Romney 47

64% of registered voters voted in the 2012 presidential election.

External

• Confidence in results apply not only to participants in study, but also to broad population

• To have strong external validity, you need a probability sample of subjects or respondents drawn using chance methods from a clearly defined population.

Pros and Cons

How to Access Validity

Face Validity

Pros and Cons

Content Validity

• Does it have all the elements?

• Democracy

Controversy:

How to Measure Roll-Call Behavior

Construct Validity

• Does the measure reflect already existing information about the relationship being studied?

Pros

Cons

Pros

Cons

NOMINATE in the 112th House

NOMINATE

ADA

• the scores tell us who has similar voting records, but does not tell us why
• ideal point estimations are not interchangeable with ideology or partisanship

Kucinich Serrano (NY-16)/Gutierrez (IL-4) Shuler Alexander Mulvaney

(OH-10) -0.779 -0.501 (NC-11) -0.048 (LA-5) 0.50 (SC-5) 1

-1 -0.5 0 0.5 1

• ADA scores are widely known among members and publicized in their districts

• broad set of data relating to legislator preferences
• complete set of roll-call votes from a Congress
• arrange legislators in order of their ideology and identify votes on which they appear to have abandoned their ideological predisposition

• Spatial measure of estimation of House member’s and Senator’s ideal points relative to one another
• Based on two dimensions and estimated across a series of roll-call votes in each congressional session
• Based on a scale from -1 (liberal) to +1 (conservative)

• The 20 most important votes during that session are evaluated.
• Each member receives 5 points if he/she voted with ADA, and does not receive 5 points if he/she voted against or was absent.
• The total possible is 100

• not necessarily comparable over time or across chamber
• ADA scores are simply the addition of votes
• shifting scale form year to year
• Could over or underestimate the influence of ideology