Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
SoTL Course: Research Designs: Overview & Evaluation
Transcript of SoTL Course: Research Designs: Overview & Evaluation
for the Social Sciences
Every research design should:
Identify the research problem
justify why you need to study it
Review previously published research
Generate and extend the hypotheses
Explain what kind of data will be needed to test the hypotheses & justify the data collection instruments
Describe what methods of analysis you will use on the data to determine whether the hypotheses are true or false
A blueprint for your research study:
Research questions to be answered
Instruments to be used
Collection and analysis of data
The research problem determines the type of research design you use.
Review a number of different research designs commonly used in Social Sciences.
Advantages & Disadvantages
To investigate a research problem when there are few or no earlier studies to refer to.
To gain insights for later investigation
Can help establish research priorities, generate formal hypotheses, define more precise research problems
Dr. Christina Gitsaki
Center for Educational Innovation
Early Stage Pitfalls
“A question well-stated is a question half-answered”
Collecting data without a well-defined plan
Questions are too general and ambiguous
The research questions are too simple or self-evident
Fuzzy or untestable hypotheses
No proper review of the literature
No theoretical or conceptual framework
Ad hoc research with no generalizability
Exploration, Description, Explanation
The use of iPads in Higher Education.
A description of how the iPads are used in English Foundations Programs in Higher Education
The use of the iPads has a positive effect on learning in higher education settings
Types of Research
to gain ideas and insights
to narrow the scope of the research topic, and
to transform ambiguous problems into well-defined ones
Usually conducted during the initial stage of the research process
to obtain measures to address research questions (research objectives are clearly defined)
Describes attitudes, perceptions, characteristics, activities and situations
Examines who, what, when, where & how questions
Provides evidence that a cause-and-effect relationship exists or does not exist.
Premise is that something (an independent variable) directly influences the behavior of something else (the dependent variable).
Does not make definitive conclusions about the findings
Often unstructured, lacks rigorous standards with limited value for decision making
"On Discourse, Agency, and Buying Eggs"
Describes the current status of a situation & gives a general overview giving points as to what variables are worth testing further
The UAE Population Census is an example of a descriptive study
Conducted in a naturalistic setting
Can collect large amount of data for detailed analyses
Results cannot be used to test a hypothesis using inferential analyses
The descriptive function of the study depends heavily on the instruments for measurement and observation
Does not offer explanations (i.e. cannot ascertain answers to "why")
To investigate cause-and-effect relationships
To understand and answer "Why" questions
To test specific hypotheses
To measure what impact a specific treatment will have
Can be replicated
Not all relationships are causal (unrelated events may appear to be related)
Too many extraneous and confounding variables in a social environment so causality can only be inferred
Sometimes hard to see which variable is the actual cause and which is the actual effect
To explore causal relationships while maintaining control over all factors that may affect the result (Quasi-Experimental Design)
True experimental designs involve a control group as well as an experimental group
Random assignment of subjects
Control and manipulation
True Experimental Designs
Control Group Pre-Test/Post-Test Design:
Control Group Time-Series Design:
Treatment Group: O1------------------X------------------O2
Control Group: O1--------------------------------------O2
Treatment Group: O1------O2-------O3-----------X-----------O4-------O5-------O6
Control Group: O1------O2-------O3------------------------O4-------O5-------O6
2x2 Factorial Design
Studies where more than one variable can vary at a time.
“Poor” Experimental Design
The “One Shot Case Study” (no control)
Weak on internal validity or causal assessment
Often used for measuring change or innovation in education but it is misleading
Conclusions are impressionistic
Treatment Group: X------------------O2
Minimal Control Experimental Design
One-Group PreTest-PostTest Design
No assurance that X is the only factor in O1-O2 difference. For example:
Treatment Group: O1----------X-----------O2
Issues in Cause & Effect Relationships
Assuming a cause-effect relationship exists between a treatment and an outcome is difficult with weak research designs because of inability to control extraneous variables (Isaac & Michael, 1997; Sechrest, Perrin, & Bunker, 1990; Schwandt, 2007).
A cause-effect relationship more likely exists when:
Changes in the outcome is associated with changes in the DV
A statistically significant relationship is established between IV & DV
Appropriate time order exists: treatment was before assumed outcome
The treatment is the most reasonable explanation for causing DV changes; (Campbell & Stanley, 1966; Cook & Campbell, 1979; Guba & Lincoln, 1989; Isaac & Michael, 1997; Lipsey, 2007; Sechrest, Perrin, & Bunker, 1990).
Identify direct causal relationships
Limit alternative explanations
Artificial settings of experiments may change subjects’ behaviour
Unsuitable for studying a problem because of ethical or logistical reasons
Action Research Design
To develop new skills or new approaches and to solve problems with direct application to the classroom.
Initially an exploratory study to understand the problem
Plans for an intervention
Observations are collected
Action Research Design
Figure 1: Action Research Protocol after Kemmis (cited in Hopkins, 1985)
Action Research Design
Focuses on pragmatic and solution-driven research
Direct relevance to practice
Research Cycle = Learning Cycle – The researcher learns consciously from the experience (PD model)
Time-consuming, complex to conduct and taxing on the researcher/teacher
Personal over-involvement of the researcher may bias research results
Harder to write up and report findings effectively
To investigate patterns and sequences of growth and change as a function of time:
A type of observational study.
Follow the same sample over time and make repeated observations.
Changes over time are related to variables that might explain why the changes occur.
Can help establish the direction and magnitude of causal relationships.
Enables the description of patterns of change over time.
Can facilitate the prediction of future outcomes based upon earlier factors.
Can take a long time to gather results.
Requires great financial support to sustain the project over a large period of time.
The data collection method may change over time resulting in loss of continuity.
The sample may change over time (e.g., attrition).
To measure change by drawing samples of different subjects
Groups are selected based on existing differences
Can provide a “snapshot” of the outcome at a specific point in time (no attrition problems)
Collect samples from different groups of students who are at different stages of proficiency to describe how their vocabulary has developed over time.
Finding subjects that are very similar apart from one specific variable can be difficult.
Differences may be artifacts of the sampling process.
Study is bound to one point in time so there is the possibility that results could be different if another time-frame was chosen.
To observe subjects where the researcher has no control over the experiment.
Overt Observational Research:
the researcher identifies himself and explains the purpose of the research.
Covert Observational Research:
the researcher does not identify himself
(ethical issues), mixes with the subjects or observes from a distance.
The researcher participates in what they are observing so as to get a finer appreciation of the phenomena.
Types of Qualitative Observational Research
Short Term Observation
Flexible, not structured around any specific hypotheses
Able to collect in-depth information about a particular (complex) behaviour
Can help discover what variables are important before applying an experiment
The in-depth data may be specific to the sample and thus not readily generalizable
The researcher may only “see what they want to see”
Can’t determine causal relationships since nothing is manipulated by the researcher
Subjects’ behaviour may be altered because of the presence of the researcher (Heisenburg Uncertainty Principle)
Case Study Design
In-depth study of a particular research problem.
To study intensely the background, current status, and environmental interactions of a given social unit: an
May be descriptive or explanatory.
Can help understand a complex behaviour
Can apply a variety of methodologies
Can provide detailed descriptions of specific and rare cases
Can bring to light important variables
Case Study Design
Provides little basis for generalizing the findings to a wider population
The intense focus may bias the researcher’s interpretation of the findings
Generally speaking, the design does not facilitate well the assessment of causal relationships
A study carried out in a deliberate, staged approach where one stage will be completed, followed by another, then another, and so on.
Each stage builds upon the previous one until enough data are gathered to test the hypothesis.
Sample size is not predetermined.
Can repeat the data collection, can make minor changes and adjustments to correct and hone the research design (useful design for exploratory studies).
Because the study is conducted serially, the results of one sample are known before the next sample is taken and analysed.
To collect, verify and synthesize evidence from the past to establish facts that test a hypothesis.
Uses secondary sources and a variety of primary documentary evidence (e.g. logs, diaries, reports, records, archives, etc.)
Emirati students’ performance on the CEPA over the past 13 years and the correlation of their CEPA results with success in post-secondary education.
It is unobtrusive: the research does not affect the results of the study.
No researcher-subject interaction that can affect the findings.
Historical sources can be used over and over to study different problems.
There may be gaps in the research due to lack of specific historical documents.
Using secondary sources: original authors bring their own interpretation and biases of past events.
": look at the data trying to artificially create hypotheses
Surveys, score records
Interviews, observations, focus groups, analysis of artifacts, diaries, logs, texts
Both quantitative & qualitative methods
Triangulation of measurement
Critical Questions to Consider
What is the research problem?
What is the population?
How large a sample do I need?
Which research design is best?
What instruments to use?
When to collect the data?
How to analyse the data?
Topics for Research:
Technology Integration in Higher Education Institutions.
How do College instructors use technology to address the learning needs of their students?
The use of technology in higher education improves student learning.
Things to Consider when choosing a Research Design:
Access to students
Reliable Data Sources