Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

DTAL: data analysis

framework
by

Leticia Britos Cavagnaro

on 26 January 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of DTAL: data analysis

lecture 1:
mindset and context
e-mail intervention group: control, resources, high standards, mindset
had learning squad (before assignment # 6)
completed assignment #6 (y/n)
activity index (forum contributions + peer reviews)
# assignments completed
course completion (or where dropped)
ideate peer evaluation score (*)
empathy peer evaluation score (*)
VQ: 10 faces
VQ: 20 circles
VQ: quantity vs quality
VQ: creative routine (where/when are you most creative?)
VQ: creative initial education (perception)
VQ: DT can be applied in their work (perception)
player?
problem solving process reflection
(graphic)
mindset reflection (text)
work/study space
(photo)
lectures 2-4:
design challenge
empathize and define
ideate
prototype and test
lecture 5:
application
takeaways
process map
application reflection
sampling of assignments in different cohorts + 'expert' blind evaluation
text analysis
PCA
Stanford Topic Modeling Toolbox
http://nlp.stanford.edu/software/tmt/tmt-0.4/
peer evaluation score 2
peer evaluation score 1
VQ 811: 20 circles: What gets in your way?
VQ 814: creative routines
VQ 804: identifying role: 10 faces
VQ 813: quantity vs quality
VQ 823: initial education and creativity
VQ 824: application of DT to workplace
Questions
Other possible approaches for data analysis
Is Productive Struggle predictive of completing the course?
Are VQ811 (20 circles test) and VQ 823 (creative education) correlated?
Is having a learning squad correlated with course completion?
How many students have a rising score (as percentage of max) from first to second peer review, vs no change, vs decrease; and is this correlated with anything (like dropping after second peer review, or MI)
Is any role (10 faces; VQ 804) correlated to peer evaluation 1 or 2?
Is a high activity index (**) correlated with any other index?
who is
profile
pre-survey
(18000)
post-survey
course data
email intervention
control
resources
resources + high expectations
resources + mindset
occupation
age
gender
location
discipline
main language

experience with design thinking
experience with online courses
motivation
creative confidence index (CCI)
mindset index (MI)
team-work preference

connectivity index
productive struggle index
learning about self
learning about interpersonal dynamics
most challenging design thinking stage
confidence to apply

engaged (3600)
non-engaged (4600)
needed validation (Q4, option 7)
auditor, no time constraint (Q1/Q2)

Is the CCI (pre) correlated to 20 circles test (VQ 811)? (or in other terms: is high CCI enriched in the the subset of people who answered in the top two buckets of the 20 circles (16 or more))
Is CCI (pre) correlated to course completion?
Is there a correlation between CCI or MI and experience in DT?
Is the MI correlated to quality vs quantity response (VQ 813)?
Is the CCI (pre) correlated to creative routines (VQ 814)?
Is any of the personal characteristics (occupation, gender, location) over-represented in either extreme of CCI or MI?
Is delta(CCI) correlated with engagement? (** we discussed the definition of "engagement" but don't remember what we settled on)
Is MI predictive of completing the course?
Is someone with a fixed mindset more likely to have dropped the course on/before the stage they found the most challenging?
Is productive struggle correlated to high MI?
OUTPUT THE DATA FOR KATHY ON SPREADSHEEET: Is any of the mail intervention groups more likely to have a high peer evaluation score (2)?
Is there a correlation between needing validation and MI?
creative confidence
mindset
other
readouts
CS 224N (Fall): looking for student projects, Chris Manning
Metrics
Engagement
Forum activity= # posts
Review activity= # peer reviews
Value of Contribution = votes + helpful reviews
About the students' paths
About the course
How well did the peer evaluation work?
correlation between the staff evaluations (calibrations) and the peer evaluation
did the above agreement change over time (in other words, did students get more "skilled" at evaluating?)
Do we need more raters? (currently 5 required after calibration)


Assessment
Ideas for the future
formally incorporate self-assessment (and perhaps create flags for intervention when the self-assessment and peer-assessment differ widely in either direction)
with distinction= certain % of helpful reviews and/or forum contribution
fortune cookie feedback
Forum
distribution of thread initiations
distribution of posts by votes


Teams (Learning Squads)
to ask in post-survey: "I put sufficient effort into assessing peers"; "peers put sufficient effort into assessing me"
how to design both for independent learners (who do "star-shaped" learning -my term, see Kio Stark's book) and those for whom traditional methods work best
how to measure how many participants are independent learners

Engagement Index= Forum activity + Review Activity
Distributions
Histogram of Engagement Index (see METRICS below for definition)
Histogram of Value of Contribution (see METRICS below for definition)
Full transcript