Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
DTAL: data analysis
Transcript of DTAL: data analysis
mindset and context
e-mail intervention group: control, resources, high standards, mindset
had learning squad (before assignment # 6)
completed assignment #6 (y/n)
activity index (forum contributions + peer reviews)
# assignments completed
course completion (or where dropped)
ideate peer evaluation score (*)
empathy peer evaluation score (*)
VQ: 10 faces
VQ: 20 circles
VQ: quantity vs quality
VQ: creative routine (where/when are you most creative?)
VQ: creative initial education (perception)
VQ: DT can be applied in their work (perception)
problem solving process reflection
mindset reflection (text)
empathize and define
prototype and test
sampling of assignments in different cohorts + 'expert' blind evaluation
Stanford Topic Modeling Toolbox
peer evaluation score 2
peer evaluation score 1
VQ 811: 20 circles: What gets in your way?
VQ 814: creative routines
VQ 804: identifying role: 10 faces
VQ 813: quantity vs quality
VQ 823: initial education and creativity
VQ 824: application of DT to workplace
Other possible approaches for data analysis
Is Productive Struggle predictive of completing the course?
Are VQ811 (20 circles test) and VQ 823 (creative education) correlated?
Is having a learning squad correlated with course completion?
How many students have a rising score (as percentage of max) from first to second peer review, vs no change, vs decrease; and is this correlated with anything (like dropping after second peer review, or MI)
Is any role (10 faces; VQ 804) correlated to peer evaluation 1 or 2?
Is a high activity index (**) correlated with any other index?
resources + high expectations
resources + mindset
experience with design thinking
experience with online courses
creative confidence index (CCI)
mindset index (MI)
productive struggle index
learning about self
learning about interpersonal dynamics
most challenging design thinking stage
confidence to apply
needed validation (Q4, option 7)
auditor, no time constraint (Q1/Q2)
Is the CCI (pre) correlated to 20 circles test (VQ 811)? (or in other terms: is high CCI enriched in the the subset of people who answered in the top two buckets of the 20 circles (16 or more))
Is CCI (pre) correlated to course completion?
Is there a correlation between CCI or MI and experience in DT?
Is the MI correlated to quality vs quantity response (VQ 813)?
Is the CCI (pre) correlated to creative routines (VQ 814)?
Is any of the personal characteristics (occupation, gender, location) over-represented in either extreme of CCI or MI?
Is delta(CCI) correlated with engagement? (** we discussed the definition of "engagement" but don't remember what we settled on)
Is MI predictive of completing the course?
Is someone with a fixed mindset more likely to have dropped the course on/before the stage they found the most challenging?
Is productive struggle correlated to high MI?
OUTPUT THE DATA FOR KATHY ON SPREADSHEEET: Is any of the mail intervention groups more likely to have a high peer evaluation score (2)?
Is there a correlation between needing validation and MI?
CS 224N (Fall): looking for student projects, Chris Manning
Forum activity= # posts
Review activity= # peer reviews
Value of Contribution = votes + helpful reviews
About the students' paths
About the course
How well did the peer evaluation work?
correlation between the staff evaluations (calibrations) and the peer evaluation
did the above agreement change over time (in other words, did students get more "skilled" at evaluating?)
Do we need more raters? (currently 5 required after calibration)
Ideas for the future
formally incorporate self-assessment (and perhaps create flags for intervention when the self-assessment and peer-assessment differ widely in either direction)
with distinction= certain % of helpful reviews and/or forum contribution
fortune cookie feedback
distribution of thread initiations
distribution of posts by votes
Teams (Learning Squads)
to ask in post-survey: "I put sufficient effort into assessing peers"; "peers put sufficient effort into assessing me"
how to design both for independent learners (who do "star-shaped" learning -my term, see Kio Stark's book) and those for whom traditional methods work best
how to measure how many participants are independent learners
Engagement Index= Forum activity + Review Activity
Histogram of Engagement Index (see METRICS below for definition)
Histogram of Value of Contribution (see METRICS below for definition)