Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Development of a Framework to Classify MOOC Discussion Forum

No description
by

on 5 December 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Development of a Framework to Classify MOOC Discussion Forum

Development of a Framework to Classify MOOC Discussion Forum Posts: Methodology and Challenges
Background
Collaborative Discussion
social constructivist learning
provides rich medium through which to observe cognitive processes
discussion forums - already transcribed!
strategies leading to productive outcomes are under study

6.002x Discussion Forum
12,696 'threads' tagged by topic
Threads initiated by asking a question / making a comment
Threads could be searched by key words
Students could add to threads by posting a reply, a comment, or another related question
96,696 posts from March - November 2012
Participation in forum was optional
52% of students who earned certificates participated in forum
Research Question
What is the topic of students’ posts and the role assumed by posters on an open discussion forum?
Answering this question will help us to
determine characteristics of productive dialogue
explore relationship between type of dialogue used, role, and achievement or persistence in course

Provided general awareness of forum content
Wide-ranging findings
advice for studying
questions about course content
commiseration with fellow students
requests for formation of outside groups
declarations of physiological states, e.g., hunger / sleep deprivation.
Step 1:
Conduct open coding of randomly selected forum posts
Step 2:
Develop tentative list of descriptive codes
Initial list
Content
User interface / course structure
Community building / interpersonal
Nonspecific affective
Step 3:
Trial codes
Trial 1
Procedure: 5 researchers coded randomly selected posts
Results:
Expanded list of codes to include additional topics
Added second dimension - role of poster
Refined code definitions
Trial 2
Procedure: 5 researchers coded 500 posts
200 per researcher
Overlapped with 2 different peers
Results:
Inter-rater reliability
75-91% for topic
76-86% for role
Further refined code definitions
Creation of codebook with examples
Product
Two dimensional coding framework classifying
posts by
Topic of post
Content
Other coursework
Social / affective
Course website / technology
Course structure / policies
Other
Data missing
Non-English
Role assumed by poster
Help-seeker (information seeker)
Help-giver (information giver)
Other
Decisions
Posts could be given dual codes for topic & dual codes for role
"One thing I'm doing while working on this course is to observe my own learning....I seem to learn more from books than from lectures. It does seem now that while I have learned a lot from our excellent video lectures, I still had to rely on the text to finally 'nail down' some of the material. It just seems more ‘continuous’ to me....I would be interested in hearing others' observations about how their favorite mode of learning feels to them. Why it works best for them. Why they favor videos (or not)... and so on. Thanks.” [this is topic: 1 role 1,2; topic 3]
Posts would be coded independent of the thread in which they appeared
Phase 2:
Coding by independent coders
Method
Method
Phase 1:
Development of a coding framework
Step 1:
Select Data
90,441 posts during 14-week course (3/4/2012 -> 6/15/2012)
Nature & number of posts changed as course progressed
37,181 during first quarter of course
17,597 during second quarter
13,858 during third quarter
21,805 during fourth quarter
Randomly selected 4500 posts to be coded - 1125 posts from each quarter

Step 2:
Train the coders
HGSE graduate students
Oriented to codebook
Discussed multiple examples
Post-training, coders were given 2500 de-identifed posts for coding
500 posts overlapped so that inter-rater reliability could be calculated


Step 3: Code
Time 1
Coded 32 posts from each quarter (128 total per coder)
Results:
Inter-rater reliability
59% for topic / role combined
Clarifed code definitions
Time 2
Coded 32 different posts from each quarter(128 total)
Results:
Inter-rater reliability
81% for topic
73% for role
Time 3
Re-coded original 128 posts
Results:
Inter-rater reliability
73% for topic
72% for role
Coded remaining 2244 posts

Results
Estimated number of posts related to:
Content - 47.90%
Other courses - 1.35%
Social / affective - 31.79%
Technology - 8.09%
Course structure - 12.41%
Other - 9.06%
Missing data - .03%
Other language - 4.85%

Estimated number of posts in which role of the poster was:
Help / information seeker - 16.68%
Help / information giver - 39.20%
Other (opinion expressions) - 17.41%

Next Steps
Explore characteristics of productive dialogue between students in this space
Explore the relationships between the subject matter of students’ posts, their role within those posts, and their achievement or persistence in the course
Collaborate with CSAIL to use this framework to categorize posts classified via natural language processing
Glenda S. Stump, Jennifer DeBoer, Jonathan Whittinghill, Lori Breslow
Full transcript