method
Introduction
- 217 undergraduate students
- Enrolled in a teacher preparation course
- Technology integration
- Pretest subject-matter
- Participate in assigned readings and regular planned learning activities
- Participate in asynchronous discussion
- Posttest subject-matter & satisfaction levels
- Assess discussion transcripts
Online and blended learning opporutnities increasing (Allen, Seaman & Garret, 2007)
- LMS typically include communication tools
- Share information, experiences, construct understanding
Higher-order thinking (HOT) skills demonstration inconsistent (Gayton & McEwen, 2007)
- Asking follow up questions might facilitate HOT skills (Schrire, 2002; Christopher, Thomas, Tallent-Runnels, 2004)
- Learners look to instructor to shape discussion interactions (Dennen, 2005)
- Often, students do not recieve facilitation beyond one well-constructed discussion prompt (Kanuka & Anderson, 1998; Daroszewski, Kinser, Loyd 2004)
- Prompt feedback may make a difference (Reddy & Andrade, 2010)
What is the effect of using a rubric on:
- Higher-order thinking skill performance
- Learning
- Satisfaction
What is the effect of instructor facilitation on:
- Higher-order thinking skill performance
- Learning
- Satisfaction
Scaffolds have been used to provide feedback in hypermedia environments (Kao & Cennamo, 1996)
- Hard scaffolds are static (e.g., rubric)
- Soft scaffolds are dynamic (e.g., instructor response prompts)
- Structure created by teachers/designers contributes to more consistent HOT skill demonstration (Smith, Savenye, Giacumo, 2009)
Results
Rubrics: common characterics
- Quantity of participation
- Cognitive quality of participation
- Meaningful connections between course content
- Abstract concepts
- Real world experiences
- Timeliness
- Collegiality
- Writing mechanics
(Ajayi, 2010; Gilbert & Dabbagh, 2005; Ho & Swan, 2007; Knowlton, 2003; Penny & Murphy, 2009; Topen & Ubuz, 2008; Vitale, 2010)
- Learning management system (Blackboard)
- Multimedia Presentation
- Guided research activity
- Assigned readings
- Discussion board prompts
Discussion Board Performance
- Formative & summative evalation tools
- Help students understand goals, expectations, and make judgements (Arter & McTingle, 2001)
- Help instructors provide clear feedback (Song, 2006)
- Correlates with increased participation (Wang, 2007)
Subject-matter learning
- Objective-type assessment
- Pretest-Posttest
Higher-order thinking skill performance
- Rubric, enroute by instructor facilitator
- Final transcript analysis
Satisfaction
Rubrics: possible influences
- Increased participation (Wang, 2007)
- Increased student satisfaction (Andrade & Du, 2005; Powell, 2001)
- Absence of rigourous research on learning (Reddy & Andrade, 2010)
Instructor facilitation in Computer-Mediated Communication
- Social interactions with more advanced persons can benefit learners (Vygotsky, 1978)
- HOT skills can be developed in asynchronous discussions (Garrison, Anderson & Archer, 2003)
- Faciliation and structure can contribute to meaningful discourse (Gilbert & Dabbagh, 2005)
Independant Variables
- Discussion board rubric
- Instructor facilitation
Dependant Variables
- Critical thinking skill performance
- Learning
- Satisfaction
Instructor facilitation: possible influences
- Positive student attitudes (Shea, Pickett, and Pelz, 2003; Swan, 2001; Zhan & de Montes, 2007)
- No significant difference in student attitudes (Giacumo & Savenye, 2010)
- Reduced student interaction (An, Shin, and Lim, 2009)
Instructor facilitation: protocols
- Give clear guidelines and opportunity for everyone to give unique response (Dennen, 2005)
- Provide feedback, but only when needed (Curran, Kirby, Parsons, Lockyer, 2003; Garrison, 2007; Jetton, 2004; Maher & Jacob, 2006)
- Remain purposely passive and contribute only if necessary (Hemphill & Hemphill, 2007)
- Should demonstrate openness for all individual contributions (McKee, 2002)
A O XA1B1 O
A O XA2B1 O
A O XA1B2 O
A O XA2B2 O
MANOVA
- Rubrics and response prompts
- Critical thinking skills
ANCOVA
MANOVA
Qualitative review
- Survey free-response questions
Research
Introduction
Method
Discussion
Wordle
Design
Materials
Results
What is the effect of using a rubric on:
- Higher-order thinking skill performance
- Learning
- Satisfaction
What is the effect of instructor facilitation on:
- Higher-order thinking skill performance
- Learning
- Satisfaction
Measures
Take Away
Context
Discussion
Discussion Board Performance
Results
- Students with a rubric outperformed students with no rubric and no facilitation prompts
- Instructor facilitation response prompts yeilded a significantly higher breadth of students performance
- The quality of students' writing was significantly better when students has instructor facilitation response prompts
- Instructor prompts tended to result in higher overall student discussion-board scores
- An interaction was found between rubrics and instructor response prompts (NEW finding)
- No statistically significant results found between the treatment groups and objective-type assessment scores
- Quiz scores were rather low throughout the treatment groups
- Students generally enjoyed blended module
- Overall, students felt good about solving problems, the overall design, and relating concepts to real-world applications
- Overall, students reported the least satisfaction with the amount of work required, student-teacher interaction, and online quiz
- percieved quiz questions as irrelevant
- Students percieved 30-item objective type assessment tool as being too long
Discussion
- Quasi-experimental design could include latent instructor effects
- Only 2 of 4 instructors actually provided response prompts
- Students have little exposure to working with rubrics as a formative assessment aid
- Students may not have noticed the rubric, as an electronic copy was only placed in the treatment discussion forms
- Rubric measures did not account for illogic
- Quiz length and wording of questions should be further investigated
Implications
for Tomorrow's Teachers and Designers
Implications
- Students should be presented with rubrics to guide discussion board performance
- Instructor faciliation ROI may not be substantial
- Beware of expecting facilitation protocols to be followed
- Consider the effects of instructing students on the use of rubrics
- Beware of expecting teachers to follow facilitation protocols
- Additional research is needed to confirm effects in different contexts, populations, and content
Presented at
SLOAN-C Blended Learning Conference
Oak Brook, Chicago
March, 28 2011
Arizona State university
Lisa Giacumo
Wilhelmina Savenye
Nichole smith
University of Phoenix
Yuyen Su
Presenter contact information
- Lisa Giacumo
- lgiacumo@asu.edu
- @Giacumo
- www.linkedin.com/in/lisagiacumo
- I want to collaborate with YOU!
- Reddy, M. & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education. Vol 35, no 4, p 435-448.
- Allen, E., Seaman, J., & Garret., R. (2007). Blending In: The extent and promise of blended education in the U.S. The Sloan Consitum.
- Aviv, R., Erlich, Z., & Geva, A. (2003). Network Analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1-23.
- Bloom, B. (1977). Behavioral Objectives and Their Application to Career Education. Journal of Career Development, 3 (4), 25-33.
- Bandura, A. (1977). Social Learning Theory. New York: General Learning Press.
- Christensen, R., Garvin, D., & Sweet, A. (1991). Education for Judgement: The Artistry of Discussion Leadership. Boston, Mass., Harvard Business School Press.
- Curran, V., Kirby, F., Parsons, E., & Lockyer, J. (2003). Discourse Analysis of Computer-Mediated Conferencing in World Wide Web-Based Continuing Medical Education. The Journal of Continuing Education in the Health Professions, 23 (1), 229-238.
- Christopher, M., Thomas, J., & Tallent-Runnels, M. (2004). Raising the Bar: Encouraging High Level Thinking in Online Discussion Forums. Roeper Review, 26 (3), 166-171.
- Daroszewski, E. B., Kinser, A. G., & Lloyd, S. L. (2004). Online, directed journaling in community health advanced practice nursing clinical education. Journal of Nursing Education, 43(4), 175-180.
- Dennen, V. P. (2005). From Message Posting to Learning Dialogues: Factors affecting learner participation in asynchronous discussion. Distance Education, 26 (1), 127-148.
- Williams, D., Yuxin, M., Feist, S., Richard, C., & Prejean, L. (2007). The design of an analogical encoding tool for game-based virtual learning environments. British Journal of Educational Technology, 38(3), 429-437.
- Duffy, T. & Jonassen, D. (1992). Constructivism and the Technology of Instruction: A Conversation. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J.
- Garrison, D. R. (2007). Online Community of Inquiry Review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1), 61-72.
- Gaytan, J. & McEwen, B. C. (2007). Effective Online Instructional and Assessment Strategies. The American Journal of Distance Education, 21 (3), 117-132.
- Good, T. (1987). Two Decades of Research on Teacher Expectations: Findings and Future Directions. Journal of Teacher Education, 38, 32-47.
- Han, S. & Hill, J. (2007). Collaborate to Learn, Learning to Collaborate: Examining the roles of context, community, and cognition in asynchronous discussion. Journal of Educational Computing Research, 36 (1), 89-123.Heejung, An. Shin, S., and Lim, K., (2009). The effects of different instructor facilitation approaches on students’ interactions during asynchronous online discussions. Computers & Education, 53 (3), 749-760.
- Hayle, M., & City, E. (2006). The Teacher’s Guide to Leading Student-Centered Discussions: Talking about Texts in the Classroom. Thousand Oaks, Calif., Corwin Press.
- Hemphill, L. & Hemphill, H. (2007). Evaluating the Impact of Guest Speaker Postings in Online Discussions. British Journal of Educational Technology, 38 (2), 287-293.
- Hyman, R. (1980). Improving Discussion Leadership. New York: Columbia Univ., Teachers College Press.
- Jackson, C. & Larkin, M. (2002). Teaching Students to Use Grading Rubrics, Teaching Exceptional Children, 35 (1), 40-45.
- Jetton, T. (2004). Using Computer-Mediated Discussion To Facilitate Preservice Teachers’ Understanding of Literacy Assessment and Instruction. Journal of Research on Technology in Education, 32 (2), 171-191.
- Johnson, D., Johnson, R., & Smith, K. (1991). Active Learning: Cooperation in the College Classroom. Edina, MN, Interaction Book Co.
- Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57-74
- Krathwohl, D. (2002). A Revision of Bloom's Taxonomy: An Overview. Teaching In Practice, 41 (4), 212-218.
- Lave, J., & Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity: Cambridge University Press.
- Lipponen, L., Rahikainenb, M., Lallimoa, J., & Hakkarainen, K. (2000). Patterns of participation and discourse in elementary students’ computer-supported collaborative learning. Learning and Instruction, 13 (5), 487-509.
- Maher, M. & Jacob, E. (2006). Peer Computer Conferencing to Support Teachers’ Reflection During Action Research. Journal of Technology and Teacher Education, 14 (1), 127-150.
- McCann, T. et al. (2006). Talking in Class: Using Discussion to Enhance Teaching and Learning. Urbana, Ill., National Council of Teachers of English.
- McIsaac, M. S., Blocher, J. M., Mahes, V., & Vrasidas, C. (1999). Student and teacher perceptions of interaction in online computer-mediated communication. Educational Media International, 36(2), 121–131.
- Nelson, B. & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: The River City case study. Educational Technology Research and Development, 56(5-6), 619-641.
- Nunn, C. (1996). Discussion in the College Classroom: Triangulating Observational and Suvery Results. Journal of Higher Education, 67, 243-266.
- Palmer, S. R. & Holt, D. M. (2009). Examining student satisfaction with wholly online learning. Journal of Computer Assisted Learning, 25, 101-113.
- Royse, D. (2001). Teaching Tips for College and University Instructors: A Practical Guide. Boston, Mass., Allyn and Bacon.Schrire, S. (2006.) Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Computers & Education, 46, 49-70.
- Tsui, L. (2002). Fostering Critical Thinking through Effective Pedagogy: Evidence from Four Institutional Case Studies. Journal of Higher Education, 73, 740-763.
- Thompson, E. & Savenye, W. (2007). Adult Learner Participation in an Online Degree Program: A program-level study of voluntary computer-mediated communication. Distance Education, 28 (3), 299-312.
- Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
- Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Networks, 8(2), 139–152.
Reddy, M. & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education. Vol 35, no 4, p 435-448.
Allen, E., Seaman, J., & Garret., R. (2007). Blending In: The extent and promise of blended education in the U.S. The Sloan Consitum.
Aviv, R., Erlich, Z., & Geva, A. (2003). Network Analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1-23.
Bloom, B. (1977). Behavioral Objectives and Their Application to Career Education. Journal of Career Development, 3 (4), 25-33.
Bandura, A. (1977). Social Learning Theory. New York: General Learning Press.
Christensen, R., Garvin, D., & Sweet, A. (1991). Education for Judgement: The Artistry of Discussion Leadership. Boston, Mass., Harvard Business School Press.
Christopher, M., Thomas, J., & Tallent-Runnels, M. (2004). Raising the Bar: Encouraging High Level Thinking in Online Discussion Forums. Roeper Review, 26 (3), 166-171.
Daroszewski, E. B., Kinser, A. G., & Lloyd, S. L. (2004). Online, directed journaling in community health advanced practice nursing clinical education. Journal of Nursing Education, 43(4), 175-180.
Dennen, V. P. (2005). From Message Posting to Learning Dialogues: Factors affecting learner participation in asynchronous discussion. Distance Education, 26 (1), 127-148.
Williams, D., Yuxin, M., Feist, S., Richard, C., & Prejean, L. (2007). The design of an analogical encoding tool for game-based virtual learning environments. British Journal of Educational Technology, 38(3), 429-437.
Duffy, T. & Jonassen, D. (1992). Constructivism and the Technology of Instruction: A Conversation. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J.
Gaytan, J. & McEwen, B. C. (2007). Effective Online Instructional and Assessment Strategies. The American Journal of Distance Education, 21 (3), 117-132.
Good, T. (1987). Two Decades of Research on Teacher Expectations: Findings and Future Directions. Journal of Teacher Education, 38, 32-47.
Hayle, M., & City, E. (2006). The Teacher’s Guide to Leading Student-Centered Discussions: Talking about Texts in the Classroom. Thousand Oaks, Calif., Corwin Press.
Hyman, R. (1980). Improving Discussion Leadership. New York: Columbia Univ., Teachers College Press.
Jackson, C. & Larkin, M. (2002). Teaching Students to Use Grading Rubrics, Teaching Exceptional Children, 35 (1), 40-45.
Johnson, D., Johnson, R., & Smith, K. (1991). Active Learning: Cooperation in the College Classroom. Edina, MN, Interaction Book Co.
Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57-74
Krathwohl, D. (2002). A Revision of Bloom's Taxonomy: An Overview. Teaching In Practice, 41 (4), 212-218.
Lave, J., & Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity: Cambridge University Press.
Lipponen, L., Rahikainenb, M., Lallimoa, J., & Hakkarainen, K. (2000). Patterns of participation and discourse in elementary students’ computer-supported collaborative learning. Learning and Instruction, 13 (5), 487-509.
McCann, T. et al. (2006). Talking in Class: Using Discussion to Enhance Teaching and Learning. Urbana, Ill., National Council of Teachers of English.
McIsaac, M. S., Blocher, J. M., Mahes, V., & Vrasidas, C. (1999). Student and teacher perceptions of interaction in online computer-mediated communication. Educational Media International, 36(2), 121–131.
Nelson, B. & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: The River City case study. Educational Technology Research and Development, 56(5-6), 619-641.
Nunn, C. (1996). Discussion in the College Classroom: Triangulating Observational and Suvery Results. Journal of Higher Education, 67, 243-266.
Palmer, S. R. & Holt, D. M. (2009). Examining student satisfaction with wholly online learning. Journal of Computer Assisted Learning, 25, 101-113.
Royse, D. (2001). Teaching Tips for College and University Instructors: A Practical Guide. Boston, Mass., Allyn and Bacon.
Schrire, S. (2006.) Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Computers & Education, 46, 49-70.
Tsui, L. (2002). Fostering Critical Thinking through Effective Pedagogy: Evidence from Four Institutional Case Studies. Journal of Higher Education, 73, 740-763.
Thompson, E. & Savenye, W. (2007). Adult Learner Participation in an Online Degree Program: A program-level study of voluntary computer-mediated communication. Distance Education, 28 (3), 299-312.
Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Networks, 8(2), 139–152.