Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

method

Introduction

Participants

Literature

Procedures

  • 217 undergraduate students
  • Enrolled in a teacher preparation course
  • Technology integration
  • Pretest subject-matter
  • Participate in assigned readings and regular planned learning activities
  • Participate in asynchronous discussion
  • Posttest subject-matter & satisfaction levels
  • Assess discussion transcripts

Research Questions

Online and blended learning opporutnities increasing (Allen, Seaman & Garret, 2007)

  • LMS typically include communication tools
  • Share information, experiences, construct understanding

Higher-order thinking (HOT) skills demonstration inconsistent (Gayton & McEwen, 2007)

  • Asking follow up questions might facilitate HOT skills (Schrire, 2002; Christopher, Thomas, Tallent-Runnels, 2004)
  • Learners look to instructor to shape discussion interactions (Dennen, 2005)
  • Often, students do not recieve facilitation beyond one well-constructed discussion prompt (Kanuka & Anderson, 1998; Daroszewski, Kinser, Loyd 2004)
  • Prompt feedback may make a difference (Reddy & Andrade, 2010)

What is the effect of using a rubric on:

  • Higher-order thinking skill performance
  • Learning
  • Satisfaction

What is the effect of instructor facilitation on:

  • Higher-order thinking skill performance
  • Learning
  • Satisfaction

Scaffolds have been used to provide feedback in hypermedia environments (Kao & Cennamo, 1996)

  • Hard scaffolds are static (e.g., rubric)
  • Soft scaffolds are dynamic (e.g., instructor response prompts)
  • Structure created by teachers/designers contributes to more consistent HOT skill demonstration (Smith, Savenye, Giacumo, 2009)

Materials

Measures

Results

Rubrics: common characterics

  • Quantity of participation
  • Cognitive quality of participation
  • Meaningful connections between course content
  • Abstract concepts
  • Real world experiences
  • Timeliness
  • Collegiality
  • Writing mechanics

(Ajayi, 2010; Gilbert & Dabbagh, 2005; Ho & Swan, 2007; Knowlton, 2003; Penny & Murphy, 2009; Topen & Ubuz, 2008; Vitale, 2010)

  • Learning management system (Blackboard)
  • Multimedia Presentation
  • Guided research activity
  • Assigned readings
  • Discussion board prompts

Rubrics: common uses

Discussion Board Performance

  • Formative & summative evalation tools
  • Help students understand goals, expectations, and make judgements (Arter & McTingle, 2001)
  • Help instructors provide clear feedback (Song, 2006)
  • Correlates with increased participation (Wang, 2007)

Subject-matter learning

  • Objective-type assessment
  • Pretest-Posttest

Higher-order thinking skill performance

  • Rubric, enroute by instructor facilitator
  • Final transcript analysis

Satisfaction

  • ELO, Palmer & Holt

Rubrics: possible influences

  • Increased participation (Wang, 2007)
  • Increased student satisfaction (Andrade & Du, 2005; Powell, 2001)
  • Absence of rigourous research on learning (Reddy & Andrade, 2010)

Follow Up

Instructor facilitation in Computer-Mediated Communication

  • Social interactions with more advanced persons can benefit learners (Vygotsky, 1978)
  • HOT skills can be developed in asynchronous discussions (Garrison, Anderson & Archer, 2003)
  • Faciliation and structure can contribute to meaningful discourse (Gilbert & Dabbagh, 2005)

Design

Independant Variables

  • Discussion board rubric
  • Instructor facilitation

Dependant Variables

  • Critical thinking skill performance
  • Learning
  • Satisfaction

Instructor facilitation: possible influences

Posttest Scores

Pretest - Posttest Gains

  • Positive student attitudes (Shea, Pickett, and Pelz, 2003; Swan, 2001; Zhan & de Montes, 2007)
  • No significant difference in student attitudes (Giacumo & Savenye, 2010)
  • Reduced student interaction (An, Shin, and Lim, 2009)

Analyses

Instructor facilitation: protocols

  • Give clear guidelines and opportunity for everyone to give unique response (Dennen, 2005)
  • Provide feedback, but only when needed (Curran, Kirby, Parsons, Lockyer, 2003; Garrison, 2007; Jetton, 2004; Maher & Jacob, 2006)
  • Remain purposely passive and contribute only if necessary (Hemphill & Hemphill, 2007)
  • Should demonstrate openness for all individual contributions (McKee, 2002)

A O XA1B1 O

A O XA2B1 O

A O XA1B2 O

A O XA2B2 O

MANOVA

  • Rubrics and response prompts
  • Critical thinking skills

ANCOVA

  • Pretest, posttest

MANOVA

  • Satisfaction survey

Qualitative review

  • Survey free-response questions

Satisfaction Survey

Follow Up

Research

Introduction

Method

Discussion

Reserach Questions

Wordle

Design

Materials

Results

What is the effect of using a rubric on:

  • Higher-order thinking skill performance
  • Learning
  • Satisfaction

What is the effect of instructor facilitation on:

  • Higher-order thinking skill performance
  • Learning
  • Satisfaction

Measures

Take Away

Context

Discussion

Discussion Board Performance

Results

  • Students with a rubric outperformed students with no rubric and no facilitation prompts
  • Instructor facilitation response prompts yeilded a significantly higher breadth of students performance
  • The quality of students' writing was significantly better when students has instructor facilitation response prompts
  • Instructor prompts tended to result in higher overall student discussion-board scores
  • An interaction was found between rubrics and instructor response prompts (NEW finding)

Posttest Performance

  • No statistically significant results found between the treatment groups and objective-type assessment scores
  • Quiz scores were rather low throughout the treatment groups

Satisfaction Survey

  • Students generally enjoyed blended module
  • Overall, students felt good about solving problems, the overall design, and relating concepts to real-world applications
  • Overall, students reported the least satisfaction with the amount of work required, student-teacher interaction, and online quiz
  • percieved quiz questions as irrelevant
  • Students percieved 30-item objective type assessment tool as being too long

Discussion

Limitations

  • Quasi-experimental design could include latent instructor effects
  • Only 2 of 4 instructors actually provided response prompts
  • Students have little exposure to working with rubrics as a formative assessment aid
  • Students may not have noticed the rubric, as an electronic copy was only placed in the treatment discussion forms
  • Rubric measures did not account for illogic
  • Quiz length and wording of questions should be further investigated

Implications

for Tomorrow's Teachers and Designers

Implications

Teachers

  • Students should be presented with rubrics to guide discussion board performance
  • Instructor faciliation ROI may not be substantial

Designers

  • Beware of expecting facilitation protocols to be followed

Researchers

  • Consider the effects of instructing students on the use of rubrics
  • Beware of expecting teachers to follow facilitation protocols
  • Additional research is needed to confirm effects in different contexts, populations, and content

Presented at

SLOAN-C Blended Learning Conference

Oak Brook, Chicago

March, 28 2011

Arizona State university

Lisa Giacumo

Wilhelmina Savenye

Nichole smith

University of Phoenix

Yuyen Su

Presenter contact information

  • Lisa Giacumo
  • lgiacumo@asu.edu
  • @Giacumo
  • www.linkedin.com/in/lisagiacumo
  • I want to collaborate with YOU!

References

  • Reddy, M. & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education. Vol 35, no 4, p 435-448.
  • Allen, E., Seaman, J., & Garret., R. (2007). Blending In: The extent and promise of blended education in the U.S. The Sloan Consitum.
  • Aviv, R., Erlich, Z., & Geva, A. (2003). Network Analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1-23.
  • Bloom, B. (1977). Behavioral Objectives and Their Application to Career Education. Journal of Career Development, 3 (4), 25-33.
  • Bandura, A. (1977). Social Learning Theory. New York: General Learning Press.
  • Christensen, R., Garvin, D., & Sweet, A. (1991). Education for Judgement: The Artistry of Discussion Leadership. Boston, Mass., Harvard Business School Press.
  • Curran, V., Kirby, F., Parsons, E., & Lockyer, J. (2003). Discourse Analysis of Computer-Mediated Conferencing in World Wide Web-Based Continuing Medical Education. The Journal of Continuing Education in the Health Professions, 23 (1), 229-238.
  • Christopher, M., Thomas, J., & Tallent-Runnels, M. (2004). Raising the Bar: Encouraging High Level Thinking in Online Discussion Forums. Roeper Review, 26 (3), 166-171.
  • Daroszewski, E. B., Kinser, A. G., & Lloyd, S. L. (2004). Online, directed journaling in community health advanced practice nursing clinical education. Journal of Nursing Education, 43(4), 175-180.
  • Dennen, V. P. (2005). From Message Posting to Learning Dialogues: Factors affecting learner participation in asynchronous discussion. Distance Education, 26 (1), 127-148.
  • Williams, D., Yuxin, M., Feist, S., Richard, C., & Prejean, L. (2007). The design of an analogical encoding tool for game-based virtual learning environments. British Journal of Educational Technology, 38(3), 429-437.
  • Duffy, T. & Jonassen, D. (1992). Constructivism and the Technology of Instruction: A Conversation. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J.
  • Garrison, D. R. (2007). Online Community of Inquiry Review: Social, cognitive, and teaching presence issues. Journal of Asynchronous Learning Networks, 11(1), 61-72.
  • Gaytan, J. & McEwen, B. C. (2007). Effective Online Instructional and Assessment Strategies. The American Journal of Distance Education, 21 (3), 117-132.
  • Good, T. (1987). Two Decades of Research on Teacher Expectations: Findings and Future Directions. Journal of Teacher Education, 38, 32-47.
  • Han, S. & Hill, J. (2007). Collaborate to Learn, Learning to Collaborate: Examining the roles of context, community, and cognition in asynchronous discussion. Journal of Educational Computing Research, 36 (1), 89-123.Heejung, An. Shin, S., and Lim, K., (2009). The effects of different instructor facilitation approaches on students’ interactions during asynchronous online discussions. Computers & Education, 53 (3), 749-760.
  • Hayle, M., & City, E. (2006). The Teacher’s Guide to Leading Student-Centered Discussions: Talking about Texts in the Classroom. Thousand Oaks, Calif., Corwin Press.
  • Hemphill, L. & Hemphill, H. (2007). Evaluating the Impact of Guest Speaker Postings in Online Discussions. British Journal of Educational Technology, 38 (2), 287-293.
  • Hyman, R. (1980). Improving Discussion Leadership. New York: Columbia Univ., Teachers College Press.
  • Jackson, C. & Larkin, M. (2002). Teaching Students to Use Grading Rubrics, Teaching Exceptional Children, 35 (1), 40-45.
  • Jetton, T. (2004). Using Computer-Mediated Discussion To Facilitate Preservice Teachers’ Understanding of Literacy Assessment and Instruction. Journal of Research on Technology in Education, 32 (2), 171-191.
  • Johnson, D., Johnson, R., & Smith, K. (1991). Active Learning: Cooperation in the College Classroom. Edina, MN, Interaction Book Co.
  • Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57-74
  • Krathwohl, D. (2002). A Revision of Bloom's Taxonomy: An Overview. Teaching In Practice, 41 (4), 212-218.
  • Lave, J., & Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity: Cambridge University Press.
  • Lipponen, L., Rahikainenb, M., Lallimoa, J., & Hakkarainen, K. (2000). Patterns of participation and discourse in elementary students’ computer-supported collaborative learning. Learning and Instruction, 13 (5), 487-509.
  • Maher, M. & Jacob, E. (2006). Peer Computer Conferencing to Support Teachers’ Reflection During Action Research. Journal of Technology and Teacher Education, 14 (1), 127-150.
  • McCann, T. et al. (2006). Talking in Class: Using Discussion to Enhance Teaching and Learning. Urbana, Ill., National Council of Teachers of English.
  • McIsaac, M. S., Blocher, J. M., Mahes, V., & Vrasidas, C. (1999). Student and teacher perceptions of interaction in online computer-mediated communication. Educational Media International, 36(2), 121–131.
  • Nelson, B. & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: The River City case study. Educational Technology Research and Development, 56(5-6), 619-641.
  • Nunn, C. (1996). Discussion in the College Classroom: Triangulating Observational and Suvery Results. Journal of Higher Education, 67, 243-266.
  • Palmer, S. R. & Holt, D. M. (2009). Examining student satisfaction with wholly online learning. Journal of Computer Assisted Learning, 25, 101-113.
  • Royse, D. (2001). Teaching Tips for College and University Instructors: A Practical Guide. Boston, Mass., Allyn and Bacon.Schrire, S. (2006.) Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Computers & Education, 46, 49-70.
  • Tsui, L. (2002). Fostering Critical Thinking through Effective Pedagogy: Evidence from Four Institutional Case Studies. Journal of Higher Education, 73, 740-763.
  • Thompson, E. & Savenye, W. (2007). Adult Learner Participation in an Online Degree Program: A program-level study of voluntary computer-mediated communication. Distance Education, 28 (3), 299-312.
  • Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.
  • Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Networks, 8(2), 139–152.

Reddy, M. & Andrade, H. (2010). A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education. Vol 35, no 4, p 435-448.

Allen, E., Seaman, J., & Garret., R. (2007). Blending In: The extent and promise of blended education in the U.S. The Sloan Consitum.

Aviv, R., Erlich, Z., & Geva, A. (2003). Network Analysis of knowledge construction in asynchronous learning networks. Journal of Asynchronous Learning Networks, 7(3), 1-23.

Bloom, B. (1977). Behavioral Objectives and Their Application to Career Education. Journal of Career Development, 3 (4), 25-33.

Bandura, A. (1977). Social Learning Theory. New York: General Learning Press.

Christensen, R., Garvin, D., & Sweet, A. (1991). Education for Judgement: The Artistry of Discussion Leadership. Boston, Mass., Harvard Business School Press.

Christopher, M., Thomas, J., & Tallent-Runnels, M. (2004). Raising the Bar: Encouraging High Level Thinking in Online Discussion Forums. Roeper Review, 26 (3), 166-171.

Daroszewski, E. B., Kinser, A. G., & Lloyd, S. L. (2004). Online, directed journaling in community health advanced practice nursing clinical education. Journal of Nursing Education, 43(4), 175-180.

Dennen, V. P. (2005). From Message Posting to Learning Dialogues: Factors affecting learner participation in asynchronous discussion. Distance Education, 26 (1), 127-148.

Williams, D., Yuxin, M., Feist, S., Richard, C., & Prejean, L. (2007). The design of an analogical encoding tool for game-based virtual learning environments. British Journal of Educational Technology, 38(3), 429-437.

Duffy, T. & Jonassen, D. (1992). Constructivism and the Technology of Instruction: A Conversation. Lawrence Erlbaum Associates Publishers, Hillsdale, N.J.

Gaytan, J. & McEwen, B. C. (2007). Effective Online Instructional and Assessment Strategies. The American Journal of Distance Education, 21 (3), 117-132.

Good, T. (1987). Two Decades of Research on Teacher Expectations: Findings and Future Directions. Journal of Teacher Education, 38, 32-47.

Hayle, M., & City, E. (2006). The Teacher’s Guide to Leading Student-Centered Discussions: Talking about Texts in the Classroom. Thousand Oaks, Calif., Corwin Press.

Hyman, R. (1980). Improving Discussion Leadership. New York: Columbia Univ., Teachers College Press.

Jackson, C. & Larkin, M. (2002). Teaching Students to Use Grading Rubrics, Teaching Exceptional Children, 35 (1), 40-45.

Johnson, D., Johnson, R., & Smith, K. (1991). Active Learning: Cooperation in the College Classroom. Edina, MN, Interaction Book Co.

Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13(1), 57-74

Krathwohl, D. (2002). A Revision of Bloom's Taxonomy: An Overview. Teaching In Practice, 41 (4), 212-218.

Lave, J., & Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity: Cambridge University Press.

Lipponen, L., Rahikainenb, M., Lallimoa, J., & Hakkarainen, K. (2000). Patterns of participation and discourse in elementary students’ computer-supported collaborative learning. Learning and Instruction, 13 (5), 487-509.

McCann, T. et al. (2006). Talking in Class: Using Discussion to Enhance Teaching and Learning. Urbana, Ill., National Council of Teachers of English.

McIsaac, M. S., Blocher, J. M., Mahes, V., & Vrasidas, C. (1999). Student and teacher perceptions of interaction in online computer-mediated communication. Educational Media International, 36(2), 121–131.

Nelson, B. & Erlandson, B. (2008). Managing cognitive load in educational multi-user virtual environments: The River City case study. Educational Technology Research and Development, 56(5-6), 619-641.

Nunn, C. (1996). Discussion in the College Classroom: Triangulating Observational and Suvery Results. Journal of Higher Education, 67, 243-266.

Palmer, S. R. & Holt, D. M. (2009). Examining student satisfaction with wholly online learning. Journal of Computer Assisted Learning, 25, 101-113.

Royse, D. (2001). Teaching Tips for College and University Instructors: A Practical Guide. Boston, Mass., Allyn and Bacon.

Schrire, S. (2006.) Knowledge building in asynchronous discussion groups: Going beyond quantitative analysis. Computers & Education, 46, 49-70.

Tsui, L. (2002). Fostering Critical Thinking through Effective Pedagogy: Evidence from Four Institutional Case Studies. Journal of Higher Education, 73, 740-763.

Thompson, E. & Savenye, W. (2007). Adult Learner Participation in an Online Degree Program: A program-level study of voluntary computer-mediated communication. Distance Education, 28 (3), 299-312.

Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.

Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. Journal of Asynchronous Learning Networks, 8(2), 139–152.

Learn more about creating dynamic, engaging presentations with Prezi