The Affordances of Mulimodal Texts and Their Impact on the Reading Process
Results
Methodology
Scott Voss
October 6, 2011
AVReading.org
Conclusion
Limitations
Implications
 Literature 
Review
Story of Midas
Story of Me
Story of Affordances
The verb to afford is found in the dictionary, but the noun affordance is not. I have made it up. I mean by it something that refers to both the environment and the animal in a way that no existing term does. It implies the complimentarity of the animal and the environment. 
--Gibson, J. (1979). The Ecological Approach to Visual Perception 
Purpose of Study
This study examines how multimodal texts might impact the reading process as evidenced by comprehension scores, self-efficacy scores, observations and interviews. 
Three Research Questions
Q1:  What impact do different modes (print, audio, and video) of text have on comprehension?
Q2:  How do students make meaning of the text as they access the text digitally?  What is the process for reading the text?  How do the students make use of the various affordances?
Q3:  What effects do the various modalities have on a student’s perception of their ability to read the text digitally?  Is there a measurable difference between a student’s self-reported feelings of self-efficacy across the modalities?  What do students say about using online textbooks and how do they feel about having them available?  
Online Comprehension
1. Print-text reading processes were necessary, but not sufficient for making meaning online(Coiro & Dobler, 2007) 
2. Processes of identifying important questions, locating information, analyzing the usefulness of that information, synthesizing information to answer those questions, and communicating the answers to others (Leu, Kinzer, Coiro, & Cammack, 2004) 

Multimodality
Social Semiotics
Cognitive Science
Most of our lives rely on “modes remote from speech and writing” (Kress, 2011) 
"a field of study concerned with how human beings use different modes of communication, like speech, writing, image, gesture, and sound to represent or make meaning in the world”  (Stein, 2008)
1.  Spit Attention (Mayer & Moreno, 1998).
2.  Mutliple modes expand memory capacity (Low & Sweller, 2005; Mayer, 2005).
3.  Redundant audio and visual had positive effect on comprehension (Mayer & Johnson, 2008).
4.  Avoiding overload by offering modes non-concurrently (Kayluga,  Chandler, & Sweller, 2004).
5.  Effects of multimodal features were dependent on cognitive ability (Schnotz & Rasch, 2005).  
Online Comprehension
Multimodality
Affordance Theory
Self Efficacy
"The complimentarity of animal and environment" or what the environment allows. (Gibson, 1979)
- Inherently part of that environment or object (Warren, 1984).
-Something perceived by an individual (Stoffgren, 2003)

Self-Efficacy has a positive effect on performance (Parajes, 1996)
1. Children tend to overestimate abilities (Stipek & MacIver, 1989)
2. These ideas become more complex as students get older (Wigfield & Eccles, 2001).
3. Students become more accurate and more specific with their thoughts on self-efficacy  (Schunk & Miller, 2002). 
4. Feelings become more entrenched as the child grows older. (Gottfried, Flemming & Gottfried,2001).


Data Analysis
Quantitative
Qualitative
Grounded Theory (Corbin & Strauss, 1990)
Open coding
Constant Comparison
Categories based on patterns
Subcategories with properties
Develop theory
Phase One: Quantitative
Phase Two:  Qualitative
(cc) image by anemoneprojectors on Flickr
Carrie--
Yeah, I kind of like this better because I can, like uhm, when he reads it to me I can comprehend in my head what I’m thinking and then go over it with the question and get more in depth and then instead of just like reading it.  That’s what I like about that. (interview, 11/23/10)
Helen 
“I liked this split screen. Like how you would have the questions on one side here, so you could read the questions” (interview, 11/23/10)
Laura: First I read it through—like each section.  And then I played the audio--to like, comprehend what I read, and then I read it one more time, and if I didn’t get it,  I would play it again—I would just keep going back and forth until I got it.
Interviewer:  So you read everything twice and listened to the audio once? 
Laura:  Yeah.  (interview, 11/24/10)

Rachel:  I can’t think of a word to explain it. This [digital text] makes me feel like I’m smarter than reading in a book. I don’t know why, but it just does (interview, 11/24/10).
Interpretation  
Perceive the affordance
Strategically use the affordance
1. No statistically signifcant difference in comprehension.
2.  Strategic readers appeared to be more efficient at navigating the affordances.
3.  Students report greater feelings of self-efficacy when given access to digital affordances.
More studies on digital textbooks
Suggests that technology alone is not the solution.
Modeling use of affordances and tools may be important to less strategic readers.