How can we measure teaching and learning in mathematics?

The Problem

Why do math instructors continue to choose the lecture method as a major component of their courses?

What knowledge of alternate instructional practices do instructors have?

How do instructors ultimately choose instructional practices in math?

VASS: Views about Science Survey

Characterize student views about knowing and learning science

Purpose: Assess the relation of student views to achievement in science courses

28,000

• 16 items
• Two scales

CCSF

ITTF

"I feel it is important to present a lot of facts in classes so that students know what they have to learn for this subject."

"In lectures for this subject, I use difficult or undefined examples to provoke debate."

"I feel a lot of teaching time in this subject should be used to question students' ideas."

Lecture

Teaching by giving a presentation on some subject for a time period longer than 20 minutes. This instructional method includes the exchange of questions and answers between the instructor and students. The key characteristic is that the students rarely interact with each other during this learning process.

Mastery Learning

Designing summative assessment check-points into the instructional program where the student is tested on their mastery of a single topic (or subtopic). The instructor may coach students during class time or outside of class to help students who struggle with understanding the concepts while they are intensely focused on learning. Note that the students do not receive partial credit for partially correct responses on mastery-based assessments.

Collaborative Lecture

Teaching by giving a series of short, focused lessons intermixed with student-centered activities that solidify the concepts of the lessons or serve to introduce the next short lesson (DeLong and Winter, 2002). The interaction during the activities is primarily between students.

Cooperative Learning

Including class time for learning that engages students in working and learning together in small groups, typically with two to five members. Cooperative learning strategies are designed to engage students actively in the learning process through inquiry and discussions with their classmates (Rogers et al., 2001).

Inquiry-Based Learning (IBL)

Designing and using activities where students learn new concepts by actively doing and reflecting on what they have done. The guiding principle is that instructors try not to talk in depth about a concept until students have had an opportunity to think about it first (Hastings, 2006).

How do you build a good inventory?

1. Understand the Problem

Ask population to respond to open-ended questions

2. Look for patterns in the statements

3. Develop a pool of questions to represent categories

4. Multiple researchers classify items and compare results. Revise as needed.

5. Pilot the inventory. Check for internal consistency reliability and perform factor analysis.

6. Revise. Pilot again (twice with same group) for reliability.

7. Final tweaking.

1,700,000

Measure of learning approach giving three orientations: surface, deep, and achieving

• Includes Surface, Deep, and Achieving subscales
• Designed for Higher Ed

Subscales

• 19 items
• Likert scale
• Fragmented and Cohesive scales

- Burrill et al., 2002

(meta-analysis of 100s of studies on the effectiveness of teaching math with graphing calculators)

"Individual projects look at specific pieces of the picture, but the pieces do not make a coherent whole and, in fact, often seem unrelated."

- National Mathematics Advisory Panel, 2008

5. We interview those instructors and look for commonalities in environments that produce change.

- Susan Ganter, 1997

(after reviewing the research from ten years of Calculus Reform)

4. We mine the data to identify situations or instructors who are seeing results.

There is no sufficient evidence to support all-inclusive policy recommendations of any of the math instructional practices that were studied .

"There are a limited number of studies that document the impact of these efforts on student learning."

6. THEN we target research at these strategies and environmental changes to see if interventions can produce results.

What we are NOT told:

The

Solution

Interactive-Engagement (IE) sections had higher normalized gain on FCI than Traditional Lecture (TL)

Hake et al, 1998

3. The research is promoted and "crowdsourced" to any instructor at any level of math who would like to particpate.

Physics Education Research

- Beyond Crossroads, 2006

FCI: Force Concept Inventory

CLASS: Colorado Learning Attitudes

about Science Survey

1. We begin using common Language for

Math Instructional Practices (MIPS) in research.

"Mathematics faculty will use a variety of teaching strategies that reflect the results of research to enhance student learning."

MPEX: Maryland Physics Expectation Survey

What we DON'T know

What we NEED

A common language and common measures of teaching and learning.

"The FCI provides a reproducible and objective measure of how a cours improves comprehension of principles, not merely how bright or prepared the studetns are, nor what they have memorized."

- Jerome Epstein

... and for which levels of math courses?

2. We design a research study to measure instructors, students, the environment, and outcomes so that it is relatively painless to participate.

Which math instructional practices are actually effective?

What we are told:

... and with what types of students are these practices effective?

number of community college math instructors

Example: Think about the math you've done so far. What do you think mathematics is?

78% of the PT and 98% of the FT have graduate degrees

47-50% of these instructors are women

67% of these instructors work part-time

Roughly 1900-2000 of them are members of AMATYC

Data from AMATYC and 2005 CBMS Statistical Report

Credits for Photos (all licensed under Creative Commons):

Ruler: http://www.flickr.com/photos/917press/456443078

EEG: http://www.flickr.com/photos/squashpicker/1480124942/

Puzzle pieces: http://www.flickr.com/photos/myklroventine/3261364899/

Puzzle on face: http://www.flickr.com/photos/eli-santana/2933926582/

Wikipedia Concept Map: http://www.flickr.com/photos/juhansonin/407874864/

Water Surface “Overflate”: http://www.flickr.com/photos/randihausken/1877810147/

Scuba diver: http://www.flickr.com/photos/30691679@N07/2891679952/

Thinker: http://www.flickr.com/photos/tmartin/32010732/

Construction signs: http://www.flickr.com/photos/15708236@N07/2754478731/

Computer classroom: http://www.flickr.com/photos/phoenixdailyphoto/1782001450/

Classroom with desks: http://www.flickr.com/photos/25312309@N05/2829580870/

Lecture hall: http://www.flickr.com/photos/kitsu/404092967/

Guy with barcode: http://www.flickr.com/photos/jaumedurgell/740880616/

B&W Stressed woman: http://www.flickr.com/photos/librarianbyday/3181780269/

Head pillars: http://www.flickr.com/photos/jannem/376980800/

Guy studying: http://www.flickr.com/photos/tripu/3441921187/

Credits for Illustrations and Cartoons

Mat Moore, Muskegon Michigan

Freelance Illustrator (garlicandcoffee@gmail.com)

How do instructors

approach teaching?

A teacher/student interaction strategy with the intention that students acquire the concepts of the discipline

How can we measure

teaching and learning

in mathematics?

Teacher-focused strategy with the intention of transmitting information to students

A student-focused strategy aimed at students changing their conceptions

Approaches to Teaching Inventory

Elementary

Algebra

Concept

Inventory

Teacher-focused strategy with the intention that students acquire the concepts of the discipline

Prosser and Trigwell, 1999

Jerome Epstein ,

NY Polytechnic University

ATI

Example

Items

Conceptual change / student-focused

Information transmission / teacher-focused

A student-focused strategy aimed at students developing their conceptions

Jerome Epstein ,

NY Polytechnic University

Calculus

Concepts

Inventory

EACI

The Instructors

CCI

Math Instructional Practices

Mathematics Concepts Test

Andersen, 2009

The Subject Area

Emphasis on

Application Problems

Including class time for students to solve problems based on data from real-world situations (present or past) problems that come from the partner disciplines of mathematics (e.g. Engineering, Chemistry, Biology, Physics, Economics, Business).

Precalculus

Concept

Assessment

Marilyn Carlson,

Arizona State University

Example from http://www.flaguide.org/tools/diagnostic/calculus_concept_inventory.php

Communication Skills

Providing opportunities for students to practice their ability to communicate mathematical and quantitative ideas using both written and oral communications.

If you know that a function f(x) is positive everywhere, what can you conclude from that about the derivative, f '(x)?

a) the derivative is positive everywhere

b) the derivative is increasing everywhere

c) the derivative is concave upward

d) you can't conclude anything about the

derivative

Example similar to the CCI

Formative Assessment

Making use of instructional strategies in the learning environment that assess where students are having problems so that students can learn more and learn better. (Gold, 1999)

Multiple Representations

Teaching by including multiple ways (e.g. graphs, diagrams, algebra, words, data, manipulatives) to represent mathematical ideas whenever possible. The rule-of-four (representing a function visually, algebraically, numerically, or with words) is an example of multiple representations.

Maria H. Andersen, Ph.D.

busynessgirl.com

@busynessgirl

PCA

Designing and assigning project work that requires students to solve a non-standard problem that requires a longer period of time than problems that would typically be assigned for homework or in class. There is often a research component where students must actively seek data, background knowledge, or formulas. Often the students work on projects in pairs or small groups. The final result of a project might include a written paper or a presentation on the findings.

Project Based Learning

The Students

Conceptions of

Mathematics

Questionairre

CMQ

How do students perceive

their learning environment?

number of CC math students in 1 year

Example

Items

Crawford et al., 1998

"Mathematics is about calculations"

"Mathematics is a logical system which helps explain the things around us"

"Mathematics is like a universal language which allows people to communicate and understand the world"

Trigwell & Prosser, 1991

The Environment

Students perceive a heavy workload and less freedom in learning.

Students perceive that there is a choice in what is learned and a clear awareness of goals and standards.

Control of Teaching

Departmental Support for Teaching

How does the environment effect what instructors do?

Enabling Student Characteristics

Links students' perception of their learning environment and their quality of learning

Course

Perceptions

Questionaaire

CPQ

68% of two-year college faculty reported at least some stress from teaching underprepared students

(Lindholm et al., 2005)

Cohesive

"To improve the quality of learning, it is more important to encourage deeper approaches to study

through the creation of a context involving good teaching, clear goals, and some independence in learning

than the discouragement of surface approaches to learning."

• Abstraction of meaning
• Understanding of reality

PTEI

- Prosser & Trigwell, 1997

Entwistle & Ramsden, 1983

Appropriate Academic Workload

5. Math is a complex logical system which can be used to solve complex problems and provides new insights used for understanding the world

Perceptions of

Teaching

Environment

Inventory

Marton and Saljo, 1997

Student Conceptions

of Learning

4. Math is a complex logical system which can be used to solve complex problems

Prosser & Trigwell, 1997

Crawford et al 1994, 1998

Five conceptions

of Mathematics

Student Conceptions of Mathematics

3. Math is a complex logical system: a way of thinking

Appropriate Learning Space

Appropriate Class Size

2. Math is numbers, rules, and formulas which can be applied to solve problems

Study

Process

Questionnaire

SPQ

How do students approach their learning?

1. Math is numbers, rules, and formulas

Entwistle & Ramsden, 1983; Biggs, 1987; Ramsden 1991, 1992; Marton et al, 1997

Biggs, 1987

NOTE: For a K-12 version, look at the Learning Process Questionnaire (LPQ)

Fragmented

• Quantitative increase in knowledge
• Memorization

ASI

52 questions

dimensions are deep, strategic, and surface learning

Approaches to

Study

Inventory

Deep

approach

Surface

approach

Ramsden & Entwistle, 1981

Entwistle & Ramsden, 1983

Entwhistle et al., 2000

• can see relevance of learning new things
• seek to develop new understanding

• tasks are "imposed" on them
• study without purpose or strategy

More likely to be associated

with higher quality learning

outcome.

Related subscales

Approaches &

Study Skills

Inventory for

Students

• Deep approach
• Strategic approach
• Surface Apathetic approach
• Preferences for different types of course and teaching

• Fear of failure
• Lack of purpose
• Syllabus-boundness
• Unrelated memorizing

ASSIST