Drexel dragonThe Math ForumDonate to the Math Forum

Evaluation of Math Forum Programs

Introduction

Program Goals and Principles

Activities and Services

Evaluation of Math Forum Programs

Sample School Web Site

Background

1992 - 2011 Timeline

Program Development and Funding

Testimonials


Prior Events

Contact Us

In keeping with its status as an innovator, the Math Forum is also a recognized leader in the emerging field of research devoted to the study of Internet-based educational services and learning communities. The evaluation team is led by Ann Renninger, Professor of Education and Psychology at Swarthmore College, and Wesley Shumar, Assistant Professor of Psychology, Sociology, and Anthropology at Drexel University. Renninger and Shumar have developed new tools and methods to study patterns of usage, community activity, and the Forum's impact on learning.

Key to the success of the Math Forum is its ability to satisfy immediate needs for both teachers and students while drawing them into longer-term usage through extended interactions and easy access to other useful resources. Usage studies show that the majority of teachers come to the Forum seeking ideas for classroom lessons and activities for their students, while most students come seeking help with math problems. The responsiveness of the Forum environment and the individualized relationships found there encourage the personal involvement that leads to positive change in teaching and learning.

Professional Development

Early studies captured the immediate appeal of the Forum for teachers.

In many schools, talking about either math or teaching math is considered taboo, even if you teach it. For teachers feeling the press of these settings, interaction [at the Forum] can be participant-driven with individuals seeking collegiality or friends with whom to talk about math. Interaction also can be private, in the sense that the Forum is a safe community in which one can review feedback, talk about concerns, and seek answers to problems. One teacher, for example, told us that he would never admit to his colleagues that he had not had geometry since high school. He used the Forum archives to learn the geometry he needed to teach. Interaction on the Forum site is a structural feature that enables participants to increase their knowledge, and because of the opportunity to build knowledge, participants come to identify with the Forum as a community. The Forum community supports participant learning and enables participants to grow as mathematical thinkers.

Teachers who continue to use the Forum tend to offer one of three reasons for doing so. They describe it as offering: (a) opportunities to talk, think, and share resources with others about mathematics, technology, and/or pedagogy, (b) interactions with expert-others who model and provide support for problem posing and problem solving, and (c) a wide and ever deepening range of quality content about mathematics, technology, and pedagogy.

The impact of the Math Forum on professional development stems in part from the long-term and multiple layers of participation encouraged by its interactive services. In-depth interviews with Forum teachers revealed that

[m]ost of the teachers interviewed had begun working with some aspect of the site--an aspect that mapped onto either an individual strength (i.e., mathematics, teaching mathematics) or a need (i.e., mathematics, teaching mathematics). Over time almost all of them began to find ways to integrate one or more of the Math Forum services into their teaching and to explore the use of other resources on the site. They also either explicitly or implicitly grew to identify the site as a resource for their own professional learning.

For teachers who come to think of the Forum as a community, the site engages them in expanding their roles as teachers, colleagues, and members of the broader community of educators much like Little's (1993) description of substantial models of teacher professional development. Unlike more traditional forms of teacher professional development, however, the Math Forum provides services for teachers and opportunities to interact around these; it does not specify what teachers need to do. It provides an inquiry-oriented extension for the mathematics classroom, as well as a "forum" through which teachers can explore and actively personalize their work with topics related to mathematics and its use.

Public forums are the largest area of interaction between professionals at the Math Forum, where the most focused of these occurs in the Teacher2Teacher service with over 7,000 contributors. In a study of the public discussions facilitated by the Math Forum it was found that

both teachers and educational psychology students developed a language for talking about learning over the course of their contributions and used this to develop their ideas, subsequently becoming more developmental in their focus (talking about individual strengths and weaknesses and change in these, rather than categorizing learners only as strong or weak)-they talked about process and product, as well as about access and reflection. There were also shifts in the breadth of both teachers' and students' perspectives. Both groups appeared to appreciate the complexity of student learning in a different way than they had at the outset of their participation.

Student Learning

Every day, the Math Forum receives letters of appreciation from all kinds of students for the insights they've gained and the interest in mathematics they've rekindled. Studies over the last seven years have confirmed and begun to explain the value of Forum services.

The most common use of the Forum's interactive services is as a supplement to classroom instruction. An early study was undertaken in conjunction with two other math projects, one of which used video-disc simulations for teaching mathematics and the other of which used a project-focused curriculum. Students in each study received two questionnaires, six months apart, asking them about their "work orientation" during problem solving, interest for mathematics in general, feelings of anxiety about math class, and situational interest for their current math class. The study showed that

[b]oth boys and girls were more likely to have experienced increased interest in math generally, and increased interest in the study of geometry more specifically, although girls were even more likely than boys to report such changes. . . .In order to further investigate these changes in light of performance, change in students' grades was also evaluated. These data suggest that student grades improved across the board with no significant differences in the performance of boys and girls.

These early studies of feelings and achievement led to in-depth study of the impact of the Problems of the Week (PoW) services on mathematical thinking. In the 2000-2001 school year, approximately 30,000 students submitted solutions for mentoring. Given the site usage statistics and the reports of many teachers who use the problems and solutions without writing in to Math Forum mentors, this is only a small fraction of the student population using the Problems of the Week.

Study of the impact of the PoW suggests that sustained work enhances students' understanding of problems, increases the likelihood that they will generate problem-solving strategies, and enables them to work independently.

Overall findings for change in levels of connectedness, strategy use, and autonomy indicate that for each PoW student group: (a) students' connectedness (ability to make real-world connections, identify what a problem involved, and choose between strategies); (b) students' strategy use (ability to explain mathematical concepts, work with mathematical terms, and explain decision-making); and (c) students' autonomy (independence in explaining their mathematical thinking; need for models, example, or scaffolds; and ability to focus on problem-solving) all increased [over time].

Methods

The Internet is a new medium for education and requires new tools for evaluation. The Forum evaluation team has been at the fore-front of this field, inventing their tools as the services unfold.

The teacher interviews, discursive analysis of the email interaction of a sample of the PoWs and Ask Dr. Math interactions, videotape analysis of classroom teaching practices, and narrative description of student work on the PoWs suggested the importance of undertaking the study of learning and change at three levels of analysis: individual, dyadic, and school norms and culture. These levels of analysis are being studied across the services and projects designed and developed by the Forum staff. Coordinated study of these different levels of analysis not only acknowledges the complexity of using the resources on the site for learning and teaching, but provides a tool for assessing changed learning and practice. It provides a detailed picture of the variety of ways in which the Forum is being used and interpreted by individuals, in turn facilitating the identification of indicators necessary for the study of changed learning and practice.

References

Little, J. W., & McLaughlin, M. W. (Eds.) (1993).
Teachers' work: individuals, colleagues, and contexts. New York : Teachers College Press.
Renninger, K.A., Farra, L., & Feldman-Riordan, C. (2000).
The impact of The Math Forum's Problems of the Week on students' mathematical thinking. Proceedings of ICLS 2000. Mahwah, NJ: Erlbaum.
Renninger, K. A., Weimar, S. A., & Klotz, E. A. (1998).
Teachers and students investigating and communicating about geometry: The Math Forum. In R. Lehrer & D. Chazan (Eds.), Designing learning environments for developing understanding of geometry and space (pp. 465-487). Mahwah, NJ: Lawrence Erlbaum Associates.
Renninger, K.A. & Shumar, W. (2002).
Community building with and for teachers: The Math Forum as a resource for teacher professional development. In K.A. Renninger & W. Shumar (Eds.),Building virtual communities: Learning and change in cyberspace. New York, NY: Cambridge University Press.
Renninger, K. A., & Shumar, W. (in press).
Learning at and with The Math Forum. In S. Barab, R. Kling, & J. Gray (Eds.), Designing for virtual communities in the service of learning. New York, NY: Cambridge University Press.
Renninger, K. A., & Shumar, W. (1998).
Why and how students work with The Math Forum's Problem(s) of the Week: Implications for design. Proceedings of ICLS 98 (pp. 348-350). Charlottesville, VA: AACE.
Shumar, W., & Renninger, K. A. (2002, April).
Sustaining online community: learning and participation at the Math Forum. Paper presented at the American Educational Research Association annual meeting, New Orleans, LA.

<< back   next >>

[Privacy Policy] [Terms of Use]

_____________________________________
Home || The Math Library || Quick Reference || Search || Help 
_____________________________________

© 1994-2014 Drexel University. All rights reserved.
http://mathforum.org/
The Math Forum is a research and educational enterprise of the Drexel University School of Education.The Math Forum is a research and educational enterprise of the Drexel University School of Education.