I just got my May, 2001, issue, Vol 32 No. 3, of the NCTM Journal for Research in Mathematics Education. Still shrink wrapped, I said to a colleague who knows quite a lot of statistics, "How many of the articles contained herein are data-based; i.e., using objectively measured evidence based on a sample size that you would consider meaningful, at least as preliminary evidence that might warrant further study? Before he really had a chance to answer, I said, "I take zero," meaning, of course, he wins with one or more. "OK, what do you think should be a minimal sample size to be meaningful; i.e., worthy of space in the most widely circulated mathematics education journal in the world? Just students, not class units, not schools, or the like." He now understands where I'm coming from and conservatively says "50". I slit the plastic and open the journal:
P. 234 "In the Beginning(s), an editorial by Ed Silver P. 236 "An Analysis of Development of Sociomathematical Norms in One-First-Grade Classroom", Kay McClain and Paul Cobb P. 267 "Building on Informal Knowledge Through Instruction in a Complex Content Domain: Partitioning, Units, and Understanding Multiplication of Fractions", Nancy Mack P. 296 "From Preservice Mathematics Teacher Education to Beginning Teaching: A Study in Recontextualizing, Paula Ensor P. 321 Review: "Holistic Perspectives on Instructional Design - A Review of 'Symbolizing and Communicating in Mathematics Classrooms: Perspectives on Discourse, Tools, and Instructional Design'", Patrick W. Thompson P. 331 Telegraphic Reviews P. 331 Announcement
The first and last three, of course, are not supposed to have such data-based evidence but what of the three main articles?
P. 236 One need not go past the title, obviously, but ... From P. 239 we have "The Setting", "eleven girls and seven boys". From the section entitled "Data and Methodology" we have "The data consist of videotape recordings from two cameras of 103 mathematics lessons."
P. 267 From the first words of the abstract we have , "Six fifth-grade students" and more detail from "Methodology", "The sample consisted of three girls (identified in this article by the pseudonyms Abby, Lisa, and Sara) and three boys (hereafter referred to as Adam, Lee, and Sam) who were fifth-grade students in ..." From the section entitled "Data Analysis" we have "all sessions were audiotaped and videotaped. Each day I wrote detailed notes from ..."
P. 296 Again from the first words of the abstract, "This article describes a two-year longitudinal study that tracked seven students through a one-year, full-time, university-based secondary mathematics methods course and into their first year of teaching ..."
I win. I'd have won if he'd have been satisfied with 20. And we also see what "data" means these days, extensively taping an activity and analyzing it to justify one's preconceptions. That doesn't make the preconceptions wrong, of course. It's just that the activity and the publication thereof are not meaningful research in mathematics education. That's what's wrong.