I agree with your comments here, and even the ones who carried out this study mention the study's weakness in using TIMSS data to reach these conclusions:
"Still, it is important to keep in mind that our results are limited to student achievement as measured by the 2003 TIMSS test scores in 8th-grade math and science in the United States. Different results might be found for different subjects, grades, or tests. Depending on the teacher, the students, the content taught, or other factors, problem-solving activities could turn out to be the more effective style. Even though lecture-style teaching seems to be a more effective method in middle-school math and science, that does not mean it would be the preferable approach to elementary-school reading.
"Also, our findings are based on student performance on the TIMSS math and science exams, which are designed to measure mastery of factual knowledge of the curricula that schools expect students to learn. Other tests intended to measure problem-solving ability and the competence to apply mathematical and scientific concepts in real-world settings (such as the Programme for International Student Assessment [PISA] administered by the Organization of Economic Cooperation and Development) might yield different results. Unfortunately, we are unable to ascertain whether this might be the case, as PISA did not ask teachers about their pedagogical approach."
Essentially, they are saying that the weakness of this study is in measuring student performance, which is based on test results and test results that emphasize mostly knowing facts. What about the ability to reason mathematically? TIMSS is multiple-choice whereas PISA is not. We can't gather much about their abilities to reason if their reasoning is not written and analyzed. And who knows how much of that reasoning they used works mainly because the test is multiple choice as opposed to free response. Similar results hold for science. And a big weakness is that any test does not measure long-term learning and understanding. In short, the picture may be incomplete.
And the study focuses just on eighth-grade math and science. What about other grade levels? What about other subjects?
Another problem is that this study does not attempt to measure how teachers actually try to implement their teaching styles in the classroom:
"Finally, our information on teaching practices, which is based on in-class time use reported by teachers, does not allow us to distinguish between different implementations of teaching practices. In other words, a certain teaching technique may be very effective if implemented in the optimal way. But the strength of our approach is that it examines which teaching style turns out to be effective, on average, for teachers in general. Optimal teaching methods that cannot be executed by teachers in general may do more harm than good."
I agree: In the hands of an inexperienced teacher or in the hands of a teacher who lacks strong understanding of math, problem-solving and other student-centered approaches to teaching can be disastrous. Also if teachers still emphasize learning to follow rules on template exercises via alternative teaching methods rather than on reasoning and learning to tackle problems for which they are not expected to know how to solve right away, for which they cannot solve by mimicking a solution discussed in class or in their textbook, these alternative methods are probably not going to help much. Focusing on memorization rather than on reasoning and understanding and solving problems generally results in disaster, no matter how we try to go about teaching it (whatever "it" may be).
On 4/19/2011 at 12:03 pm, Ihor Charischak wrote:
> Hi Wayne, > I just read the article you referenced and it > inspired me to comment at that site. Here's what I > wrote: > > What this study has shown is that teaching to the > tests produces better test scores. No one should > question that. But do students better understand the > material as a result? That?s the endless debate > that?s been going on for years. Would I send my kid > to a high powered ?no excuse? school? It depends. If > I want to brag to my friends what a great student I > have in math I might be tempted. But I would prefer a > more liberal education where the "Royal Road to > Calculus" is not just one way (or the highway) > approach as the KIPP schools are prone to brag about. > But also include detours into some fascinating > investigations where students apply their math > skills. In fact, they may be so attracted by the > detour that they may never get back on the Royal Road > and become doctors or lawyers where knowing the > advanced intricacies of Calculus may not be required > for entry. > -Ihor > > Ihor Charischak > CLIME - Booth #326 NCTM Conference 2011 > Council for Technology in Math Education > Website: http://clime.org > Blog: http://climeconnections.blogspot.com > > On Apr 19, 2011, at 11:11 AM, Wayne Bishop wrote: > > > Once again, Reid Lyon got it right back in 2003. > > > http://educationnext.org/eighth-grade-students-learn-m > ore-through-direct-instruction/ > > > > Expect the NCTM and AAAS to change course > immediately. And all pigs to sprout wings. > > > > Wayne