Susan Skidmore's citations do remind us [Thank you for that alert!] that the issues of educational research sometimes qualifying as being "scientific" actually are part of a broader " ... context of how the quality of education research is perceived both within and outside the field ...." Indeed, the NRC Center for Education Committee was created in response to the federal government's implied concerns that educational research all too often has not been duly credible. My own concern with the NRC report is that it seems pre-destined to nurture disastrous perceptions, "... both within and outside the field ..." , that anything allowed by that report is "scientific" educational research.
I am a bit confused by Schremmer's "only" [ in (2)] ... since there would seem to be other ways of using mathematical knowledge. But everything after his "... that bothers me, namely its total reliance on experimentation " is well said ... and is very central to the issue of posing any *criteria* for the science-ness of educational research. To expand on his point:
Research in mathematics instructology traditionally has relied on undue hopes that, as outcomes of piecemeal explorations, resulting "bricks and boards" statistical data might somehow serve as *sources* for later construction of major instructological theories ... the ultimate being scientific theories of instructology. Generations of persistence with such hopes ... indeed, such grievously misplaced faith ... have been largely futile and horribly wasteful of research-resources. Its a bit like stockpiling home-construction materials in hopes of someday owning a dwelling ... collected long before knowing what property might be used, or owning a vision of the building, or knowing what materials would be needed, or having the know-how for how to assemble what you already have.
For sure, such fragment-studies often are informative, or (even much more often) are at least suggestive. And it sometimes is possible for such a fragment-study to identify some specific *scientific* truth, and to disclose how to continually confirm it. [E.g.: Every "normal" American child can learn to recite the English chant for the Arabic numerals, zero through one hundred, before reaching ten years of age ... (by definition) a scientific fact; annually verified .] Thus, we might gain some assurance that at least some of the bricks and boards can withstand the stresses that are imposed by scientific theory. Nonetheless, without an amalgamating, motivating, nourishing, guiding, conceptual framework, all bricks-and-boards statistical data are just that ... and nothing more. [Perhaps that is one reason why educational researchers invest so much effort and space in referring to other educational researchers.]
So far, all education-research efforts to use question-driven experimentation (a la how "the scientific method" is naively taught in many schools) have miserably failed to generate any viable theory of mathematics instructology ... much less a science of mathematics instructology.
Schremmer is right. Instructological research cannot be made "scientific" by continuing to focus it on statistical results from fragmentary experiments. Even though a trifle of experiments might generate some genuinely scientific facts, even those can become important or significant only if couched within the context of some conceptual "schema."
Mackey's alert that "The IES requires randomized cluster trials now." seems to be consistent with the experimental/statistical orientation of traditional educational research in mathematics instructology. For sure, the reliability of experimental statistics should be maximized by requiring scientifically reliable modes of sampling. But if those experiments/statistics serve only to establish fragmentary findings ... even if scientific truths ... such results might say nothing of consequence (... yielding better data for confirming essentially useless results).
Sorely needed is a widely accepted, way-paving, pace-setting model of how a genuinely scientific educational theory which can assimilate, accommodate, motivate, and orient important and significant elements and pursuits. The most fruitful place to begin is with widespread collaboration on developing a genuine science of instructology.
Cordially, Clyde -------------------------------------------------- From: "Alain Schremmer" <email@example.com> Sent: Wednesday, October 31, 2012 12:41 PM To: <firstname.lastname@example.org> Subject: Re: In Defense of the NRC's "Scientific Research in Education'"
> > On Oct 31, 2012, at 9:37 AM, Susan S wrote: > >> The following articles may be particularly relevant to the current >> thought provoking discussion. >> >> "Propagation of Misinformation About Frequencies of RFTs/RCTs in >> Education: A Cautionary Tale" >> http://edr.sagepub.com/cgi/content/full/41/5/163?ijkey=M7uRdX7PI5G2g&keyty >> pe=ref&siteid=spedr >> >> "Things (We Now Believe) We Know" >> http://edr.sagepub.com/cgi/content/full/41/5/177?ijkey=7FInHgkj8RK6g&keyty >> pe=ref&siteid=spedr >> >> warm regards, >> Susan > > (1) I do not quite see how so. It seems to me that, as the title > indicates, the article is about "[p]ropagation of misinformation" rather > than about what constitutes "Scientific Research in Education'". Not to > say that this is not important. It is. But I don't think that this is the > point being argued here. > > (2) To return to the subject of "Scientific Research in Education'", it > seems to me that there is an inherently huge difficulty in experimental > designs in mathematics education in that the result of the learning of > mathematics can only appear in the learning of further mathematics. Else > we are not talking about mathematics but whatever else. > > And then there is something else about most of the "Scientific Research > in Education'" that I have read that bothers me namely its total reliance > on experimentation. But Physics, and now even Chemistry, do not rely only > on experimentation but, at the very least, on "common sense" aka theory, > that is on statements (logically) entailed from statements already > verified. In fact, these days, it seems that experimentation is mostly > used to verify/falsify statements made on theoretical bases. > > Of course, there are basic statements in physics whose truth can only be > verified by their consequences. The question then is how these basic > statements are chosen in the first place. But none were chosen out of > blind experimentation. Galileo had a pretty good idea of what he was > looking for. And, if the Cern was looking for the Higgs boson, it was > because its existence seemed to appear necessary on logical grounds. (I > say "seems" because the logical entailment in this case is > extraordinarily complicated.). And, yes, Kepler used Tycho Brahe's data. > But it was somewhat of a rare case. > > None of the above, though, seems to take place in "Scientific Research in > Education'". For example, it seems to me that to assume that memorization > can play much of a role in the learning of mathematics is a waste. And to > want this to be ascertained only by experimentation is akin to accepting > that eagles can fly better than elephants only after a few thousands > randomly chosen eagles and elephants have been dropped from randomly > chosen heights, 127m, 913m, etc > > And, just to make sure, I am not saying that nothing but Bourbaki will > do. The logic to be invoked with students depends on the students, in > particular on what point of their mathematical education they have > reached. Just like how intricate an attorney's reasoning in front of a > jury is going to be depends on how educated the jury is. Which is why, > many times, the jury has to be educated before the case can be made. > > Regards > --schremmer > **************************************************************************** > * To post to the list: email email@example.com * > * To unsubscribe, email the message "unsubscribe mathedcc" to > firstname.lastname@example.org * > * Archives at http://mathforum.org/kb/forum.jspa?forumID=184 * > ****************************************************************************