In my earlier response to Hake's "Yes!", I am truly not challenging (or even talking about) Boaler or Bishop or Milgram. I am disputing the credibility of the NRC's Education Center's attempts to defend at least some of educational research (to date) as being in some way "scientific." I certainly would not claim that no educational research could be genuinely scientific research ... for I am personally engaged in furthering the (managerial) science of mathematics instructology. But most of what the Center would call "scientific educational research" is far from scientific. The appropriate answer to Hake's question is, "Perhaps sometimes, but very rarely."
To clarify my meaning for "[My personal skepticism about the science-ness of traditional "educational research" precludes my own serious investigations into her works.]:
My personal skepticism does not "disqualify" me studying educational research ... it disqualifies most of educational from warranting my study. The latter is precluded simply because I do not have the time, energy, or interest to deeply delve into "educational research" reports, in order to detect whether or how they might somehow fit into some scientific theory which is not disclosed by the researchers (and probably not even conceived by them). Unless the report begins by disclosing the scientific theory within which it is reported, it cannot be a *scientific* contribution.
Re. GS's, "... 'scientific theories' that have developed NOT through some scientific questions or the other?)."
Every scientific theory can be *examined* with regard for what "questions" it raises and answers. But I am not enough of a historian to know of some scientific theory that WAS developed to answer a specific question. As far as I know, all were developed to explain and/or predict phenomena. In my view, that requires far more than raising and answering specific questions ... which apparently is what most educational researchers try to do. As far as I know, NO theory was developed to answer specific questions. In fact, curricular descriptions of "the scientific method" are attempts to provide students with an algorithmic process for scientific inquiry ... a bit like trying to teach algorthms for solving word-problems.
Re: " A piece of research does not have to be earthshaking in order to be important - and Jo Boaler's work might well be important."
Please re-read! The disclosure of any genuinely *scientific theory of education* would, indeed, be an earthshaking event. Not all scientific findings are earthshaking contributions. Not all important research-findings are scientific. Even if hers were found to be *important*, that would not make them scientifically credible or reliable ... or qualify her research as being "scientific."
Re: " So far as I know, she claims to have demonstrated: "...students who engage actively in their mathematics learning, rather than simply practicing procedures, achieve at higher levels".
FAR too nebulous to qualify even as a scientific assertion, ... much less qualifying as a scientific fact. "Achieve at higher levels"?#! ... on high-risk scholastic tests? What means "levels" ... for what kind of "achievements"?#! "Higher" than what or whom? "Higher" (on what scale) than the same students otherwise would have achieved through alternative environments/circumstances? That would seem to call for one-to-one matching of students from rigorously matched instructional environments. With that kind of ambiguity, educational researchers never will be able to scientifically prove anything (and rarely even to clearly state it).
"Demonstrated?" ... as in demonstrating the meaning of some truth? ... a near impossibility. "Supports" or "suggests" or "is consistent" ... maybe. If that assertion seems, to you, to be "obviously true" ... then perhaps you could try to explain precisely what that assertion means ... in 500 pages, or less,
Re: "... most students leave school fearing/loathing math; and that (in the US or in India) it is entirely to state "I'm useless at math!" [while it certainly NOT acceptable to state "I am illiterate!"].
Now THAT is a scientific fact ... which can be confirmed by repeated studies. There does exist a managerial science of assessing attitudes held by various populations ... [this is a U.S. election year] ... and your assertion is a well-defined conclusion about relatively well-defined concepts. There also is an emerging (MLD) science of Mathematics-Learning Distress ... within which the same truth has substantial significance.
However, your claim does illustrate that some assertions can be scientific truths, without being discernibly encompassed by an evolving scientific theory. In fact, they might conceivably result from classical educational research. But just as the scientific truth of gravity is learned by every infant from every species of animals, the learning of a scientific truth does not make the learning process to be a process of scientific research.
Re: "By the way, I'd love for you to cast your skeptical eye on my development 'One Page Management System' (OPMS) [which has considerable significance for, amongst other things, ER - and for education as a whole]."
Gladly ... mostly because the scientific theory of mathematics instructology is a *managerial* science ... adapted from the managerial science of operations analysis/research. Our pursuits probably overlap at some points. Bt the moment, my plate is too full for that kind of diversion. How about reminding me in about two months? (:-)
- -------------------------------------------------- From: "GS Chandy" <firstname.lastname@example.org> Sent: Sunday, October 21, 2012 12:11 AM To: <email@example.com> Subject: Re: [Hake's] Is "Education Research" "Scientific Research" ? YES!"
> Clyde Greeno posted Oct 20, 2012 12:21 PM (GSC's remarks precede and are > interspersed): > > It is refreshing indeed to read a counter-argument that is based on a > rational examination of the matters in question. I am not able to (nor am > I qualified to) judge the 'scientific validity' of Jo Boaler's work in > 'Education Research' - as I have read nothing by her except "When Academic > Disagreement Becomes Harassment and Persecution"; also, in current > circumstances, I am not in any case able to investigate and judge > 'scientific validity' of a piece of 'educational research'). Hence my > remarks are based purely on a reasoned examination of Clyde Greeno's > remarks in his message: > >> Is Hake's "Yes!" defensible??? >> >> Even without reference to Boaler's papers, the NRC's >> Education Center's criteria for "scientific >> educational research" (as published by the National >> Academies Press) might be worth examining in some >> detail ... before even attempting to use them in >> support or denial of the scientific credibility of >> research by Boaler or anyone else. >> >> Of course, the over-riding question is of how well >> the Education Center's criteria characterize >> "scientific research in education." For, those >> criteria appear to be glibly superficial and badly >> unknowing about the nature of science. >> > I've not read the publication on the criteria for evaluating "scientific > educational research". (I shall try and get hold of this publication, but > I'm somewhat handicapped in this respect of access to an adequate > library). >> >> (1) Pose Significant Questions That Can Be >> Investigated Empirically: >> Scientific knowledge does not advance only through >> merely posing "questions." >> > True enough. But the publication in question does seem to have brought up > several criteria to attention. >> >>All substantial scientific >> advancements have come through the development and >> verification of scientific *theories* which describe >> whatever phenomena they respectively attend. True, >> "questions" have their place in motivating particular >> pursuits within the context of specific scientific >> theories. But to assert that all scientific theories >> are developed through pursuing specific "questions" >> is a bit far fetched and much too scholastic for >> serious scientists to swallow. Rather, that assertion >> appears to be the Center's defense of educators' >> claims that their "boards and bricks" kinds of >> fragmentary investigations can in some ways be >> "scientific." >> > I do believe that all scientific advances (if not entire theories) have > come about through 'scientific questions' which must have occurred to > specific investigators. (I believe a good case might be made to broaden > this assertion to include "all scientific theories" as well, but I have no > references to provide for this; could you perhaps provide some references > to 'scientific theories' that have developed NOT through some scientific > questions or the other?). >> >> In the macrocosm, what is the scientific theory which >> Boaler's works serve to advance? In the microcosm, >> what are the "questions" that she has pursued? Are >> they genuinely *scientific* questions about specific >> phenomena that are sufficiently well-defined so that >> other researchers can readily focus on exactly the >> same questions? ... or are they too nebulous to >> determine what her "answers" actually say, if >> anything. >> > I'm not in a position to respond to those questions as I've not studied > Boaler's works in 'Education Research' (ER, if I happen to refer to it in > future). >> >> And when do the Education Center's scientific >> questions become "significant"? When the answers open >> the way for progress ... if not for mankind, then at >> least for further development of the scientific >> theory. No theory? ... no significance. No leads >> toward important future developments? ... no >> significance. >> > I'll buy the above most enthusiastically. >> >> The scientific importance, significance, and >> credibility of Boaler's works (or anyone else's) can >> be assessed only within the context of how >> well-defined her scientific "questions" are, within >> the broader context of whatever scientific theories >> she is helping to develop. >> > Fair enough. >> >> Traditionally, educational research has badly failed >> to be "scientific", because it typically has been >> about collecting "bricks and boards data" that merely >> hint at possible truths about nebulous populations or >> phenomena ... rather than about developing >> g scientific theories about well defined kinds of >> entities. >> > Probably true. >> >> [My personal skepticism about the >> science-ness of traditional "educational research" >> precludes my own serious investigations into her >> works.] >> > I could agree that much of what I've seen of 'traditional educational > research' is lacking in 'science-ness'. I've not examined enough of the > literature to be 'universally skeptical' abut ER: but it is entirely > possible that you are not incorrect to be skeptical. > > However, I do not agree that your skepticism precludes any serious > investigation by you: in fact, it is probably the right attitude to have > towards ANY claim of scientific advances - with the proviso that it is NOT > a 'blanket skepticism' one should hold, but a 'healthy skepticism'. Of > course, we do need to define both terms 'blanket' and 'healthy' to > describe the quality of the skepticism in question (but I shall leave that > aside at this time!) >> >> (2) Link Research to Relevant Theory >> Not just to "relevant theory", but specifically to >> the *encompassing scientific* theory. To speak of >> "scientific research" within the context of badly >> non-scientific "theories" ... that are no more than >> personal/committee/community opinions ... is a gross >> misuse of the label. >> > Yes. >> >> Only if/whenever Boaler's works/questions/answers can >> be regarded within the context of an emerging, but >> definitive scientific theory can they be regarded as >> being *scientific findings*. [True, some persons >> might feel that they perceive a scientific theory >> into which her works might fit ... but unless/until >> she provides such a scientific context, her works do >> not qualify as being "scientific." [Again, my >> skepticism about traditional "educational research" >> precludes my investigations into her works.] >> > I agree (taking into account my proviso above about your skepticism > disqualifying you). >> >> (3) Use Methods That Permit Direct Investigation of >> the Question, >> Methods which *others* can use for achieving *the >> same (well-defined) answers* to the same, >> well-defined "questions" ... or, in the broader >> context, methods which invariably yield exactly the >> same findings, about phenomena of essentially the >> same kind. Boaler's "conclusions" can be >> scientifically credible only if her "experiments" can >> be replicated by others ... and invariably produce >> the same results. Only if the nature and proceedings >> of her investigations are so clear that others can do >> as she did, can she hope for her works to be >> scientifically credible. [Again, my skepticism about >> traditional "educational research" precludes my >> investigations into her works.] >> > Yes - but see remarks made earlier. >> >> (4) Provide a Coherent and Explicit Chain of >> Reasoning >> Of *scientific* reasoning ... done within the context >> of an evolving scientific theory. Such a theory must >> clearly describe the kind of things it is talking >> about ... clearly define its crucial concepts ... >> clearly specify its established facts and conclusions >> ... and combine all of those with some newly acquired >> concepts/facts to logically derive the "answers" to >> the newly posed "questions." If/when Boaler's >> conclusions have that kind of theoretical basis, they >> thereby become admissible as candidates for >> scientific credibility. [Again, my skepticism about >> traditional "educational research" precludes my >> investigations into her specific works.] >> > Yes - but see above. >> >> (5) Replicate and Generalize Across Studies >> The scientific *studies* must be replicable ... and >> their *results* must generalize across all such >> studies. But true reliability and generality can >> happen only to the extent that the "studies" and >> results are well-defined within the context of an >> evolving scientific theory. Nebulous "studies" are >> of no scientific value ... and their "findings" enjoy >> no scientific credibility. Likewise for educators' >> "bricks and boards" data-collections, in general. >> . But the attacks on her "studies" and the "findings" >> of her experiments are scientifically naïve (or >> r scientifically irresponsible) straw men. >> Identifying weaknesses in her methodologies cannot >> (as the challenges should know) disprove her claims >> of generality. The only responsible mode of refuting >> her "findings" lies in subsequent studies "of the >> same" kind failing to produce findings of her kind >> ... or in producing even findings that directly >> contradict hers. Otherwise, their attacks on her >> experimental methods can be regarded only as being >> non-scientific efforts to uphold some contrary >> beliefs. On the other hand, her only academic defense >> lies in subsequent studies "of the same" kind >> producing findings "of the same kind." Endeavors of >> either kind necessarily rely on her studies being so >> well defined that they can be replicated by others. >> [Again, my skepticism about traditional "educational >> research" precludes my investigations into her >> specific works.] >> > Yes - but see above. >> >> (6) Disclose Research to Encourage Professional >> Scrutiny and Critique. >> "Disclose research"??? ... whatever might the >> Education Center mean by that!? My best guess is that >> they should mean "disclose the theoretical >> framework, the motivations for the particular >> investigations, the procedures used for conducting >> the investigations, the conclusions, the rationale >> for reaching the conclusions, and any empirical >> verifications of those conclusions" ... all of which >> are to be done within the context of the evolving >> scientific theory. For, these six criteria are >> supposed to be "... six guiding principles that >> underlie all scientific inquiry, including education >> research. [Again, my skepticism about traditional >> "educational research" precludes my investigations >> s into her specific works.] >> > Yes - but see above. >> >> I am even more skeptical about trying to use the >> Education Center's description of "scientific >> educational research" as a basis for supporting or >> denying the scientific-ness of any educational >> research effort ... or of anything else, for that >> matter. I am amazed that the NRC would allow such a >> publication. >> > As stated earlier, I must try and get hold of that publication and look at > it seriously - and then my comments would perhaps have some specific > validity. >> >> So what does the above analysis say about Boaler's >> works? Absolutely nothing ... except that they cannot >> be scientifically credible without being >> scientifically definitive, within the context of a >> scientific theory! What about the challengers? >> Absolutely nothing ... except that they cannot be >> scientifically credible without being scientifically >> definitive within the context of a scientific theory! >> Their sociological disputes over her methods of >> educational research have no scientific meanings. >> > I believe you've provided a very good context to investigate the value (or > lack of it) of Boaler's work. In fact, I'd believe that she'd do well to > ask you to investigate her findings (as a counter to the Milgram/Bishop > findings. Because you have been skeptical, any positive findings from you > would have special value). >> >> About Hake's, "Yes!": If Richard knows of some >> evolving scientific theory to which Boaler has >> contributed to to its development, I'm all ears ... >> not so much because of her own contributions, as >> because of the earthshaking importance and >> significance of any such scientific theory. >> > I'd agree with the above if you would remove the (perhaps pejoratively > intended) adjective 'earthshaking'. A piece of research does not have to > be earthshaking in order to be important - and Jo Boaler's work might well > be important. So far as I know, she claims to have demonstrated: > > "...students who engage actively in their mathematics learning, rather > than simply practicing > procedures, achieve at higher levels". > > To me the above seems obviously true. If she has actually done so, it is > an important finding (if this assertion is being denied by the > 'traditionalists'). I frankly lack the resources or time to investigate Jo > Boaler's research. > > I have found that Professor Bishop has consistently displayed the most > unscientific of attitudes in his interactions with me at this forum. If he > has displayed the same attitude in respect of his critique of Jo Boaler's > work, then he deserves to be criticized for that. Professor Milgram has > not, in my opinion, done himself any favors by associating himself with > Professor Bishop. >> >> Methinks >> that, unfortunately, Richard's "Yes!" stems not from >> science, as such, but from his other instructological >> information which has yet to congeal as a bona fide >> e science. >> > This is possibly true. >> >>If so, Boaler's "findings" (whatever her >> methodology) might somehow help nurture the >> development of such a science. But unless such a >> science already hides within an inner sanctum of >> mathematics instructology, it is worse than rash to >> claim that her findings are scientifically reliable. >> > As noted, I've not read Boaler's work - except for "When Academic > Disagreement Becomes Harassment and Persecution" (which is not alluded to > as being a piece of her research). This document (I feel) may have some > credibility, for reasons expressed above. >>> >> Yes, there is growing, scientifically reliable >> evidence that some specific kinds of >> mathematics-instruction "reforms" will result in >> easier, more productive ways of achieving even >> traditional goals of curricular mathematics >> education, plus others. Some of those reforms are >> clearly in harmony with some of Hake's contentions. >> But that is quite a different matter from proclaiming >> specific educational research efforts to be in some >> way "scientific." We (the world) are more than >> anxious to learn of whatever educational research >> efforts qualify as being genuinely "scientific." They >> are not at all easy to locate. >> > There is plenty of evidence available to indicate that most students leave > school fearing/loathing math; and that (in the US or in India) it is > entirely to state "I'm useless at math!" [while it certainly NOT > acceptable to state "I am illiterate!"]. In my opinion, for a literate > person to state "I'm useless at math!" indicates some serious societal > defect. > > I'd generally accept much of Clyde Greeno's reasoning. However, the way > Professors Milgram and Bishop have gone about their 'demolition job' on Jo > Boaler is not, in my opinion, an appropriate way to critique or disprove > her findings. The underlying issue here is the unprofessional and > 'unprofessorial' attitudes of Professors Milgram and Bishop. > > By the way, I'd love for you to cast your skeptical eye on my development > 'One Page Management System' (OPMS) [which has considerable significance > for, amongst other things, ER - and for education as a whole]. I attach > herewith a list of some Missions for the OPMS (attempted successfully and > unsuccessfully; still ongoing; proposed). Some of these proposed Missions > could be, I claim, 'earthshaking' (when they come to be). Some information > about the OPMS is available as attachments to my initial posting at the > thread "On realistic and practical tools for the US education system..." - > http://mathforum.org/kb/thread.jspa?threadID=2405705. > > In conclusion, I'd answer the question in the title of this thread as > follows: "Education Research" (as it should be practiced) is certainly > "Scientific Research!" > > GSC