The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Education » math-teach

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: [Hake's] Is "Education Research" "Scientific Research" ? YES!"
Replies: 4   Last Post: Oct 22, 2012 1:48 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
GS Chandy

Posts: 8,307
From: Hyderabad, Mumbai/Bangalore, India
Registered: 9/29/05
Re: [Hake's] Is "Education Research" "Scientific Research" ? YES!"
Posted: Oct 21, 2012 1:11 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

Clyde Greeno posted Oct 20, 2012 12:21 PM (GSC's remarks precede and are interspersed):

It is refreshing indeed to read a counter-argument that is based on a rational examination of the matters in question. I am not able to (nor am I qualified to) judge the 'scientific validity' of Jo Boaler's work in 'Education Research' - as I have read nothing by her except "When Academic Disagreement Becomes Harassment and Persecution"; also, in current circumstances, I am not in any case able to investigate and judge 'scientific validity' of a piece of 'educational research'). Hence my remarks are based purely on a reasoned examination of Clyde Greeno's remarks in his message:

> Is Hake's "Yes!" defensible???
> Even without reference to Boaler's papers, the NRC's
> Education Center's criteria for "scientific
> educational research" (as published by the National
> Academies Press) might be worth examining in some
> detail ... before even attempting to use them in
> support or denial of the scientific credibility of
> research by Boaler or anyone else.
> Of course, the over-riding question is of how well
> the Education Center's criteria characterize
> "scientific research in education." For, those
> criteria appear to be glibly superficial and badly
> unknowing about the nature of science.

I've not read the publication on the criteria for evaluating "scientific educational research". (I shall try and get hold of this publication, but I'm somewhat handicapped in this respect of access to an adequate library).
> (1) Pose Significant Questions That Can Be
> Investigated Empirically:
> Scientific knowledge does not advance only through
> merely posing "questions."

True enough. But the publication in question does seem to have brought up several criteria to attention.
>All substantial scientific
> advancements have come through the development and
> verification of scientific *theories* which describe
> whatever phenomena they respectively attend. True,
> "questions" have their place in motivating particular
> pursuits within the context of specific scientific
> theories. But to assert that all scientific theories
> are developed through pursuing specific "questions"
> is a bit far fetched and much too scholastic for
> serious scientists to swallow. Rather, that assertion
> appears to be the Center's defense of educators'
> claims that their "boards and bricks" kinds of
> fragmentary investigations can in some ways be
> "scientific."

I do believe that all scientific advances (if not entire theories) have come about through 'scientific questions' which must have occurred to specific investigators. (I believe a good case might be made to broaden this assertion to include "all scientific theories" as well, but I have no references to provide for this; could you perhaps provide some references to 'scientific theories' that have developed NOT through some scientific questions or the other?).
> In the macrocosm, what is the scientific theory which
> Boaler's works serve to advance? In the microcosm,
> what are the "questions" that she has pursued? Are
> they genuinely *scientific* questions about specific
> phenomena that are sufficiently well-defined so that
> other researchers can readily focus on exactly the
> same questions? ... or are they too nebulous to
> determine what her "answers" actually say, if
> anything.

I'm not in a position to respond to those questions as I've not studied Boaler's works in 'Education Research' (ER, if I happen to refer to it in future).
> And when do the Education Center's scientific
> questions become "significant"? When the answers open
> the way for progress ... if not for mankind, then at
> least for further development of the scientific
> theory. No theory? ... no significance. No leads
> toward important future developments? ... no
> significance.

I'll buy the above most enthusiastically.
> The scientific importance, significance, and
> credibility of Boaler's works (or anyone else's) can
> be assessed only within the context of how
> well-defined her scientific "questions" are, within
> the broader context of whatever scientific theories
> she is helping to develop.

Fair enough.
> Traditionally, educational research has badly failed
> to be "scientific", because it typically has been
> about collecting "bricks and boards data" that merely
> hint at possible truths about nebulous populations or
> phenomena ... rather than about developing
> g scientific theories about well defined kinds of
> entities.

Probably true.
> [My personal skepticism about the
> science-ness of traditional "educational research"
> precludes my own serious investigations into her
> works.]

I could agree that much of what I've seen of 'traditional educational research' is lacking in 'science-ness'. I've not examined enough of the literature to be 'universally skeptical' abut ER: but it is entirely possible that you are not incorrect to be skeptical.

However, I do not agree that your skepticism precludes any serious investigation by you: in fact, it is probably the right attitude to have towards ANY claim of scientific advances - with the proviso that it is NOT a 'blanket skepticism' one should hold, but a 'healthy skepticism'. Of course, we do need to define both terms 'blanket' and 'healthy' to describe the quality of the skepticism in question (but I shall leave that aside at this time!)
> (2) Link Research to Relevant Theory
> Not just to "relevant theory", but specifically to
> the *encompassing scientific* theory. To speak of
> "scientific research" within the context of badly
> non-scientific "theories" ... that are no more than
> personal/committee/community opinions ... is a gross
> misuse of the label.

> Only if/whenever Boaler's works/questions/answers can
> be regarded within the context of an emerging, but
> definitive scientific theory can they be regarded as
> being *scientific findings*. [True, some persons
> might feel that they perceive a scientific theory
> into which her works might fit ... but unless/until
> she provides such a scientific context, her works do
> not qualify as being "scientific." [Again, my
> skepticism about traditional "educational research"
> precludes my investigations into her works.]

I agree (taking into account my proviso above about your skepticism disqualifying you).
> (3) Use Methods That Permit Direct Investigation of
> the Question,
> Methods which *others* can use for achieving *the
> same (well-defined) answers* to the same,
> well-defined "questions" ... or, in the broader
> context, methods which invariably yield exactly the
> same findings, about phenomena of essentially the
> same kind. Boaler's "conclusions" can be
> scientifically credible only if her "experiments" can
> be replicated by others ... and invariably produce
> the same results. Only if the nature and proceedings
> of her investigations are so clear that others can do
> as she did, can she hope for her works to be
> scientifically credible. [Again, my skepticism about
> traditional "educational research" precludes my
> investigations into her works.]

Yes - but see remarks made earlier.
> (4) Provide a Coherent and Explicit Chain of
> Reasoning
> Of *scientific* reasoning ... done within the context
> of an evolving scientific theory. Such a theory must
> clearly describe the kind of things it is talking
> about ... clearly define its crucial concepts ...
> clearly specify its established facts and conclusions
> ... and combine all of those with some newly acquired
> concepts/facts to logically derive the "answers" to
> the newly posed "questions." If/when Boaler's
> conclusions have that kind of theoretical basis, they
> thereby become admissible as candidates for
> scientific credibility. [Again, my skepticism about
> traditional "educational research" precludes my
> investigations into her specific works.]

Yes - but see above.
> (5) Replicate and Generalize Across Studies
> The scientific *studies* must be replicable ... and
> their *results* must generalize across all such
> studies. But true reliability and generality can
> happen only to the extent that the "studies" and
> results are well-defined within the context of an
> evolving scientific theory. Nebulous "studies" are
> of no scientific value ... and their "findings" enjoy
> no scientific credibility. Likewise for educators'
> "bricks and boards" data-collections, in general.
> . But the attacks on her "studies" and the "findings"
> of her experiments are scientifically naïve (or
> r scientifically irresponsible) straw men.
> Identifying weaknesses in her methodologies cannot
> (as the challenges should know) disprove her claims
> of generality. The only responsible mode of refuting
> her "findings" lies in subsequent studies "of the
> same" kind failing to produce findings of her kind
> ... or in producing even findings that directly
> contradict hers. Otherwise, their attacks on her
> experimental methods can be regarded only as being
> non-scientific efforts to uphold some contrary
> beliefs. On the other hand, her only academic defense
> lies in subsequent studies "of the same" kind
> producing findings "of the same kind." Endeavors of
> either kind necessarily rely on her studies being so
> well defined that they can be replicated by others.
> [Again, my skepticism about traditional "educational
> research" precludes my investigations into her
> specific works.]

Yes - but see above.
> (6) Disclose Research to Encourage Professional
> Scrutiny and Critique.
> "Disclose research"??? ... whatever might the
> Education Center mean by that!? My best guess is that
> they should mean "disclose the theoretical
> framework, the motivations for the particular
> investigations, the procedures used for conducting
> the investigations, the conclusions, the rationale
> for reaching the conclusions, and any empirical
> verifications of those conclusions" ... all of which
> are to be done within the context of the evolving
> scientific theory. For, these six criteria are
> supposed to be "... six guiding principles that
> underlie all scientific inquiry, including education
> research. [Again, my skepticism about traditional
> "educational research" precludes my investigations
> s into her specific works.]

Yes - but see above.
> I am even more skeptical about trying to use the
> Education Center's description of "scientific
> educational research" as a basis for supporting or
> denying the scientific-ness of any educational
> research effort ... or of anything else, for that
> matter. I am amazed that the NRC would allow such a
> publication.

As stated earlier, I must try and get hold of that publication and look at it seriously - and then my comments would perhaps have some specific validity.
> So what does the above analysis say about Boaler's
> works? Absolutely nothing ... except that they cannot
> be scientifically credible without being
> scientifically definitive, within the context of a
> scientific theory! What about the challengers?
> Absolutely nothing ... except that they cannot be
> scientifically credible without being scientifically
> definitive within the context of a scientific theory!
> Their sociological disputes over her methods of
> educational research have no scientific meanings.

I believe you've provided a very good context to investigate the value (or lack of it) of Boaler's work. In fact, I'd believe that she'd do well to ask you to investigate her findings (as a counter to the Milgram/Bishop findings. Because you have been skeptical, any positive findings from you would have special value).
> About Hake's, "Yes!": If Richard knows of some
> evolving scientific theory to which Boaler has
> contributed to to its development, I'm all ears ...
> not so much because of her own contributions, as
> because of the earthshaking importance and
> significance of any such scientific theory.

I'd agree with the above if you would remove the (perhaps pejoratively intended) adjective 'earthshaking'. A piece of research does not have to be earthshaking in order to be important - and Jo Boaler's work might well be important. So far as I know, she claims to have demonstrated:

"...students who engage actively in their mathematics learning, rather than simply practicing
procedures, achieve at higher levels".

To me the above seems obviously true. If she has actually done so, it is an important finding (if this assertion is being denied by the 'traditionalists'). I frankly lack the resources or time to investigate Jo Boaler's research.

I have found that Professor Bishop has consistently displayed the most unscientific of attitudes in his interactions with me at this forum. If he has displayed the same attitude in respect of his critique of Jo Boaler's work, then he deserves to be criticized for that. Professor Milgram has not, in my opinion, done himself any favors by associating himself with Professor Bishop.
> Methinks
> that, unfortunately, Richard's "Yes!" stems not from
> science, as such, but from his other instructological
> information which has yet to congeal as a bona fide
> e science.

This is possibly true.
>If so, Boaler's "findings" (whatever her
> methodology) might somehow help nurture the
> development of such a science. But unless such a
> science already hides within an inner sanctum of
> mathematics instructology, it is worse than rash to
> claim that her findings are scientifically reliable.

As noted, I've not read Boaler's work - except for "When Academic Disagreement Becomes Harassment and Persecution" (which is not alluded to as being a piece of her research). This document (I feel) may have some credibility, for reasons expressed above.
> Yes, there is growing, scientifically reliable
> evidence that some specific kinds of
> mathematics-instruction "reforms" will result in
> easier, more productive ways of achieving even
> traditional goals of curricular mathematics
> education, plus others. Some of those reforms are
> clearly in harmony with some of Hake's contentions.
> But that is quite a different matter from proclaiming
> specific educational research efforts to be in some
> way "scientific." We (the world) are more than
> anxious to learn of whatever educational research
> efforts qualify as being genuinely "scientific." They
> are not at all easy to locate.

There is plenty of evidence available to indicate that most students leave school fearing/loathing math; and that (in the US or in India) it is entirely to state "I'm useless at math!" [while it certainly NOT acceptable to state "I am illiterate!"]. In my opinion, for a literate person to state "I'm useless at math!" indicates some serious societal defect.

I'd generally accept much of Clyde Greeno's reasoning. However, the way Professors Milgram and Bishop have gone about their 'demolition job' on Jo Boaler is not, in my opinion, an appropriate way to critique or disprove her findings. The underlying issue here is the unprofessional and 'unprofessorial' attitudes of Professors Milgram and Bishop.

By the way, I'd love for you to cast your skeptical eye on my development 'One Page Management System' (OPMS) [which has considerable significance for, amongst other things, ER - and for education as a whole]. I attach herewith a list of some Missions for the OPMS (attempted successfully and unsuccessfully; still ongoing; proposed). Some of these proposed Missions could be, I claim, 'earthshaking' (when they come to be). Some information about the OPMS is available as attachments to my initial posting at the thread "On realistic and practical tools for the US education system..." -

In conclusion, I'd answer the question in the title of this thread as follows: "Education Research" (as it should be practiced) is certainly "Scientific Research!"


Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.