Some subscribers to
MathEdCC might be interested in a recent post "Randomized Control
Trials: The Strange Case of the Contradictory Graphs (was In Defense
of the NRC's 'Scientific Research in Education')" [Hake (2012)].
The abstract reads:
Skidmore at <http://bit.ly/Uov4sU> alerted the Math-Teach list
to her valuable articles with Bruce Thompson in the June/July 2012
issue of the "Educational Researcher": (a) "Propagation
of Misinformation About Frequencies of RFTs/RCTs in Education: A
Cautionary Tale" [Skidmore & Thomson (2012a)] at
<http://bit.ly/SPN361>, and (b) "Things (We Now Believe) We
Know" [S&T (2012b)] at <http://bit.ly/ZSQ5v5>.
discuss the CONTRADICTORY GRAPHS of cumulative numbers of Randomized
Control Trials (RCT's) vs time for Criminology, Education, Psychology,
and Social fields (showing education first, tied for second, and
last) presented by influential scholars in prominent settings [that,
along with the attendant sequence of events] "may have
gratuitously damaged the already fragile reputation of education
research as a field."
After reviewing the
history, S&T (2012b) conclude: "We believe that the errors were
unintentional . . . . . . "But the history as recounted
by S&T (2012a)
[and in the same "Educational Researcher" issue
by Robinson (2012)
<http://bit.ly/WHhdiU> and Petrosino (2012) at
<http://bit.ly/SU3K3O>] seems to contradict S&T's conclusion.
Cook submitted an
article with the title "A critical appraisal of the case against using
experiments to assess school (or community) effects" [Cook (2001a)] at
<http://bit.ly/Uyd3CY> with NO GRAPH to the Hoover Institution's
"Education Next" <http://educationnext.org/>.
Evidently without Cook's knowledge, his academic article was heavily
edited and published as "Sciencephobia: Why education researchers
reject randomized experiments" [Cook (2001b)] at
<http://bit.ly/SQox50> WITH A GRAPH of cumulative numbers of Randomized Control
Trials (RCT's) vs time for Criminology, Education, Psychology, and
showing education LAST, consistent with the provocative new title. The
graph was erroneously attributed to Boruch, De Moya, & Snyder
(2001) - the data should have been 2002) - at
<http://bit.ly/UoX3sA>, despite the fact that the Boruch et al.
graph showed education tied for second, not last. Are we to believe
that Education Next's degradation of the accurate academic Cook
(2001a) to the inaccurate hooverized Cook (2001b) was
A side issue: to those who regard RCT's as the "gold standard"
of education research, the higher the curve of cumulative numbers of
Randomized Control Trials (RCT's) vs time for a field, the higher the
merit of research in that field. But not everyone would agree -
see e.g., "A
Summative Evaluation of RCT Methodology: & An Alternative Approach
to Causal Research" [Scriven (2008] at
<http://bit.ly/93VcWD>, "Seventeen Statements by
Gold-Standard Skeptics #2" [Hake (2010)] at
<http://bit.ly/TNpTR9>, and the present signature quote of
Thomas Cook and Monique Payne.
To access the
complete 46 kB post please click on <http://bit.ly/TQdfhX>.
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: <http://bit.ly/a6M5y0>
Links to Socratic Dialogue Inducing (SDI) Labs:
education, there is almost nothing of proven
Whitehurst, former director, Institute of Education Sciences,
by Sharon Begley (2004)
"In some quarters, particularly medical ones, the randomized
experiment is considered the causal 'gold standard.' It is clearly not
that in educational contexts, given the difficulties with implementing
and maintaining randomly created groups, with the sometimes incomplete
implementation of treatment particulars, with the borrowing of
some treatment particulars by control group units, and with the
limitations to external validity that often follow from how the random
assignment is achieved."
- Thomas Cook and Monique Payne in "Evidence Matters"
[Mosteller & Boruch (2002)]
"Physics educators have led the way in developing and using
objective tests to compare student learning gains in different types
of courses, and chemists, biologists, and others are now developing
similar instruments. These tests provide convincing evidence that
students assimilate new knowledge more effectively in courses
including active, inquiry-based, and collaborative learning, assisted
by information technology, than in traditional
- Wood &
REFERENCES [URL's shortened by <http://bit.ly/> and accessed on
17 Nov 2012.]
Begley, S. 2004.
"To Improve Education, We Need Clinical Trials To Show What
Works," Wall Street Journal, 17 December, page B1; online as a 41
kB pdf at <http://bit.ly/SSmaym>, thanks to David
Hake, R.R. 2012.
"Randomized Control Trials: The Strange Case of the Contradictory
Graphs (was In Defense of the NRC's 'Scientific Research in
Education')" online on the OPEN! AERA-L archives at
<http://bit.ly/TQdfhX>. Post of 17 Nov 2012 10:45:11-0800 to
AERA-L and Net-Gold. The abstract and link to the complete post are
being transmitted to several discussion lists and are also on my blog
"Hake'sEdStuff" at <http://bit.ly/Uy5ufa> with a
provision for comments.
Mosteller, F. &
R. Boruch, eds. 2002. "Evidence Matters: Randomized Trials in
Education Research." Brookings Institution, publisher's
information at <http://bit.ly/UoX3sA>. Amazon.com
information at <http://amzn.to/n6T0Uo>. An expurgated
Google book preview is online at
Wood, W.B., &
J.M. Gentile. 2003. "Teaching in a research context,"
Science 302: 1510; 28 November; online as a 213 kB pdf at
<http://bit.ly/SyhOvL> thanks to Portland State's