From the Wall Street Journal, Saturday, January 5, 2013.
THE NUMBERS GUY
Don't Let Math Pull the Wool Over Your Eyes
By Carl Bialik
This column will make the case that many people, including holders of
graduate degrees, professional researchers and even editors of
scientific journals, can be too easily impressed by math. A
mathematical model (Tpp = T0 - T0d2f - Tpdf) is developed to
describe sequential effects.
Did that second sentence make the first more persuasive? It did for
most participants in a recent intriguing experiment whose result
suggests people often interact with math in a way that isn't very
logical. Other research has shown that even those who should be
especially clear-sighted about numbers-scientific researchers, for
example, and those who review their work for publication-are often
uncomfortable with, and credulous about, mathematical material. As a
result, some research that finds its way into respected journals-and
ends up being reported in the popular press-is flawed.
In the latest study, Kimmo Eriksson, a mathematician and researcher of
social psychology at Sweden's Mälardalen University, chose two
abstracts from papers published in research journals, one in
evolutionary anthropology and one in sociology. He gave them to 200
people to rate for quality-with one twist. At random, one of the two
abstracts received an additional sentence, the one above with the math
equation, which he pulled from an unrelated paper in psychology. The
study's 200 participants all had master's or doctoral degrees. Those
with degrees in math, science or technology rated the abstract with
the tacked-on sentence as slightly lower-quality than the other. But
participants with degrees in humanities, social science or other
fields preferred the one with the bogus math, with some rating it much
more highly on a scale of 0 to 100.
"Math makes a research paper look solid, but the real science
lies not in math but in trying one's utmost to understand the real
workings of the world," Prof. Eriksson said.
Several of the authors of the three papers whose material Prof.
Eriksson harvested said they generally agreed with that conclusion.
"Disciplines with less math emphasis could benefit from more
exposure to mathematical logic so that when it is used it is accepted
for its rigor rather than because it awes the reader," said Geoff
Kushnick, co-author of the evolutionary anthropology paper, published
Prof. Eriksson's finding, published in November in the journal
Judgment and Decision Making under the title "The Nonsense Math
Effect," is preliminary but unfortunately not surprising, other
researchers said. It documents a familiar effect, said Daniel
Kahneman, professor emeritus of psychology and public affairs at
Princeton University. "People who know math understand what other
mortals understand, but other mortals do not understand them. This
asymmetry gives them a presumption of superior ability."
Prof. Kahneman and other scholars have documented how numbers can warp
rather than enhance logical thinking. For example, German researchers
asked 52 lawyers to propose sentences for hypothetical repeat
shoplifters after having rolled dice to determine the hypothetical
sentencing demand by prosecutors. Lawyers who rolled dice that showed
a higher requested sentence tended to hand out longer sentences
themselves, even with identical facts in the case and the knowledge
that the demanded sentence was entirely random. Tying decisions to a
proposed number, even when it is random or misguided, is known by
psychologists as anchoring and helps explain why a "50% off"
sale can seem so compelling even when the original price was vastly
Prof. Eriksson's study shows how people can stumble when exposed to
math outside their field of expertise. Articles in scientific
journals, however, typically are reviewed by experts who read the
whole paper, not just abstracts. Yet even here, there is evidence that
professional researchers often botch statistics in their work, but are
allowed to publish problematic data. Writing in Nature last month,
cell biologist David L. Vaux decried the continuing publication,
including in Nature, of papers with problems such as miscalculated
margins of error. "The fact that these scientifically sloppy
papers continue to be published means that the authors, reviewers and
editors cannot comprehend the statistics, that they have not read the
paper carefully, or both," said Prof. Vaux, of the University of
Melbourne in Australia.
Martin T. Wells, a biostatistician at Cornell University, has another
explanation, chalking up occasional sloppiness in statistics to the
pressure to publish early and often. "It's a bit frustrating,"
he said. "As a biostatistician you want to do things right, and
some researchers just want to get off on the cheap analyzing
Alan Sokal, the physicist who in 1996 hoaxed a journal of cultural
studies with an article he wrote as a jest that the journal published,
called Prof. Eriksson's study "very interesting." He would
like it replicated with a more credentialed group, such as university
professors, who are more likely to review articles for
Prof. Eriksson said he "would love to conduct a follow-up study
with other samples," adding his paper didn't itself contain any
deliberately faulty statistics to test whether readers would notice.
"I wasn't clever enough to think of putting in some nonsense math
of my own," he said.
A version of this article appeared Jan. 5, 2013, on page A2 in some
U.S. editions of The Wall Street Journal, with the headline: Don't Let
Math Pull the Wool Over Your Eyes.
Jerry P. Becker
Dept. of Curriculum & Instruction
Southern Illinois University
625 Wham Drive
Mail Code 4610
Carbondale, IL 62901-4610
Phone: (618) 453-4241 [O]
(618) 457-8903 [H]
Fax: (618) 453-4244