Date: Jul 7, 2014 8:50 AM
Author: Jerry P. Becker
Subject: [ncsm-members] Did Shanghai cheat on PISA?
From The Washington Post, December 15, 2013. See
Did Shanghai cheat on PISA?
By Valerie Strauss
Everybody knows (or should) by now that Shanghai's 15-year-old
students ranked No. 1 on the most recent Program for International
Student Assessment for the second time, which, if you believe the
recently released results, means that a representative sample of kids
got the best scores in the world in reading, math and science. And
15-year-olds in the United States performed no better than average of
65 countries and education systems, which is pretty much what they
always do on these international tests.
There are a lot of reasons to disbelieve that the results mean what
PISA enthusiasts say they do - that the average performance of U.S.
kids means the future of the U.S. economy and national security is at
stake (here's a good explanation) - but new questions have been
raised about whether the Shanghai students who took the 2012 PISA
really are representative of the city's population of 15-year-olds.
The Organization for Economic Cooperation and Development, which
sponsors PISA and released the 2012 data earlier this month, has said
that the schools that were used in the sample do represent the city's
15-year-old population and that people who say otherwise don't
understand changes in Shanghai policy that have allowed migrant
students from rural parts of China to enroll in the city's schools.
But one researcher has called out the OECD on this subject, and some
China experts say that migrant children are still routinely excluded
from schools in Shanghai, which is wealthier than the rest of China
and is not itself representative of the rest of China (though OECD
officials have said that unpublished results of some rural areas of
China from where the Shanghai migrants presumably came show good
Tom Loveless, a senior fellow at the Brown Center on Education Policy
at the Brookings Institution, who is a former teacher and Harvard
University professor, wrote a scathing piece on the Brookings Web
site titled, "Attention OECD-PISA: Your Silence on China is Wrong."
He says that even though official Shanghi policy changed in 2008 to
ostensibly allow migrant children to enroll in city schools up to the
age of 14, most of them are still excluded anyway - and it is nearly
impossible for migrant students to enroll in the city's high schools.
This is a result, he said, of China's hukou system, which assigns
what amounts to every Chinese a registration of sorts in a home city
or village. Migrants into a city are barred from that city's public
services unless official policy changes to allow participation.
Loveless wrote in part [See
Shanghai is portrayed as a paragon of equity in PISA publications. A
2010 OECD publication, Strong Performers and Successful Reformers in
Education, highlights model systems that the world should emulate.
Shanghai is singled out for praise. One section on Shanghai is
entitled, "Ahead of the pack in universal education." The city is
described as an "education hub," and the only discussion that even
remotely touches upon migrants is the following:
"Graduates from Shanghai's institutions are allowed to stay and work
in Shanghai, regardless of their places of origin. For that reason,
many 'education migrants now move to Shanghai mainly to educate
their children.' "
That description is surreal. PISA's blindness to what is really going
on in Shanghai was also evident in the official release of PISA's
latest scores. The 2012 data appear in volumes organized by themes.
Volume II is entitled, PISA 2012 Results: Excellence through Equity,
Giving Every Student the Chance to Succeed. Shanghai is named as one
jurisdiction where schools "achieve high mathematics performance
without introducing greater inequities in education outcomes (p. 28)"
and one with "above average socio-economic diversity (p. 30)." In the
336 pages of this publication on equity, the word "migrant" appears
only once, in a discussion of Mexico. The word "hukou" does not
appear at all.
Andreas Schleicher, the OECD deputy director for education and skills
and special adviser on education policy to the OECD's secretary
general, wrote a post on the organization's website entitled, "Are
the Chinese cheating in PISA or are we cheating ourselves?" that
counters such criticism about Shanghai. He writes that policies that
keep migrants out of Shanghai's schools have "long changed," and that
"resident migrants were covered by the PISA samples in exactly the
way they are covered in other countries and education systems." [See
Still, it seems to be easier to cling to old stereotypes than keep up
with changes on the ground (or to read the PISA report).
True, like other emerging economies, Shanghai is still building its
education system and not every 15-year-old makes it yet to high
school. As a result of this and other factors, the PISA 2012 sample
covers only 79% of the 15-year-olds in Shanghai. But that is far from
unique. Even the United States, the country with the longest track
record of universal high-school education, covered less than 90% of
its 15-year-olds in PISA - and it didn't include Puerto Rico in its
PISA sample, a territory that is unlikely to have pulled up U.S.
(Puerto Rico isn't part of the United States so, in fact, it wouldn't
make sense to include it in a U.S. sample, but let's move on.)
I asked Post reporters who have covered China about Shanghai, and
they said that despite the 2008 change in Shanghai policy, things
have improved slightly for migrant families but not very much.
Children who do not have Shanghai registration can now in theory get
into Shanghai public schools, but only through very complicated and
often arduous means that require either a brilliant kid, valuable
inside connections or bribe money. (See this Washington Post story
that talks about how Chinese parents pay bribes to get their children
into top schools -
And while the government recently announced that the hukou
registration system is going to be reformed in the next few years,
this does not mean families can suddenly get registered in big cities
such as Shanghai.
This Sept. 10, 2013, story in the South China Morning Post is
revealing about attitudes toward migrant children in Shanghai. [See
]Titled "School place for migrant pupil sparks 'locust' row in
Shanghai," it says that some some Shanghai residents accused a
6-year-old girl who got into a Shanghai school of wasting school
resources. It says in part [See
Shanghai residents, many of whom are also parents, were angered when
Diao Qianwen, a six-year-old girl living with migrant worker
grandparents in a wet market, was finally allowed to enroll in a
local primary school after her plight was aired on a state TV news
show. Some angry Shanghainese even copied their Hong Kong
counterparts and called Qianwen a little "locust."
The girl, originally from Shandong, has been raised by her
grandparents, who work as vegetable vendors in a local market. Her
mother had abandoned her at birth and her father is serving a
sentence in a Shandong jail, according to reports.
Diao had been turned down by local schools prior to the show. Besides
lacking official papers to prove they were the girl's "legal
guardians," schools say the elderly couple had not been contributing
to the city's social security fund - a prerequisite for all new
pupils. Her grandparents, after reaching a dead end, grew demoralized
as the new semester approached.
After the girl's plight was revealed on television, she was allowed
into the school, but many Shanghai residents complained because
legally registered students often have a hard time getting into city
schools, the newspaper says.
Schleicher is correct when he wrote that "international comparisons
are never easy and they are never perfect" and that the PISA exams
"are administered under strict and internationally comparable
conditions." Anyone who "really wants to find out," he wrote, "can
review the underlying data."
The question isn't whether the OECD tried to make valid international
comparisons. It is whether it succeeded - and whether international
comparisons can ever be "apples to apples" given the very different
education systems, cultures, school enrollment policies and
data-collection processes that exist across the globe. And the
question that Loveless raises about whether Shanghai cheats by its
exclusionary admissions policies still begs an answer.
Valerie Strauss covers education and runs The Answer Sheet blog.