The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Policy and News » mathed-news

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: New York City Test Errors - More Info
Replies: 0  

Advanced Search

Back to Topic List Back to Topic List  
Jerry P. Becker

Posts: 16,576
Registered: 12/3/04
New York City Test Errors - More Info
Posted: Sep 17, 1999 2:22 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

From New York Times, September 17, 1999

Miscalculation on Scores Shows a Weakness of Tests

By Anemona Hartocollis

[T] he revelation that a national testing company miscalculated the reading
and math scores of thousands of New York City students opens a window into
the fragility of standardized testing, even as school districts across the
country are increasingly using test scores to make high-stakes decisions.

Officials for both the New York City school system and the testing company,
CTB/McGraw-Hill, one of the three biggest in the country, have admitted
that if not for the persistence of one whistle-blower -- the city's testing
director -- the errors might never have been discovered.

Yet those errors led to one of the bleakest chapters in the tenure of
Chancellor Rudy Crew, who had made literacy a priority and then in June was
forced to announce the original low scores. The Chancellor used the scores
to make a host of decisions with huge impact on the lives of children and
educators: from taking over failing schools to removing superintendents to
sending more than 8,600 children to summer school who should not have been
forced to be there.

Now thousands of schoolchildren are trying to salve the ache of being
unfairly branded as failures, and the mistake has tarnished the reputation
of standardized testing carefully cultivated by the testing companies

"I don't think these kinds of errors are rampant," said Michael J. Feuer,
director of the board on testing and assessment of the National Academy of
Sciences. "But obviously, when you're dealing with attempts to analyze
large quantities of data, errors are always possible."

The error on the CTB/McGraw-Hill test, which the company said also affected
eight other systems, was not the first for the testing industry. In the
mid-1990's, California introduced a statewide test and was criticized after
improperly releasing scores to hundreds of schools that were based on
inadequate samplings of students.

The problems, critics say, graphically illustrate why many educators and
testing companies themselves advise against using a single test given on a
single day as the sole criterion for making decisions about promotion or

"It's clear that CTB/McGraw-Hill messed up," said Monty Neill, executive
director of the National Center for Fair and Open Testing in Cambridge,
Mass. "But CTB did not make the Mayor, the Chancellor or the Board of
Education decide to use test scores only, in violation of the standards of
the testing industry, to make decisions about kids."

In the New York City episode, the testing company and Dr. Crew find
themselves on the defensive. Yet Dr. Crew, who just three months ago said
he would make "no excuses" for some of the worst test scores in years, now
boasts just the opposite situation: one of the brightest pictures in years.
The adjusted statistics show that 48.5 percent of New York City's third,
fifth, sixth and seventh graders read at or above grade level, not 44.6
percent as announced in June. After a calculation to make the 1999 test
comparable to the 1998 test, Dr. Crew announced on Wednesday that reading
scores had risen 4.9 percent from 1998, an unusually high one-year gain for
any school system.

Yet instead of basking in the glow of positive headlines, the Chancellor
has found that achievement virtually ignored in the atmosphere of
skepticism created by the errors. While almost no one -- except the school
system's own director of assessment, Robert Tobias -- raised doubts about
the dramatic 10-point decline in math scores announced in June, the new
reading scores have drawn a different reaction. (The board has not yet
recalculated math scores.)

"The fix is too good to believe," said Stephen Ivens, vice president of
Touchstone Applied Science Associates, in Brewster, Putnam County, which
supplied its reading test, Degrees of Reading Power, to New York City
schools for 12 years before it was replaced by CTB/McGraw Hill.

"Could you as a publisher make the data come out more or less favorable?"
he asked. "The answer is, yes you can, particularly once I know what the
results are."

Dr. Crew said yesterday that he saw no reason to abandon the use of
standardized testing in making high-stakes decisions like whether to
promote students. "That is a really, really cynical view," he said of
comments like Dr. Ivens'. " 'I paid them off'; 'I sold them my first-born.'
There's a level of it being absolutely insidious -- when the scores look
like they might be anywhere near decent, to just make this a scandal. These
teachers and these children, they earned something."

Dr. Crew said he used the reading and math tests to screen for summer
school, despite Tobias's suspicions, because the testing company
strenuously denied until last week that there was any problem.

David Taggart, the president of CTB/McGraw-Hill, said yesterday that such
scoring problems occur rarely, and are normally detected, as in New York
City's case, by school district officials who notice something amiss when
they put the standardized test scores in the context of everything else
they know about their students.

CTB/McGraw-Hill has posted a prominent warning on its Web site, advising
school districts that "no single test can ascertain whether all educational
goals are being met."

Dr. Crew has removed five superintendents and put four more on probation,
again citing low scores.

Now, eight of the nine districts where he removed superintendents or put
them on probation show gains in reading instead of declines.

Robert Riccobono, who was ousted by Dr. Crew as superintendent of district
19 in East New York, Brooklyn, said he believed his fate and that of the
other superintendents would have been different if the erroneous scores had
never been announced. Riccobono is the only superintendent appealing his
ouster. The new data, he said, could only help his case.

"Wow," he said, as he learned of the revised scores. "It has to help,
because it's clear to me that the only reason we were under scrutiny to
begin with had to do with the lower performance on the reading test."

Jerry P. Becker
Dept. of Curriculum & Instruction
Southern Illinois University
Carbondale, IL 62901-4610 USA
Fax: (618) 453-4244
Phone: (618) 453-4241 (office)
(618) 457-8903 (home)

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.