The Math Forum

Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Math Forum » Discussions » Policy and News » mathed-news

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Test Score Errors: Enormous Mistake!
Replies: 0  

Advanced Search

Back to Topic List Back to Topic List  
Jerry P. Becker

Posts: 16,576
Registered: 12/3/04
Test Score Errors: Enormous Mistake!
Posted: Sep 16, 1999 9:20 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

From New York Times; New York; September 15, 1999; p. 1. [East Coast Edition]
[New York City Public Schools]

Chancellor Cites Test Score Errors

By Anemona Hartocollis

Chancellor Rudy Crew revealed yesterday that significant errors were made
in scoring the high-stakes tests given last spring to determine who would
be assigned to summer school and that some children had been compelled to
attend by mistake.

Dr. Crew announced the problem in a news release, and declined to answer
questions about it. But an aide said the number of children mistakenly
assigned to summer school could range from 2,000 to

Dr. Crew attributed the confusion to a miscalculation by the test
publisher, CTB/McGraw-Hill, and the company confirmed yesterday that
similar scoring errors had occurred on tests given in other parts of the

But yesterday's revelation was just the latest in a series of
embarrassments for the Chancellor's plan to end automatic promotion of
failing students. In mid-August, Dr. Crew proudly announced that he
expected most of the 35,000 summer school students to go on to the next

But on Sept. 1, he had to revise that figure, saying that 60 percent -- or
21,000 -- would be held back, largely because 14,000 students either did
not show up in summer school or did not take the tests.

Education and testing experts said yesterday that the disarray in New York
City was a lesson in how high-stakes testing across the country might
encourage educators to make snap judgments about students that turn out to
be unfair.

"What's wrong with it is who ends up getting hurt in this process: the
kids," said Seymour C. Fliegel, a senior fellow at the Center for
Educational Innovation, a New York City research institute.

As a result of the miscalculation, some parents were forced to give up
summer vacations -- with immigrants sometimes canceling expensive airline
tickets for homeland visits -- so that their children could go to classes.

Jill Chaifetz, executive director of Advocates for Children, a nonprofit
advocacy group, said that many parents had been surprised to hear that
their children were failing, and now, she said, it might turn out that
their surprise was justified.

She said the revelation of the flawed scoring would only compound the
distress of parents over what she criticized as the hastiness of the summer
school program.

It is now possible, she said, that some children passed the tests last
spring, were sent to summer school by mistake, and then failed the tests at
the end of summer, forcing them to repeat the grade.

Advocates for Children has challenged the Board of Education's summer
school program in court, arguing that it violates the board's own policy of
looking at factors besides test scores -- like homework and the judgment of
teachers -- in determining promotion. Oral arguments in the case took place

New York City's summer school program has been strongly pushed by Mayor
Rudolph W. Giuliani, who has argued that one way to improve schools is to
get tough with failing students. A spokeswoman for Mr. Giuliani declined to
comment on the scoring problems yesterday.

The mistake came in a percentile score that showed how the students
compared with a national sample of children across the country. The
publisher gave the test questions to a sample of students in different
cities. But in some cases, the test company used answers from the wrong
questions in computing the norm, or national average.

The company said that the scores were wrong for 4,500 students, or 3
percent of the 150,000 students in the national sample.

Steven H. Weiss, a spokesman for CTB/McGraw-Hill, acknowledged yesterday
that the company was responsible for the scoring errors. He refused to say
what other school districts had received flawed test results, saying he did
not have their permission to release that information. But he said none of
the other districts had used the test scores to make high-stakes decisions.

Karen Crowe, a spokeswoman for Dr. Crew, said the test company had told New
York City that nine other jurisdictions were affected by the problem.

Mr. Weiss said that New York City's director of testing, Robert Tobias, had
been the first person to suspect that there was an error in the reading and
math tests, and had urged CTB/McGraw-Hill to investigate. Ms. Crowe said
that Mr. Tobias's suspicions were aroused by the precipitous drop in test
scores from the year before.

Although the tests were new and scores typically drop on new tests, Mr.
Tobias believed, she said, that the drop was abnormally sharp.

Mr. Tobias first alerted the testing company to his concerns last spring,
New York City officials said.

None of the children sent to summer school by mistake have been notified
yet, because the board is still computing who they are and how many there
are. Yesterday's estimate of 2,000 to 3,000 children is very preliminary,
officials stressed.

Ms. Crowe said that it had taken until Monday for the testing company to
confirm the problem, and that New York City officials were still trying to
figure out the extent of the scoring error. Mr. Weiss said the delay
occurred because the company had to review tens of thousands of test forms.

Saying that he was "distressed" by the problems, Dr. Crew called for an
independent audit of the quality control procedures used by
CTB/McGraw-Hill. Ms. Crowe said the school system was considering suing the
test company.

"We pay an enormous amount of money to a company to provide accurate test
results, which we use to make high-stakes decisions, and they should be
accurate," Ms. Crowe said. In 1998, the board spent more than $2 million in
contracts with the testing company, and in 1999, it is authorized to spend
$3.3 million, she said.

Mr. Weiss said that the underlying integrity of its tests had not been
questioned, and that the company believed that tests were one of the most
reliable instruments available to measure student performance.

Monty Neill, executive director of the National Center for Fair and Open
Testing, an advocacy group in Cambridge, Mass., said that similar scoring
problems had occurred at least once before, on a test designed by the same
company and used in Indiana. Mr. Neill said that in 1996-97, the
superintendent of schools in Fort Wayne noticed a large drop in percentile
scores despite an overall rise in the number of correctly answered
questions on the test.

Mr. Weiss, the CTB/McGraw-Hill spokesman, said yesterday that he could not
confirm or deny that case. But he suggested that Mr. Neill, as a frequent
critic of testing practices, was unreliable.

"Given his well-known position against testing, it's not surprising he
would choose to capitalize on this situation to advance his own cause," Mr.
Weiss said.

But some testing experts said that New York City school officials had
violated a widely accepted tenet in education by basing their decisions to
promote children on the results of just one test.

"Whenever we focus on one factor alone, we risk making major mistakes like
this one," said Jay P. Heubert, a professor at Teachers College at Columbia
University who has studied high-stakes tests.

The reading and math tests, given last spring to 300,000 New York City
students in grades 3, 5, 6 and 7, showed that 44.6 percent of students were
reading at grade level. The board said that after equating, a process to
make the tests comparable to the previous year's tests, reading scores
across the grades were stable, while math scores dropped 10 percentage

Scores in the third grade, which Dr. Crew has made a focus of attention,
dropped more than any in other grade in both reading and math.

It was the drop in third-grade scores, despite the resources devoted to
that grade, that aroused Mr. Tobias's suspicions, Ms. Crowe said.

Jerry P. Becker
Dept. of Curriculum & Instruction
Southern Illinois University
Carbondale, IL 62901-4610 USA
Fax: (618) 453-4244
Phone: (618) 453-4241 (office)
(618) 457-8903 (home)

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2018. All Rights Reserved.