>The problem purposefully omitted a piece of critical >information - the weight of the jar - and the instructions >asked only that students create a plan to solve the >problem, not calculate the actual answer.
>Most students got that question wrong, she said.
In 4th grade? Well, Duuhhh. A stupid assessment of ÃÂhigher order thinkingÃÂ stupidly assessed by the people who got us here shows us how far the nation has yet to come. One of my favorites from the 4th grade NAEP is (with calculators allowed throughout, of course) was something like: Add 234 and 573. If your answer is 134082, what key did you probably hit instead of the ÃÂ+ÃÂ key. Anything to avoid algorithmic competence and straightforward word problem competence at well publicized grade-specific content levels.
Although CaliforniaÃÂs Prop 38 mandates the stateÃÂs student assessment for voucher redeeming schools - and I believe thatÃÂs an important thing to do so - thereÃÂs nothing like this to see why some folk think thatÃÂs a mistake, the industry canÃÂt be trusted to even write a test. Mandate a nationally normed standardized test, perhaps, or better, CaliforniaÃÂs (identifiable!) standards-based exam scrutinized by mathematically competent but not pedagogically impaired people, but not these mis-conceived notions that have been floating around ed schools for decades. Dismiss the design team, the writing team, and all such ÃÂprofessionalÃÂ assessment outfits. Let some real people (but knowledgeable people) do it or donÃÂt do it at all.
Posted at 12:00 a.m. Pacific; Thursday, August 24, 2000
4th_grade math test too hard? Yes and no
by Jolayne Houtz Seattle Times staff reporter
Teachers have complained for three years that the state's fourth_grade math test is too tough for their students. Now, a new study confirms their suspicions, at least in part.
An analysis of the math test released yesterday concludes that while the test is basically well_constructed and age_appropriate, there are problems with some test questions, the length of the test and the way it is administered.
Terry Bergeson, the state's schools superintendent, characterized the problem as requiring a "tweaks_plus" response, "but certainly not a basic foundational change in the test." Last year state lawmakers mandated a review of the math section of the fourth_grade Washington Assessment of Student Learning after numerous complaints from teachers and others that the test was too much for 9_year_olds.
Independent experts hired by Bergeson's office to conduct the analysis found four areas of concern.
Ten of the 120 test questions experts studied that appeared on tests from 1998 through last spring were either developmentally beyond the abilities of fourth_graders or not aligned to the state's academic standards.
That means some students might not have been taught skills they needed to answer those questions.
Seven of those 10 questions required students to "create a plan" to answer the question rather than "follow a plan" as called for in the academic standards teachers use.
Bergeson cited one math problem from this year's test. It asked students to tell how they would solve a story problem calculating how much weight a shelf, which held a jar of pennies, could support.
The problem purposefully omitted a piece of critical information _ the weight of the jar _ and the instructions asked only that students create a plan to solve the problem, not calculate the actual answer.
Most students got that question wrong, she said.
Altogether, about 10 percent of test questions in each of the past two years were judged either too tough or not aligned to the standards. Those questions will be rewritten or eliminated from the bank of test questions.
But Bergeson said students still had plenty of opportunity to pass the test by doing well on other items.
Other areas of concern:
The test is too long, with children needing two 80_minute blocks to complete the math test. Bergeson said she wants to reconfigure the test so it can be given over three or four sessions.
The layout of some test questions can be confusing, masking students' mathematical abilities. Bergeson said state officials will work with the testing contractor to rewrite those questions or tinker with the test format to address those concerns.
How the test is administered is inconsistent. Some teachers said they were unclear about what materials could be left on classroom walls and what guidance they could provide during the test. Some teachers provide breaks and snacks, while others don't.
Bergeson said she will work to clarify the guidelines for administering the test.
The analysis was conducted by the Northwest Regional Educational Laboratory in Portland. Separately, four education professors conducted their own reviews of aspects of the test: Verna Adams at Washington State University; Cindy Walker at the University of Washington; John Woodward at the University of Puget Sound; and Stanley Pogrow at the University of Arizona.
In addition, state education officials visited 10 schools statewide to watch students take the test and interview teachers and principals about it.
And they surveyed 38 schools that showed the greatest jump in math scores and found two key reasons for the rise: more and better math instruction.
Scores on the math test have been rising statewide since it first was given in 1997. Last year 37 percent of students met the math standard, up from 21 percent in 1997.
Bergeson said that's proof the test is sound.
"A hard_working, well_taught fourth_grade child can meet the standard on this assessment," she said. "This shows us places where we need to make corrections, and it shows us it was a good test to begin with."
The study found the process used to create the standards and the test was sound as well.
Yet Bergeson said the study also validates those teachers who have complained the test is too long, too intense, too confusing for their students.
The findings don't require recalculations of this year's test scores, due in a couple of weeks, or those from previous years, she said. Each year's test only included three or four questions the experts found problematic, out of a total of 40 questions _ not enough to change an individual student's score, Bergeson said.
The state will conduct a similar analysis of the math tests taken by seventh_ and 10th_graders, but those tests have not prompted the same level of criticism as the fourth_grade test.
Critics have charged that the test overall is unreasonable and unfair.
A WSU education professor, Donald Orlich, last year conducted his own test review and concluded that as many as 70 percent of questions made public on an example test were too hard for fourth_graders. He predicted math scores may not budge much beyond their current level.
He could not be reached for comment yesterday.
Arthur Hu of Kirkland is a software engineer and longtime test critic running against Bergeson for state superintendent of public instruction.
His campaign has focused primarily on "killing the WASL" test because Hu contends it is unfair and Orwellian.
He called the study "a piece of garbage" and said his own analysis found more than half the test problems are not aligned with academic standards. "The WASL is not even compliant with its own rules," he said.
But some educators who have been critical of the test said they feel vindicated by the study.
"I'm trying to be optimistic," said Tricia Lewicki, a fourth_ and fifth_grade teacher at Beacon Hill Elementary in Seattle.
She said she hopes state officials don't downplay the changes as "minor tweaks" but use the opportunity to do a major overhaul of the test, including cutting back on the number of math questions.
"If that happens, then that's something I can live with," she said.
Margie Kates, who retired this year as Beacon Hill's principal, said she felt encouraged that state officials took teachers' concerns to heart. "We knew all along . . . this was meant to be a work in progress," she said. "It sounds as if they have thoughtfully addressed the concerns."
Sen. Rosemary McAuliffe, a Bothell Democrat who chairs the Senate Education Committee, said the study also validates the state's go_slow approach with education reform, particularly in attaching consequences to test scores.
"The lesson we've learned is . . . it's too early for high stakes," she said.