Math Forum Discussions

RayM

Posts: 308
Registered: 12/3/04

Re: Time for Moderation
Posted: Jul 29, 2000 10:00 PM

Plain Text

----------
> From: me@talmanl1.mscd.edu
> Hidden in the "etc." is one that puts it all at question, right, Wayne?
> If you teach taking tests, kids will learn to take tests. And the
> standardized tests will suggest, falsely, that the kids know reading,
> and mathematics, and science, and history, and geography. You've
> mentioned the effect yourself, though I can't see why it doesn't bother
> you. When a new test is introduced, scores go down. As the district
> becomes more and more familiar with the test, the scores go up. But
> then when a new test is introduced, they go back down again. This is
> prima facie evidence that they learn to take the test--not the content
> the test allegedly covers.
>
> --Lou Talman

Well, one solution to that problem is to give every student every possible
test once every year. So you would administer the SAT itself and even the
GRE and MCAT to grades K through 12. You'll probably find less than 100 10
year olds that do well on the GRE and the MCAS. A careful researcher would
then go and interview them to characterize how well they really knew the
material. Perhaps 1 or 2 of the scores would be statistical fluctuation.
Some might be found related to Asperger's syndrome. But there really are
10 year olds that bright and well read. The research literature on IQ 180+
kids goes back more than 70 years. THAT pool of kids would max out on most
of the tests like ERB and SAT-9. There is also another pool of kids, the
profoundly retarded, that would get minimum scores on most of the tests.
In those two pools of kids, you could probably identify physiological
mechanisms and/or genetic markers that would account for most of the
variation. If you then proceeded to administer the same suite of tests
every year for several years you could probably statistically characterize
the "test-taking" effect Lou refers to with great precision. You would
have a very good picture of the degree of correlation between tests and
between tests and some genetic markers and between tests and various other
social and economic variables. You would also have succeeded in wasting
several years of education and hundreds of millions of dollars. There is
a simpler solution. Acknowledge that the test taking effect exists
alongside real measurement capability. Look at the Olympics: pole
vaulting, discus, 100 meter, free style swimming, archery, etc. are all
very carefully controlled standardized tests. They all have a component of
measuring real ability and also have the test-taking effect problem. It's
not likely that anyone naive to the Olympics or similar competitions would
do well regardless of innate ability. And it is certainly true that even
with a lot of practice in many competitions, some people just don't have
what it takes to win even a local marathon. And athletic competitions
also do a better job of measuring past experience than future performance:
Athletes wear out and the next generation replaces them. That's just the
way life is. Why do some people have trouble making similar judgements
about academic achievement?

Standardized tests measure exactly what they test.
for example:
______ growth in the economy was mainly responsible for the profit _______.
(A)Little.. upturn
(B)Sluggish.. slump
(C)Rapid.. decrease
(D)No.. growth
(E)Buoyant.. decline
If a student has no idea what the word responsible means, they're gonna
have a tough time with this one aren't they? So a reasonably large
vocabulary is needed. Next you need some vague idea of how profit is
calculated. There is a segment of the population that would have serious
difficulty with this problem for one or both of those reasons. I can also
see how a professional margin trader might get this one 'wrong'. But I
have a hard time believing that many readers of this list would have much
problem with this question. It does seem to test content to me. I can
even see it being used as the basis for a constructivist learning
situation. We could discuss a few thousand similar test items on this
listserve. The devil's in the details you know. It is easy to lampoon
tests in general or a few particular questions, but it is more difficult to
rule out the relevance of each and every question.

My son's private school did a small version of the first paragraph this
year. The ERB and one of the SAT's was administered to each third grader.
I only saw the scores for my son. I was struck by how good the correlation
was between the subtest scores. Had they done only the ERB, I might have
dismissed two of the lower scores as statistical fluctuations. Were the
test scores useful? Again, I'd say yes. I am now motivated to find
materials for practicing reading comprehension. I have no intention of
teaching to the test. Those tests are gone by now. But I do believe
reading comprehension is important and I do believe that it is a skill that
can be developed through practice as much as batting or pitching or soccer.