Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
|
|
TIMSS 12th GRADE - U.S.: Part II
Posted:
May 23, 1998 11:13 AM
|
|
(Continued - Part 2 of 2)
4.Unbelievable numbers
Q. There are many instances of numbers that are not believable-ones
that are inconsistent with other data (within or outside of TIMSS)
or with common sense. Given that, why should we trust any of the
findings of the study?
A. There have been only a handful of numbers, out of the many
contained in the published reports about the end of secondary school
phase of TIMSS, which have been challenged by critics. In most
cases, the seeming problems with the data reflect a misunderstanding
or misrepresentation of the data.
For example, TIMSS data about the amount of time U.S. seniors work
has been alleged to be inconsistent with similar data from the
Bureau of Labor Statistics. However, the data being compared are not
comparable, either in terms of the populations being compared
(seniors in the case of TIMSS and the entire population ages 16 to
19 from BLS) or the questions used to gather the data (hours worked
on a "normal school day" in TIMSS and hours worked per week in the
other). Data gathered from 1992 high school seniors in February
through April of 1992 by the National Education Longitudinal Survey
of 1988 (NELS:88) reveal patterns of working near the end of the
senior year in high school that are not inconsistent with those in
TIMSS.
Another allegation is that only about half of the eligible
population in each country participated in TIMSS, on average, and
was ten percent or less in a few countries. However, this was not
the case. The statistics being cited do not represent the percentage
of the eligible population participating and the very low
percentages are for countries that did not participate in the
general knowledge assessments. Of the 21 countries participating in
the general knowledge assessments, 14 excluded less than 10 percent
of the internationally defined eligible population from the TIMSS
sample. Five of the seven countries that excluded at least 10
percent of the eligible students performed the same or worse than
the U.S. on one or both of the general knowledge assessments.
It is also important to remember that TIMSS is a sample survey. All
the estimates have a margin of error (called the standard error).
Numbers that seem inconsistent or implausible may not be so when the
margin of error (sometimes a rather large margin of error) is taken
into account. This can particularly be a factor when the total
sample size is relatively small, or when the entire sample is
subdivided into smaller subcategories. For example, the standard
error for the estimate of the average score for all U.S. students on
the science general knowledge assessment was 3.3 points. The
standard errors for the average score ranged from 3.5 to 13.5 points
for subgroups of U.S. students classified on the basis of how many
science courses they were currently taking at the time of TIMSS.
5.Cultural differences
Q. Because of all the differences between education systems and
between cultures, we are really comparing apples with oranges.
Because countries have different educational goals and philosophies,
not to mention curricula and school systems, how valid are these
types of comparisons?
A. Far from rendering international comparisons meaningless,
cultural differences are often what international comparisons such
as TIMSS seek to shed light upon. Some factors may be nearly
impossible to measure across cultures, such as pressure to do well
in school, but TIMSS offers an excellent opportunity to study
several important differences between cultures and education systems
(such as part-time employment, TV watching, hours spent doing
homework, and factors related to school environment) and their
relationship to achievement. What is remarkable is that-at least in
initial analyses-so few of the factors examined one-by-one could
account for our relatively poor performance. While the U.S. compared
unfavorably to the international average on such factors as taking
mathematics or science in the senior year, amount of homework, theft
of property and personal threats at school (indications of a poor
school environment), and hours working at a part-time job, none of
these factors had a statistically significant relationship to our
relative performance. Perhaps the explanation for our poor
performance lies in cultural factors which may be difficult to
change, such as pressure to do well in school; but it is also
possible that they are due to school-related factors over which we
have more control, such as curriculum and instruction.
6.The relevance of the results
Q. The end of high school is too soon to tell whether our young
people acquire the necessary skills and knowledge. Don't most of
them catch up in college, while their counterparts in other
countries may not attend college or may take higher education less
seriously than our students?
A. Many people have made the observation that our students do not
become serious until they reach college. However, we cannot assume
that all our students will attend college and take mathematics and
science courses, and that students in other countries are standing
still, waiting to be caught. While the majority of our high school
graduates do enroll in either two-or four-year postsecondary
institutions, a substantial proportion does not. Thirty-eight
percent of 1994 high school graduates were not enrolled in a
postsecondary institution in the fall after graduation. For those
who do enroll, catching up often means taking remedial classes: 24
percent of 1995 freshmen took remedial mathematics.
If students complete college-and many do not, they can graduate
having met only the minimum course requirements for mathematics and
science, which in many cases may be no more than a single course in
each. Data from the NELS:88 study show that of 1992 high school
graduates who attended a postsecondary institution by spring of
1994, 32 percent had not taken any mathematics classes above the
remedial level. Furthermore, the students who were most in need of
"catching up," those with low levels of mathematics achievement at
the end of their senior year in high school, were less likely than
others to take mathematics above the remedial level in college.
While some of our students may catch up in college, by taking either
regular or remedial mathematics classes, many may never close the
gap between them and their international peers. Even if it is true,
as some have argued, that college students in other countries do not
work as hard as our students, we know that students in other
countries start out at a higher level than ours, continue to take
mathematics and science in college, and are more likely to
specialize in mathematics- and science-related fields.
Q. Why should we worry about the results of a paper-and-pencil test?
We know that other skills, such as the ability to apply knowledge,
critical thinking, and creativity, are more important for success in
the real world?
A. TIMSS is not a superficial test of knowledge and skills. The
purpose of the general knowledge assessments was to see how well
students could apply what they had learned in mathematics and
science to situations likely to occur in their everyday lives. Items
on all four of the end of secondary school assessments included not
only multiple choice questions, but also short and extended free-
response items as well. Typical items on the mathematics assessment
required students to analyze a stated problem, decide which
mathematical tools to use to solve the problem, and then solve it.
Science items presented students with an observation, situation, or
hypothesis and asked them to use their knowledge of science to
either explain the cause, predict the results, or describe how one
might go about testing the claim. Thus, the items on the TIMSS
general knowledge assessments required a strong base of knowledge,
but also a variety of other intellectual skills, including
reasoning, application of knowledge, and designing multi-step
solutions.
Q. U.S. students may not excel in terms of academic achievement in
mathematics and science, but there are more important qualities
required to succeed in life, such as creativity, flexibility,
innovation, and entrepreneurial skill, none of which were addressed
by the TIMSS examination. Why should we be concerned about the
results of TIMSS when the strength of our economy shows that we do
not have to perform well on these tests to prosper?
A. The strength of an economy or the quality of its workforce will
never rest solely on academic achievement. As Robert J. Samuelson
points out in "Stupid Students, Smart Economy?" (Washington Post,
3/12/98), there are many factors that contribute to a productive
workforce, the education system being only one of them. Other
factors include the availability of jobs, the work environment, and
corporate practices. Thus, a successful economy does not mean we do
not have serious problems in mathematics and science education, or
that these problems do not matter, only that currently, our economy
is succeeding in spite of these problems. For our long-term economic
well-being, nearly all experts in the field of national economic
productivity believe that the knowledge and skill levels of
America's students in science, mathematics, and technology will
become increasingly important.
Q. The majority of us use very little science and mathematics in our
jobs. Why do individuals need a strong background in mathematics and
science beyond basic computational skills?
A. There are several ways in which individuals benefit from having
high levels of mathematics and science skills, both in the labor
market and in their daily lives. Higher academic achievement is
associated with higher wages and lower unemployment. Data on those
currently employed show that those with high levels of mathematics
and science achievement earn significantly more and are less likely
to face periods of unemployment than those with lower levels of
achievement, even among people with the similar levels of
educational attainment (NCES, Education and the Economy, NCES97-279,
pp. 33-41).
As crucial as academic achievement may be to success in the labor
market, the importance of science and mathematics knowledge reaches
beyond economic productivity. The TIMSS general knowledge
assessments were designed to test students' ability to apply their
knowledge to situations they might encounter in everyday life, which
is becoming more complicated. As adults, today's students will need
to make sense of a rapidly changing world. They will need to make
difficult decisions for themselves and their families regarding
finances and health care. Collectively, as members of a community,
they will need to understand and make decisions regarding how to
treat the world in which they live.
*************************************************************
Jerry P. Becker
Dept. of Curriculum & Instruction
Southern Illinois University
Carbondale, IL 62901-4610 USA
Fax: (618)453-4244
Phone: (618)453-4241 (office)
E-mail: JBECKER@SIU.EDU
|
|
|
|
