Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.
|
|
|
|
TIMSS 12th GRADE - U.S.: Part I
Posted:
May 23, 1998 11:02 AM
|
|
[From the TIMSS-Forum list -- Part II will follow shortly.]
http://nces.ed.gov/timss/responses.html
Statement by Pascal D. Forgione, Jr., Ph.D
U.S. Commissioner of Education Statistics
National Center for Education Statistics
RESPONSES TO RECENTLY-RAISED ISSUES
REGARDING 12TH-GRADE TIMSS
Since the release of the TIMSS report on mathematics and science
achievement in the final year of secondary school and the
corresponding third volume of NCES' Pursuing Excellence, several
questions have arisen about the validity of the comparisons and the
relevance of the results. Some observers have argued that the
methodology was biased against students in the United States in
several ways, including the fact that the average age of U.S.
students was lower than in other countries, that the populations
tested in other countries represented a more selective group of the
age cohort, and that not all our students who took the advanced
mathematics examination had been exposed to the material on the
assessment.
Several arguments have also been advanced about why the results of
the assessments do not matter. Some say we should not be so worried
about achievement at the end of high school because our students
"catch up" in college. Critics cite the fact that our economy is the
strongest in the world as evidence that people do not need the type
of mathematics and science knowledge and skills measured on the
TIMSS assessment. Howard Gardner writes that TIMSS measures only "a
kind of lowest common denominator of facts and skills." More
crucial, he argues, is the ability to "apply scientific and
mathematical concepts to the world around them." Others mention
abilities and qualities such as creativity, flexibility, and
innovation as more crucial for economic success.
Following are responses to the most common questions and criticisms
regarding the validity and relevance of the TIMSS assessment of
students at the end of secondary school.
1.Differences in age and grade levels
Q. The students in other countries, on average, were older than the
U.S. students. Is this because secondary schools in other countries
go beyond grade twelve? If so, how fair or useful is it to compare
U.S. students with those in other countries who are one to three
years older and have more years of instruction?
A. The purpose of this component of TIMSS was not to compare
students of the same age or years of schooling, but rather to
compare students at a similar point in the education system: the end
of secondary school. In many societies, including the United States,
students completing secondary school are deemed "ready" to enter
adult society, part of which is the workforce. Thus, a comparison of
students at this point sheds light on the level of knowledge
expected of and attained by the general population.
While there was a range of ages among students taking the
assessment, and NCES found that age was related to achievement, the
gap between the average ages of U.S. students and those in other
countries, and consequently, the impact it has on the results, has
been overstated. At the extreme, the students participating in TIMSS
in Iceland had an average age of 21.2 years, 3.1 years higher than
the average age of the U.S. students, 18.1. However, the
international average was far lower: 18.7 years, much closer to the
U.S. average. This gap was even smaller on the physics and advanced
mathematics assessments, where the average age of our students was
18.0 years for both assessments. The international averages on these
assessments were just a few months higher, 18.4 years in physics and
18.3 years in advanced mathematics.
Regarding the years of schooling, in other countries, both college
preparatory and vocational programs may go through grade thirteen or
fourteen, but most of those countries also have other secondary
programs that end earlier, often before grade 12. While eight
countries (out of 21 in the general knowledge assessments) included
some students in grades above twelfth grade, all of those countries
also included students in twelfth grade and five of them also
included students below grade 12. It is also important to note that
in some countries, the older average age of students reflects a
later school starting age. In Denmark, Slovenia, Norway, Sweden, and
parts of the Russian Federation and Switzerland, students start the
first grade at the age of seven, compared to our typical starting
age of six.
The differences in average ages and years of instruction also need
to be considered in light of the content of the general knowledge
assessments. If the content were based on high-level curriculum
topics, then younger students and students with fewer years of
schooling might be at a disadvantage, since it is reasonable to
think that they would be less likely to have been exposed to these
topics than older students or students with more years of schooling.
However, the TIMSS general knowledge assessments did not represent
advanced-level content. While the items on these assessments were
not based on any one curriculum, TIMSS analysts have found the
topics on the mathematics general knowledge assessment to be most
similar to topics covered by the seventh grade in most countries,
and the topics on the science general knowledge assessment similar
to topics covered by the ninth grade. When they looked at the items
in terms of U.S. curricula, they found these topics to be introduced
at later grade levels, by the ninth grade for mathematics and by the
eleventh grade for science. Thus, it appears that students in the
United States and in other countries were being tested on topics
they should have already covered, several years earlier in most
cases.
In the end, it is difficult to use the higher ages of students in
other countries to explain our relatively poor performance. While
many of the countries which outperformed the U.S. included students
whose average age was higher than ours, but we were also
outperformed by countries in which the average age of students was
lower than ours. For example, students in New Zealand and Australia
were, on average, younger than U.S. students but still scored
significantly higher than our students on both general knowledge
assessments. It is also interesting to note that students in the
Russian Federation performed similarly to U.S. students on both
general knowledge assessments, but were more than a year younger and
were all in eleventh grade.
2.Differences in enrollment rates
Q. In other counties, only the best students are still enrolled in
secondary school in the late teenage years. Isn't it unfair to
compare our general teenage population-the vast majority of whom are
still in school-with elite populations in other countries?
A. There are several ways to look at secondary school enrollment.
One way is to divide the number of young people enrolled in
secondary school by the total number in the population within the
corresponding age cohort. Recognizing that this can often overstate
enrollment if young people outside the age cohort are enrolled, OECD
data indicate that the majority of countries participating in this
component of TIMSS, including the U.S., have over 85 percent of the
age cohort enrolled in secondary schools. Thus, while variation in
enrollment rates does exist, the countries are roughly comparable,
and more so than in previous years.
Looking at enrollment another way, among seventeen-year-olds, the
U.S. actually has a smaller proportion enrolled in school than the
average for the other TIMSS countries for which this information is
available (75 percent vs. 82 percent). If higher school enrollment
rates were associated with lower average scores, the bias would be
in favor of U.S. students rather than against them. However, in
TIMSS, using either of the methods of measuring enrollment, students
in countries with higher enrollment rates than the U.S. tended to
score significantly higher than the U.S. on both the mathematics and
science general knowledge assessments. Furthermore, in TIMSS, the
pattern generally appears to be that higher secondary enrollment
rates are associated with higher levels of performance, rather than
the reverse.
3.The "unfair" definition of our advanced mathematics population
Q. The advanced mathematics test had calculus questions, but U.S.
students whose highest mathematics course was precalculus were
included in the test sample. Wouldn't it have been a fairer
comparison to look only at those students who had actually taken
calculus?
A. The advanced mathematics assessment was not primarily a calculus
test. Calculus items comprised only about 25 percent of the
questions. Interestingly, even with precalculus students included in
our sample of advanced mathematics students, our weakest content
area was not calculus, but geometry. This is even more surprising
when we take into consideration that advanced mathematics students
in the United States will typically have studied geometry for a full
year by the twelfth grade.
The purpose of the advanced mathematics assessment was to compare
advanced mathematics students across countries on content above the
level of "general knowledge." For this assessment, countries were
asked to create course- or program-based definitions of advanced
students such that the resulting population represented from between
10 to 20 percent of the age cohort. To meet this specification, the
United States needed to include students in precalculus classes, as
only seven percent of the age cohort were taking or had taken a
calculus class. (Would it be fair to compare seven percent in the
U.S. with over 16 percent in Canada, 20 percent in France, or 33
percent in Austria?) The resulting population in the United States
represented 14 percent of the age cohort, compared to the
international average of 19 percent. In this respect, the group of
U.S advanced mathematics students was somewhat more selective than
in some other countries.
Even with precalculus students included in our advanced mathematics
population, one cannot assume that U.S. students were at any more of
a disadvantage than students in other countries. In none of the
countries were students chosen on the basis of whether they had
taken calculus. An in-depth curriculum analysis might reveal that
students in other countries had more exposure to calculus (or other
topics) than U.S. students, but it might also show that U.S.
students had relatively more exposure to some topics than students
in other countries. If U.S. advanced mathematics students do face a
curriculum that includes fewer advanced topics than those faced by
their international peers, that would be an important finding.
End of Part 1 of 2
********************************************************
Jerry P. Becker
Dept. of Curriculum & Instruction
Southern Illinois University
Carbondale, IL 62901-4610 USA
Fax: (618)453-4244
Phone: (618)453-4241 (office)
E-mail: JBECKER@SIU.EDU
|
|
|
|
