U.S. Math, Science Ach't Exceeds World Average

The math and science achievement of U.S. students continues to surpass the global average for nations taking part in a prominent assessment, results issued Tuesday show, but several East Asian countries and jurisdictions far outpace the United States, especially in mathematics.

The most striking contrast comes in the 8th grade, where nearly half of all students tested in South Korea, Singapore, and Chinese Taipei (Taiwan) reached the "advanced" level in math, compared with only 7 percent of American test-takers, according to the Trends in International Mathematics and Science Study, or TIMSS, for 2011. [See http://timss.bc.edu/ ]

"One obvious stark takeaway of some concern in a global environment is the huge gap that the Asian countries have achieved in mathematics," said Ina V.S. Mullis, the co-executive director of the TIMSS & PIRLS International Study Center at Boston College. "This is a gap that has its roots in 1995 [when TIMSS was first administered], and the gap has not narrowed over the years. And in some cases, such as [South] Korea, it's even widening."

The Russian Federation, Quebec, Hong Kong, and Japan also outscored the United States by statistically significant margins in grade 8 math.

In fact, Russia surpassed the United States in that category for the first time, thanks to an improvement in its score compared with four years earlier, while the U.S. average stayed about the same as in 2007.

In one notable twist that's likely to spark debate, Finland, which drew international attention and acclaim two years ago based on its strong results on a different global assessment, did not produce the same standout results in math on TIMSS. Its 4th and 8th grade math scores were about the same as those of the United States, and several U.S. states participating in the exam-including Massachusetts and Minnesota-posted higher scores.

SOURCE: Trends in International Mathematics and Science Study, 2011
---------------------------------------------
In all, 63 countries and 14 regional jurisdictions (including some individual U.S. states) participated in TIMSS 2011, which takes place every four years. Also today, new data from a high-profile global assessment of literacy was released. This report, Progress in International Reading Literacy Study, or PIRLS, is focused on 4th graders.

With results available for 4th and 8th grade math and science, U.S. students have improved by a statistically significant margin in just one category, 4th grade math, since 2007. The average score in the category rose by 12 points, to 541, on the TIMSS scale. (Scores are reported on a scale from 0 to 1,000. A score of 500 was the average for participating nations and education jurisdictions, excluding a small number of "benchmarking systems" whose scores were not factored into the average, such as the individual U.S. states that took part.)

The United States trailed seven nations and jurisdictions in 4th grade math: Singapore, South Korea, Hong Kong, Taiwan, Japan, Northern Ireland, and Flemish Belgium. Among the more than 40 entities that the United States outpaced in the subject were Germany, Ireland, Hungary, and Australia.

TIMSS vs. PISA

The TIMSS data contrast to some extent with the high-profile results issued two years ago from PISA, the Program for International Student Assessment. On PISA, which tests 15-year-olds, U.S. students trailed the global average of participating students in math, though the nation for the first time reached the international average in science.

Experts note that several factors may help explain differences in the U.S. performance on PISA, including the pool of countries taking part. Although the participants overlap significantly, they are not identical. The international averages for PISA are based on a set of industrialized nations from the Organization for Economic Cooperation and Development (though some other countries participate); the TIMSS average includes a number of less developed nations on the lower end of the achievement scale, such as Morocco, Yemen, and Indonesia, that help push the average downward.

"The OECD countries are for the most part our chief economic partners and our competitors," Jack Buckley, the commissioner of the National Center for Education Statistics, said during a conference call Monday with reporters. "They tend to be wealthier nations." The TIMSS average includes "fewer of our wealthiest competitors, ... and is a more diverse group of countries," he said.

Another difference is that PISA is a test explicitly for 15-year-olds, while TIMSS tests 4th and 8th graders, Mr. Buckley noted.

Finally, the assessments themselves are very different.

"TIMSS and PIRLS are curriculum-based assessments," said Michael O. Martin, the co-executive director of the International Study Center at Boston College. "They try to assess what is being taught in schools. ... PISA has a more skills-based approach, [focused on] transitions to the work world."

The 'Finnish Miracle'?

Nine U.S. states opted to provide large enough samples of students so that they could be directly compared with participating nations on TIMSS, though only Florida and North Carolina did so in grade 4. The biggest standout was Massachusetts, which was especially strong in science, with an average score of 567. The only nation to score higher was Singapore, while South Korea and Taiwan were not measurably different. A full one-quarter of Massachusetts students reached the advanced level. (In Singapore, the figure was 33 percent.)

In addition, Minnesota, with an 8th grade science score of 553, was outperformed only by Singapore and Taiwan.

Mr. Buckley of the education statistics center, a branch of the U.S. Department of Education, praised the strong performance of such states. He suggested that U.S. policymakers looking around the world for lessons on creating a strong education system may want to take a look closer to home first.

"It's not necessary to travel halfway around the world to see this," he said.

At the same time, he cautioned that even Massachusetts falls short of a few global peers when looking particularly at getting more math students to the advanced level.

"There clearly is some room for improvement, even among our higher-performing systems," Mr. Buckley said.

Tom Loveless, a senior fellow at the Brookings Institution, a Washington think tank, said in an email that the new results call for some rethinking of what he calls the "Finnish miracle story."

"If Finland were a state taking the 8th grade NAEP, it would probably score in the middle of the pack," he said, in a reference to the National Assessment of Educational Progress. He said that four of the U.S. states that participated in the 8th grade TIMSS-Massachusetts, Minnesota, North Carolina, and Indiana-posted scores that were higher than Finland's by statistically significant margins in math, while three more had results that were about the same.

"Finland's exaggerated reputation is based on its performance on PISA, an assessment that matches up well with its way of teaching math (applying math to solve 'real world' problems)," he wrote. "In contrast, TIMSS tries to assess how well students have learned the curriculum taught in schools."

At the same time, Finland made a stronger showing than the United States in 4th and 8th grade science on TIMSS. In the 8th grade, for instance, Finland scored 552, compared with 525 for the United States. Measured another way, 53 percent of Finnish 8th graders reached either the "high" or the "advanced" level, the top two categories, compared with 40 percent of their peers in the United States.

Even so, Finland's performance fell short of the results for the top-performing East Asian countries. It also was lower than Massachusetts' score of 567.

For his part, Mr. Buckley said, "I've always been a little puzzled" by the high level of attention trained on Finland.

"Finland captured the world's attention for a variety of reasons," he said, "but as these results show, there are other places to look for case studies."

Measuring Student Engagement

With the 2011 report, TIMSS includes a number of new indicators to better help put student achievement in context. They include children's learning experiences prior to school attendance, the extent of students' engagement in math and science lessons, and their experiences with bullying.

"One thing we've worked on is [getting] better indicators of what goes on in classrooms," Mr. Martin of the International Study Center said. "We've sharpened our focus on student engagement. [One] measure is based on asking students how engaged they feel in their classroom. That makes a very nice scale that relates to achievement."

Another new scale, he said, is based on asking teachers what they do to engage students.

In a finding that may come as little surprise, students across nations seem to lose some enthusiasm for math as they get older. Nearly half (48 percent) of 4th graders said they "like learning mathematics," but that slipped to one-quarter (26 percent) by the time they hit 8th grade. And at both levels, that attitude has a correlation with test scores. That is, the less students like math, the lower their achievement, on average.

Another troubling indicator is that, across the globe, students report a drop in engagement with math lessons as they move from 4th to 8th grade. And once again, that level of engagement is correlated with a slip in average scores. The Engaged in Mathematics Lessons scale was based on responses to five questions, including "I know what my teacher expects me to do," and "I am interested in what my teacher says."

Meanwhile, many 4th graders around the world (69 percent) had math teachers who reported making efforts to use instructional practices intended to interest students and reinforce learning, such as posing questions to elicit reasons and explanations, and bringing interesting items to class. At the 8th grade, however, only 39 percent of students internationally reported that their teachers frequently related lessons to their daily lives, and just 18 percent said they had teachers who routinely brought interesting materials to class.