Southwest Economy, First Quarter 2011
On the Record: PISA Results Shed New Light on U.S. Education Debate
Lori Taylor, associate professor at Texas A&M University’s Bush School of Government and Public Service, is an expert on the costs of education. She reviews test results from the Program for International Student Assessment, known as PISA, administered through the Organization for Economic Cooperation and Development (OECD). The exam is given every three years to 15-year-olds in nations across the globe—most recently in 2009 to students in 65 countries. Results were released late last year.
Q. What nations stand out with respect to student scores? Any surprises?
A. Many countries in Asia performed particularly well on the PISA exams. Children from China, Korea, Japan and Singapore did significantly better than the average for OECD countries in all three assessment areas—reading, math and science.
This was the first time that students from mainland China were included in PISA, and only students from Shanghai, Macao and Hong Kong were tested. Therefore, China’s results may not reflect the country as a whole. Nevertheless, the level of performance was impressive.
Of course, Asian countries were not the only ones to perform well on the PISA exams. Students from Finland, Canada, New Zealand, Australia, Belgium and the Netherlands also did better than the OECD average across the board. At the other end of the spectrum, seven OECD countries (Mexico, Chile, Turkey, Israel, Luxembourg, Spain and Italy) and many of the non-OECD countries performed significantly below the OECD average in all three subject areas.
Q. In what areas do U.S. students outperform? Underperform?
A. Students from the U.S. performed at or slightly above the OECD average in reading, at the OECD average in science and significantly below the OECD average in mathematics. Among the 34 OECD countries, the U.S. ranked 14th in reading, 17th in science and 23rd in mathematics.
The U.S. fell particularly short in the share of students achieving the highest level of math performance. Only 2 percent of U.S. students reached the top rung. By comparison, more than 4 percent of students in Canada, Finland, Korea and eight other OECD countries attained that level—as did 11 percent of the Hong Kong students and 27 percent of the Shanghai pupils.
Even average performance on PISA is in many ways disappointing. According to the OECD, “Level 2 on the PISA reading scale can be considered a baseline level of proficiency, at which students begin to demonstrate the reading competencies that will enable them to participate effectively and productively in life.” In 2009, 18 percent of U.S. 15-year-olds failed to reach that level of proficiency. While 18 percent below the baseline in reading is close to average for OECD countries, it cannot be acceptable if we hope to have a functioning, literate society.
The U.S. performance in reading and mathematics was essentially unchanged from earlier comparisons of international performance, but science was significantly improved. It was largely due to gains at the low end of the performance scale. The share of U.S. students achieving only the lowest levels of proficiency in science stood at 18.1 percent in 2009, an improvement from 24.4 percent in 2006. The share of U.S. students achieving only the lowest levels of proficiency in mathematics also fell slightly—to 23.4 percent in 2009 from 25.7 percent in 2003—but the difference is not statistically reliable. Even after the improvements, however, we remain no better than the OECD average in science.
Q. Can cross-country differences tell us anything about why U.S. students do poorly in math?
A. There seem to be three interesting patterns that help differentiate high-performing countries from low-performing ones. First, countries that spend their education resources on high-quality teachers tend to perform better than countries that spend their education resources on lower pupil–teacher ratios. Second, gaps in performance between economically advantaged and disadvantaged students tend to be smaller in countries that use standards-based external examinations. Finally, early childhood education seems to matter. School systems with a larger share of students who attended prekindergarten classes seem to have higher levels of student performance among 15-year-olds.
Q. Do rich countries always have higher-performing students? Is education funding the main explanatory variable?
A. There isn’t much of a relationship between education funding and student performance on PISA. U.S. mediocrity is clearly not attributable to a lack of resources devoted to education. Only Luxembourg spends more per pupil on schools than the U.S.
There is a little more evidence that how schools spend the money matters. For the same amount of money, a country can choose to have higher teacher salaries and larger class sizes or lower teacher salaries and smaller class sizes. The PISA results suggest that countries that spend their education resources on higher teacher salaries tend to perform better than countries that spend their education resources on lower pupil–teacher ratios. The U.S. is a low salary/low- class-size country.
Q. How does socioeconomic status play into students’ results? How about immigrant status?
A. The U.S.’s mediocre performance is also not attributable to a high fraction of immigrant students. Although students from an immigrant background generally performed less well on PISA exams, excluding immigrants raises the average U.S. reading score only slightly (to 506 from 500). On the other hand, the higher fraction of economically disadvantaged students helps explain the results. Across countries, differences in the students’ socioeconomic status can account for 14 percent of the difference in reading performance. However, poverty is not destiny. As the PISA scores illustrate, it is not uncommon for students from the poorest 25 percent of a country’s population to exceed expectations and place among the top 25 percent of students (after accounting for socioeconomic background). The PISA report calls such students “resilient.” In the U.S., 28 percent of the economically disadvantaged students are considered resilient; in Korea, Shanghai and Hong Kong, economically disadvantaged students are roughly twice as likely to be considered resilient.
“Socio-economic disadvantage translates more directly into poor educational performance in the United States than is the case in many other countries,” according to the PISA report. In general, PISA results suggest that economically disadvantaged students who attend schools where most of their peers are also economically disadvantaged tend to perform poorly, while economically disadvantaged students who attend schools where most of their peers are economically advantaged tend to perform better. It isn’t clear whether these performance differences arise because schools with few economically disadvantaged students tend to have lots of resources or because having advantaged peers affects student performance; it’s probably both. Quite likely, we may not be as aggressive as other countries at targeting additional resources to low-income schools and kids.
Q. What is the most important takeaway for U.S. policymakers from the PISA results? Are there international or regional differences from which we can learn?
A. The education system in the U.S. is broken. We spend more than nearly every other country on K–12 education, and our performance is mediocre at best. We have to make education reform a policy priority and rethink almost everything about how we go about accomplishing our educational goals.
We especially need to do a better job of integrating educational research into the design and implementation of education policy. For example, there are dozens of studies telling us that high-quality teachers are the cornerstone of high-quality schools. However, those studies also tell us that the teacher characteristics that largely determine a teacher’s pay—years of experience and advanced degrees—are not good indicators of teacher quality. There are lots of outstanding teachers with only a few years of experience—and too many ineffective teachers with more than 20 years of experience. It is a waste of scarce resources to base teacher pay on things that don’t translate into classroom effectiveness. We need to get away from the rigid salary schedules found in many U.S. school districts and do a better job of rewarding teaching excellence rather than teaching endurance.
We also need to do a better job of holding onto our best teachers. The Bureau of Labor Statistics estimates that over 60,000 teachers were laid off during 2009. One of the minor tragedies of the budget crises triggering many of those terminations is that seniority rules in states such as California, Ohio, New York, New Jersey and Pennsylvania prevented school districts from taking quality into consideration when deciding who to let go. Last hired were first fired, no matter how effective they were in the classroom. Texas is one of the few states where classroom performance has been the deciding factor in layoff decisions. More states need to follow Texas’ lead on this issue.
Policymakers should also become a little more flexible in their approach to class-size regulations. States and countries that focus on maintaining small class sizes find it difficult to staff all those classrooms with high-quality teachers. The PISA results indicate that countries that emphasize teacher quality over teacher quantity (by paying higher salaries and accepting larger class sizes) tend to outperform other countries.
The PISA results also suggest that standardized tests can have a positive impact on school systems. In particular, PISA researchers found that gaps in student performance between economically advantaged and disadvantaged students tend to be smaller in countries that use standards-based external examinations. Such examinations are an important part of the No Child Left Behind Act, which is up for reauthorization. The Obama administration has proposed a number of changes to the law that should strengthen it—but pointedly has not backed away from the basic premise that students should be tested regularly and that school districts should be required to publish the results. In fact, the Obama administration has strongly encouraged states to adopt more rigorous and consistent educational standards and to hold school districts accountable for meeting those standards. I think that approach has considerable potential.
About Southwest Economy
Southwest Economy is published quarterly by the Federal Reserve Bank of Dallas. The views expressed are those of the authors and should not be attributed to the Federal Reserve Bank of Dallas or the Federal Reserve System.
Articles may be reprinted on the condition that the source is credited and a copy is provided to the Research Department of the Federal Reserve Bank of Dallas.
Southwest Economy is available free of charge by writing the Public Affairs Department, Federal Reserve Bank of Dallas, P.O. Box 655906, Dallas, TX 75265-5906, or by telephoning (214) 922-5254.