These seemingly contradictory findings suggest how and why global rankings on international assessments can be used to justify changes in policy and public spending even when – as previous studies have shown -- the public is confused, skeptical or uninformed about the results.

The Teachers College study‌ led by Oren Pizmony-Levy, Assistant Professor of International & Transcultural Studies at Teachers College, is scheduled to be published later this year in a book about international assessments. It is the first international survey measuring public attitudes toward global standardized tests such as the Program for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS). The online survey of more than 4,000 respondents in 17 languages was designed to gauge whether respondents believed the tests accurately measure the value of a nation’s education system and contribute to improving schools.

“International assessments such as TIMSS and PISA are very powerful in shaping perceptions of public education systems,” says Pizmony-Levy. “Past research, including our own, has shown that the media's reporting of international assessments can shape public perception of the quality of education and attitudes toward public spending on education. We don’t know yet, but we suspect, that shallow media coverage that portrays the assessments like a sports competition, without context and nuance, is behind the patterns we found in the public opinion survey.”

Pizmony-Levy led an international team of 50 graduate students who conducted public opinion surveys and analysis of media coverage of the test results. The student team was coordinated by Linh Doan, a Ph.D. student in Comparative Education and Sociology of Education, and Jonathan Carmona and Erika Kessler, master’s degree students in International Educational Development.

Other studies have shown that education policies designed to boost a country’s international competitiveness can backfire by lowering people’s estimation of the quality of their local schools without raising their willingness to spend on improving t

International large-scale assessments are “a public matter,” the authors note in their report. They are “funded with public money to inform stakeholders about the output of what is often considered a large component of the governmental sector.”

Yet in another study published earlier this year, Pizmony-Levy found that the general public in Israel and the United States hold inaccurate perceptions about their country’s performance on PISA. In that study, higher-educated respondents even further underestimated their country’s performance on PISA.

This despite what Pizmony-Levy describes as “choreographed, highly publicized rituals” by the Organisation for Economic Co-operation and Development (OECD) when it publishes the results of the PISA, a test of scholastic performance in mathematics, science, and reading of 15-year-olds in member and non-member countries; and by the International Association for the Evaluation of Educational Achievement (IEA) when it releases results of the TIMSS assessments of the mathematics and science knowledge of 4^th- and 8^th-grade students around the world.

Average test scores for each participating country are reported, along with country rankings and charts and graphs illustrating country-by-country analysis of test results. The stated purpose of the reports is to inform policymakers and the public about the quality of their education system compared to that of other countries and jurisdictions.

The OECD and IEA invest considerable effort and resources in designing, analyzing and disseminating these international, large-scale assessments and results. Their thick reports and analyses are often big news among education policymakers and politicians. But they don’t often garner much mainstream interest – or the news coverage is inaccurate or lacks enough context for the general public to grasp its implications.

Pizmony-Levy's most recent study was conducted in November and December of 2016 and found:

1. Respondents question the accuracy of international assessments. Slightly more than one-third (36%) agreed with the statement that “international comparisons tests such as PISA and TIMSS accurately measure student achievement across nations.” The rest responded that they neither agree or disagree (41.6%) or disagree (22.4%). Respondents in all three categories said they support the continued use of global assessments.
   
   
2. Respondents are about evenly split on the question of whether international testing can lead to school improvement. Less than half (48.7%) agreed with this statement, whereas the rest neither agreed or disagreed (26.1%), or disagreed (20.3%).
   
   
3. Although respondents doubted the accuracy of global standardized tests and were unsure about whether they could help improve schools, a large majority of them (79.7%) viewed good performance as important (42.5% answered somewhat important, and 37.2% answered very important).
   
   
4. Slightly more than two-thirds (68.2%) indicated they support participation in international comparisons tests in science, mathematics, and reading in the coming years, with a small minority (9.5%) opposing participation.

The TC research team concluded that additional research is needed on public opinion about international assessments.

“National and international standardized tests have become a central element in accountability,” Pizmony-Levy says. “These and other studies suggest that we need to pay more attention to public engagement and reaction to these standardized tests. Otherwise we might be missing the point and wasting resources.” – Patricia Lamiell

Published Monday, Jun 12, 2017