This story, “Why School Ratings Can Backfire,” was originally published on October 21, 2021, by The Hechinger Report, a nonprofit, independent news organization focused on inequality and innovation in education and based at Teachers College, Columbia University.

By Aaron Pallas, Arthur I. Gates Professor of Sociology and Education

Last week, U.S. News & World Report released its first-ever rankings of public elementary and middle schools in the United States. Relying solely on aggregated state standardized test scores and demographic characteristics of students attending a given school, the periodical ranked elementary schools and middle schools within states and school districts. Schools in the bottom quarter of all schools in a given jurisdiction were not ranked, but simply identified as being in the bottom quarter.

Although U.S. News has proclaimed itself “the global authority in education rankings,” a healthy degree of skepticism is appropriate. Even “global authorities” can screw up! We already have good evidence that school and college rankings can distort normal educational processes, reinforcing social hierarchies that govern who enrolls in a school, how those students are treated and what happens to them thereafter.

At the K-12 level, we have seen how school ratings can boost or depress property values and shift who seeks to enroll in a given school.

The GreatSchools.org ratings attached to real estate websites such as Zillow, realtor.com and Redfin often are lower for schools serving low-income students and higher concentrations of Black and Hispanic students. If parents vote with their feet, the ratings may independently contribute to even greater levels of segregation across schools.

In truth, we know that, on average, schools serving lower-income children and children from historically marginalized racial and ethnic groups will inevitably receive lower ratings from schemes—like the new U.S. News rankings—that rely on the percentage of students who are deemed proficient in English and math.

The U.S. News ratings seek to blunt such criticisms by claiming that they take into account not only raw proficiency rates, but also rates that are adjusted for the mix of students in a given school. By blending these two ways of thinking about a school’s performance—the raw performance level of a school’s students, and an adjustment that seeks to identify schools that are doing better than other schools serving similar students—and then ranking schools on this blend, U.S. News intends to factor in both a school’s overall academic performance and its relative performance compared to similar schools.

The trouble is, it doesn’t work. At least that is the conclusion I reached after looking at data on more than 400 traditional public middle schools in New York City, where the rankings are dominated by students’ absolute proficiency levels. Moreover, those proficiency levels are closely tied to students’ family economic status and their entering levels of proficiency in English and math.

What the U.S. News rankings do, essentially, is identify schools that have been successful in enrolling higher-achieving, more affluent students. The focus is rather problematically on inputs rather than outputs. Shouldn’t we be far more interested in outputs, such as how schools contribute (or not) to student learning and development?

I chose New York City middle schools to illustrate this point because New York publishes a lot of data on its public schools. All districts and states do; the 2015 Every Student Succeeds Act requires states and school districts to publish report cards that include information on student performance, accountability ratings, per-pupil expenditures, and other information that parents and school officials deem relevant.

The U.S. News percentile rating for New York City public middle schools is predicted almost perfectly by the average percentage of students judged proficient in English and math on New York’s state assessments for grades 6 through 8.

The correlation between the two is r = .95, where 1.0 represents a perfect correlation. The U.S. News effort to introduce an adjustment for the kinds of students that a given school enrolls seems to be of little consequence. All schools in the bottom quartile of the U.S. News rankings have an average proficiency rate of 28 percent or lower.

There is also strong evidence that the U.S. News rankings reflect the composition of the students attending schools, much more so than anything that schools actually do. Schools with concentrations of low-income students, based on New York City’s economic need index, are much more likely to have low U.S. News rankings than schools with economically advantaged students. The correlation is r = -.68, a strong association between student needs and a school’s ranking: the more students in poverty, the lower the ranking.

Finally, U.S. News ranking percentiles for New York City middle schools are tightly linked to students’ academic performance at the time they entered middle school. The correlation of r = .88 indicates that we can predict a middle school’s U.S. News rating very, very well simply by knowing how its students did in fifth grade — before entering middle school.

Is there a better way to rank schools? Some questions answer themselves. The right question to start with is: What problems are school rankings intended to solve?

Are the U.S. News rankings intended to provide parents with information that can help them choose a school that is right for their child? Ranking schemes such as U.S. News’ do nothing to inform parents about what kind of experience their child might have in a school. Moreover, not all families have the opportunity to choose a desirable elementary or middle school without moving to a new neighborhood. As recently as five years ago, the parents of only 41 percent of students in grades 1-12 reported that public school choice was available to them.

Are the U.S. News rankings designed to hold schools accountable for their performance? If so, we would want a robust public debate about which outcomes matter most.

I guarantee you it wouldn’t boil down to scores on standardized tests. Moreover, policy entrepreneurs now realize that there is as much, if not more, interest in a school’s contribution to a child’s growth than in a snapshot of where children are at a particular moment in time.

Are the U.S. News rankings intended to be an accessible tool for parents to compare schools to one another? If so, what’s the point? State and local school report cards already do this.

It’s hard not to conclude this without a joke about where I would rank the new U.S. News rankings among all school rating and ranking schemes. As a matter of courtesy, I’ll simply say that they are somewhere in the bottom quarter of all ranking schemes.