A New Approach to Ranking Colleges

When the public policy think tank Third Way released a report earlier this year ranking colleges on an economic mobility index measuring the return on investment for low-income students and the percentage of Pell Grant-eligible students that each institution enrolls, the results were surprising.

Out of 1,320 colleges and universities ranked, the institutions that provided the greatest economic mobility for their students after they graduated were Hispanic-serving Institutions in California, New York and Texas, according to the report. Historically Black colleges and universities also performed better using Third Way's economic mobility index than on traditional rankings. But the colleges that typically got high marks for return on investment, or ROI, for low-income students fell substantially because they enroll relatively few low-income students. Under the new index released earlier this year, Duke University and Stanford University, which were previously ranked first and second based on how their Pell-eligible students did, dropped to 722 and 548 after the ranking also took into account how many of those students attended those universities. Harvard ranked 847th and Princeton ranked 426th.

Third Way released a new update to its Economic Mobility Index last week that groups colleges and universities into a series of higher and lower tiers based on how they serve low-income students. It also added new metrics to make the index a more informative tool for higher ed leaders, researchers, federal policy makers "and other oversight entities with actionable information about which institutions are delivering on their promise of providing intergenerational mobility," a Third Way blog post explained.

The updated index now organizes institutions into five tiers, grouping together colleges and universities that offer similar levels of economic mobility. The differences between tiers, and individual institutions, can be explored using a new interactive map. The change was made to demonstrate that institutions in the same tiers can provide almost equal returns, a point researchers worry can get lost in a normal ranking structure.

" … If a school is ranked number 15, then it may be perceived as being twice as good as a school that's ranked as number 30, but that's not necessarily the case," said Michael Itzkowitz, higher education senior fellow at Third Way and previously the director of the College Scorecard. "They both do a really good job."

For example, the blog post highlighted that the University of North Texas at Dallas ranks twice as high as Florida International University in the index, but both are high-level institutions when it comes to upward mobility. About half of the students at both institutions are recipients of Pell Grants, the primary federal aid program for low-income students, and both institutions offer those students low out-of-pockets costs and high economic returns. Low-income students who graduated from North Texas earned $18,000 more than the typical high school graduate in the state, while those who attended Florida International earned $24,684 more than the typical high school graduate in the state.

Artem Gulish, senior policy strategist and research faculty at Georgetown University Center on Education and the Workforce, said the tiers could help prospective students and parents trying to assess colleges.

"A tiered approach does provide an interesting nuance," Gulish said in an email. " … A broad classification by tier, rather than specific ranking, may make it easier for students and families to obtain information they need to make enrollment decisions, without getting lost in the weeds of which school ranks 5th and which ranks 6th in a given year."

Breaking the ranking down into tiers also showed a significant difference between institutions with high levels of economic mobility and low levels of economic mobility in terms of the time it took students to recoup their financial losses, the blog post noted. Low-income students at institutions in the top tier earned back the costs of their education within 2.2 years, while low-income students at institutions in the lowest tier took 13.7 years, six times as long.

In addition to the tiers, the updated data includes how much federal financial aid annually flows to each institution. One hopeful finding was that institutions in the top tier for economic mobility received more aid dollars than other institutions. The 264 colleges and universities in the top tier received about $25.4 billion in federal student aid during the 2019-20 award year, while the 263 institutions that provided the least economic mobility received about 8.8 billion.

"Taxpayers contribute a substantial amount to higher education every year," Itzkowitz said. "So, we wanted to see whether or not that's actually lifting students up and which tiers of economic mobility that's flowing to. The promising news is that's actually mostly flowing to schools that do a really good job on providing economic mobility for students who enroll."

He noted however that billions of dollars still go to institutions in the lowest strata "where if students attend, they may be limited on their return on investment and schools may not be enrolling a high proportion of low or moderate income students in the first place."

Economic mobility indexes have become a popular tool to hold institutions accountable for student outcomes.

Gulish noted that Georgetown Center on Education and the Workforce also released a report this year about which colleges and universities deliver the highest returns for low-income students. He believes the Carnegie Classifications of Institutions of Higher Education, a longstanding framework for categorizing colleges and universities, should include these kinds of metrics and that economic mobility rankings should be better publicized so students and families can take advantage of them. The Carnegie Foundation and the American Council on Education have already started moving in that direction.

"While higher education is generally seen as the pathway to economic opportunity in the United States, it has in fact reinforced intergenerational reproduction of race and class privilege," he said. "This is why it's so important to shine a light on which institutions are actually delivering on the promise of economic mobility for low-income students. And institutions that deliver on this promise need to be recognized."

Brendan Cantwell, associate professor of educational administration at Michigan State University, worries these rankings can sort individual institutions into "who's good and who's bad" in a way that can obscure more subtle obstacles to upward mobility.

To him, the updated index is an "improvement" because the tiers take some of the focus off of individual institutions. But the changes don't alleviate his concerns that the index doesn't pay attention to factors at the state and community levels that contribute to some institutions falling to the bottom and others to rising to the top when it comes to economic mobility.

"The whole point of this is to hold institutions accountable or to create policies or momentum to hold institutions accountable for economic mobility in order to get federal financial aid, and without addressing these other deeply rooted contexts … holding those institutions accountable doesn't do much," he said.

For example, he noted that Pennsylvania has very few public institutions that make the upper tiers of Third Way's ranking, but he doesn't believe that's necessarily because those individual institutions are not well-serving low-income students.

He said there are other factors at play: the middle region of the state has an economy with "high inequality and low growth" where it's hard for graduates to find jobs, and the state has some traditional industries, such as mining and manufacturing, where people without college degrees can earn well-paying salaries. The state has also had a "disastrous public policy toward funding higher education, one of the worst in the country."

He'd prefer to see "more nuanced" conversations about economic mobility happening at a local level.

Itzkowitz said the index takes geography into account by comparing the earnings of college graduates to the typical high school graduate in the state to determine whether a college degree offered increased returns.

But he also recognizes there's a "balance of the perfect with the good," and he hopes the updated index is "a great starting point for folks to build off of, whether that be ourselves or others in future iterations."

"This provides a broad understanding which hopefully helps folks dig deeper and look for context within the data itself," he said. "That's what this is there for."