A Shift to Outcomes?

U.S. News announces some shifts in its methodology, and predicts many changes in this year's rankings. But critics aren't expecting to become fans.

September 4, 2013

Next week will be the annual round of hype over the U.S. News & World Report rankings -- as some institutions boast and others, having dropped a few notches, develop a newfound skepticism for the evaluations. The magazine announced Tuesday that there are more changes than usual this year in the methodology, and that they will lead to more movement than is typical in the rankings.

Some of the changes (without much detail) were announced in the blog of Robert Morse, who oversees the college rankings for U.S. News. But a spokeswoman for U.S. News said that Morse would provide no further details until next week. So some questions raised by the announcement (such as whether some categories are getting more weight, and which are getting less) aren't clear. Morse wrote that the changes would "reduce the weight of input factors that reflect a school's student body and increase the weight of output measures that signal how well a school educates its students." But one of the changes will result in SAT or ACT scores of incoming students counting more than they have in the past in one category of the rankings.

Here are the changes announced by U.S. News:

  • The "student selectivity" portion of the methodology will count for 12.5 percent of a college's total, not 15 percent.
  • Within the student selectivity formula, class rank will count for 25 percent, not 40 percent. The change is attributed to the increase in the proportion of high schools that do not report class rank. SAT/ACT scores, meanwhile, rise to 65 percent from 50 percent of that score. (The rest isn't explained but has in the past been based on colleges' acceptance rates.)
  • Graduation rate performance (a measure that attempts to reward colleges for doing better than expected with their student body) will be applied to all colleges, not just the "national" ones at the top of the rankings.
  • "Peer assessment" -- one of the most widely criticized criteria, based on a survey of presidents -- will be cut from 25 to 22.5 percent of the formula for evaluating regional colleges. (One of the questions U.S. News declined to answer was whether there would be any change in the weighting for national universities.)
  • Graduation and retention rates will matter more for national universities, going from 20 percent to 22.5 percent.

Lloyd Thacker, founder of the Education Conservancy and a longstanding critic of rankings, said that the changes didn't address underlying problems with the system. "Well-intentioned people have lobbied U.S. News for years to make changes to encourage more of an emphasis on educational values," he said. "I'm skeptical this changes anything. Ordinal rankings are fundamentally flawed."

Robert Schaeffer, public education director of the National Center for Fair and Open Testing, a longtime critic of the SAT and other standardized tests, said via e-mail that as more colleges end test requirements for admissions, it makes "no sense" for U.S. News to pay more attention to SAT scores. "The truth is that all ranking systems are no better than parlor games -- the factors selected and the weights assigned to them determine the ultimate result," he added.

Share Article

Scott Jaschik

Scott Jaschik, Editor, is one of the three founders of Inside Higher Ed. With Doug Lederman, he leads the editorial operations of Inside Higher Ed, overseeing news content, opinion pieces, career advice, blogs and other features. Scott is a leading voice on higher education issues, quoted regularly in publications nationwide, and publishing articles on colleges in publications such as The New York Times, The Boston Globe, The Washington Post, Salon, and elsewhere. He has been a judge or screener for the National Magazine Awards, the Online Journalism Awards, the Folio Editorial Excellence Awards, and the Education Writers Association Awards. Scott served as a mentor in the community college fellowship program of the Hechinger Institute on Education and the Media, of Teachers College, Columbia University. He is a member of the board of the Education Writers Association. From 1999-2003, Scott was editor of The Chronicle of Higher Education. Scott grew up in Rochester, N.Y., and graduated from Cornell University in 1985. He lives in Washington.

Back to Top