A Shift to Outcomes?

Next week will be the annual round of hype over the U.S. News & World Report rankings -- as some institutions boast and others, having dropped a few notches, develop a newfound skepticism for the evaluations. The magazine announced Tuesday that there are more changes than usual this year in the methodology, and that they will lead to more movement than is typical in the rankings.

Some of the changes (without much detail) were announced in the blog of Robert Morse, who oversees the college rankings for U.S. News. But a spokeswoman for U.S. News said that Morse would provide no further details until next week. So some questions raised by the announcement (such as whether some categories are getting more weight, and which are getting less) aren't clear. Morse wrote that the changes would "reduce the weight of input factors that reflect a school's student body and increase the weight of output measures that signal how well a school educates its students." But one of the changes will result in SAT or ACT scores of incoming students counting more than they have in the past in one category of the rankings.

Here are the changes announced by U.S. News:

• The "student selectivity" portion of the methodology will count for 12.5 percent of a college's total, not 15 percent.
• Within the student selectivity formula, class rank will count for 25 percent, not 40 percent. The change is attributed to the increase in the proportion of high schools that do not report class rank. SAT/ACT scores, meanwhile, rise to 65 percent from 50 percent of that score. (The rest isn't explained but has in the past been based on colleges' acceptance rates.)
• Graduation rate performance (a measure that attempts to reward colleges for doing better than expected with their student body) will be applied to all colleges, not just the "national" ones at the top of the rankings.
• "Peer assessment" -- one of the most widely criticized criteria, based on a survey of presidents -- will be cut from 25 to 22.5 percent of the formula for evaluating regional colleges. (One of the questions U.S. News declined to answer was whether there would be any change in the weighting for national universities.)
• Graduation and retention rates will matter more for national universities, going from 20 percent to 22.5 percent.

Lloyd Thacker, founder of the Education Conservancy and a longstanding critic of rankings, said that the changes didn't address underlying problems with the system. "Well-intentioned people have lobbied U.S. News for years to make changes to encourage more of an emphasis on educational values," he said. "I'm skeptical this changes anything. Ordinal rankings are fundamentally flawed."

Robert Schaeffer, public education director of the National Center for Fair and Open Testing, a longtime critic of the SAT and other standardized tests, said via e-mail that as more colleges end test requirements for admissions, it makes "no sense" for U.S. News to pay more attention to SAT scores. "The truth is that all ranking systems are no better than parlor games -- the factors selected and the weights assigned to them determine the ultimate result," he added.