‘Breaking Ranks’ With ‘U.S. News’

Steven S. Koblik gave a gift to subsequent presidents of Reed College: in 1995, he declared that the college would no longer cooperate with U.S. News & World Report on its college rankings.

Colin Diver, who succeeded Koblik, writes that when he arrived at Reed in 2002, he thought, “I’m no longer subject to the tyranny of college rankings. I don’t need to worry about some newsmagazine telling me what to do.”

In a book being published tomorrow, Breaking Ranks: How the Rankings Industry Rules Higher Education and What to Do About It (Johns Hopkins University Press), Diver describes his experience as a college president (and previously as a law dean at the University of Pennsylvania) in dealing with rankings. He absolutely agrees with the decisions Reed made about the rankings. And he believes that rankings aren’t going anywhere but that colleges can, and should, fight them.

Diver writes that Reed’s “rebellious stance” on U.S. News was part of what attracted him to Reed in the first place. “I took it to be a statement that Reed viewed education as a path to a genuinely fulfilling life, not just a ticket to a high-paying job.”

In an interview, he said that his perspective on rankings came nearly as much from his experience as a law dean as it did from his experience as a president. (He is currently the Charles A. Heimbold Jr. Professor of Law and Economics Emeritus at Penn.)

Why Oppose Rankings

Diver starts his book with a review of why he sees it as dangerous to rank colleges.

In a chapter called “Apples, Oranges and Refrigerators: Should Colleges Be Ranked?” he discusses how Consumer Reports evaluated refrigerators, noting (with praise) the way it conducts actual tests on them.

“It makes perfectly good sense to rate refrigerators,” the book says.

On colleges? Not so much, he writes.

But he first acknowledges reasons people might value college rankings. He notes that college is “one of the most complex and expensive” purchases one makes. And bachelor’s degrees at various colleges typically require the same number of credits. For residential colleges, students might think they are buying the same product, with fees for residence halls, food, etc.

But he also notes the reasons not to rank colleges.

Giving examples, he writes, “Chad is a conscientious student from a working-class family who is skilled at carpentry and craves the thrill of a packed football stadium. Tanya is the only African American student in her high-school class who took both physics and advanced calculus. The third child in a devoutly Catholic family, Maureen is the student government president and captain of her tennis team.” After several other examples, Diver asks, “In a world of such diverse stories, and so much human variety, is there a single ‘best’ college? Are there even 500 best colleges?”

But there are other reasons to question rankings, Diver writes. “Unlike apples and oranges, or refrigerators and cars, college is not simply a short-term consumption activity. It is a long-term investment in human capacity—the ability to do financially, socially, emotionally, and even spiritually rewarding work; to teach oneself and others how to learn, adjust and adapt; to analyze, reason, evaluate and create; to appreciate beauty, ingenuity, order, complexity and subtlety.”

A Focus on ‘U.S. News’

Diver focuses on U.S. News, he writes, because it dominates undergraduate rankings in the United States. (He does comment on others in the industry—more on that later in the article.)

And he shares stories that could make one doubt the credibility of U.S. News. “In 1999, the U.S. News statisticians made an obscure change in the way the magazine plugged spending per student into its overall score computation,” Diver writes. As a result, the next year, the California Institute of Technology vaulted from ninth in the rankings to first place. “Oops! The editors made short work of that statistical adjustment, and Caltech settled back to its ‘proper’ position in the pecking order, below the perennial top dogs (Princeton, Harvard, Yale and Stanford Universities) … Imagine an applicant who, in 2000, chose Caltech because it was ranked 1st, in preference to, say Princeton (then 4th). A year later, that person wakes up to discover that the two schools have traded places. By graduation time, Princeton is still 1st, while Caltech has sunk to 8th.”

One of the things about U.S. News that particularly bothers Diver is that it punishes colleges that resist it. He notes that U.S. News responded to Reed’s approach not by “simply dropping Reed” from the rankings, but by giving it the lowest scores on subjective parts of the methodology for national liberal arts colleges. Reed, which had appeared as high as ninth, ended up in the bottom quartile after taking a stand against rankings. The late Alvin Sanoff, at the time in charge of the rankings, admitted the change was “more punitive than logical.”

Writes Diver, “His choice of words is instructive: punishment is a treatment ordinarily dispensed by someone in authority.”

At the same time, he notes that the University of Chicago was for years a laggard (at No. 15) in U.S. News rankings, at least compared to the reputation of its faculty. Robert Zimmer, at the time Chicago’s president, in 2006 launched a campaign to get more applicants, based on U.S. News favoring that strategy. It worked, raising Chicago’s applicant pool from 9,100 to 34,600 in 2020. Chicago now is usually in the top 10.

“Many observers, especially among Chicago’s old guard, have grumbled that this accomplishment came at the sacrifice of the university’s uncompromising intellectual standards,” Diver writes.

Law Schools

Diver also spends time on the U.S. News rankings of law schools. And here he confesses to playing the U.S. News game when he was dean at Penn’s law school.

At the time, U.S. News calculated the average of the 25th percentile score on the Law School Admissions Test and the 75th percentile score. “My admissions dean would sometimes charge into my office, gushing with excitement about an unusually talented applicant—perhaps a person of color, or an older student who had overcome enormous life challenges. I would listen sympathetically and invariably say, ‘yes, as long as your prized candidate doesn’t drag down our 25th percentile number.’ Shame on me.”

Another time, when he was at Reed, he worked closely with a student. “I had spent the academic year supervising his senior thesis project on a constitutional law topic. The finished product was superb, equivalent in quality to the most student-authored writings published by leading law reviews,” Diver writes. “I was on good terms with the dean of the law school he aspired to attend and offered to put in a good word. No sooner had I concluded my spiel than my dean friend fired back, ‘What’s his LSAT score?’ My answer elicited precisely the response I had expected: ‘Well, that’s below our 25th percentile. I’ll have to put him on the waiting list.’”

Diver believes that the law school, and plenty of undergraduate colleges, are making decisions based on whatever U.S. News wants.

Making student financial aid decision based on some definition of “merit” (frequently high test scores) is another practice Diver dislikes and links to rankings. Undergraduate colleges and law schools give money to students who could afford to enroll without the aid to capture their high SAT or LSAT scores, he writes.

The Future

The rankings, at least those of U.S. News, “are here to stay for a while,” Diver said. Applying to college is complicated, and there are people who like rankings, even if they don’t understand the flaws in them, he added.

So what are the possibilities for reform?

Diver offers four recommendations for colleges:

• Don’t fill out peer reputation surveys. “Educators need to be honest with themselves about the limits of their knowledge regarding other institutions,” he said.
• Don’t publicize rankings you consider illegitimate. “If a particular ranking, such as the one published by U.S. News, is fundamentally incompatible with your values, then don’t brag about your score on it.”
• Celebrate rankings that truly reflect your values. Berea College or City College of the City University of New York might publicize a ranking that focuses on educating low-income students, he says.
• Give everyone equal access to your data. “Educators should not give anyone, including specific college rankings organizations, privileged access to all or even some portion of their data.”

What the Rankers Say

Of course there are rankings beyond U.S. News. While the book’s focus is on U.S. News, Diver acknowledges that there are other rankings. Generally, he sees them in the same way he sees U.S. News—critically.

One of the other rankings is done by Times Higher Education, which in January bought Inside Higher Ed. The book discusses briefly Times Higher Education’s ranking of American colleges that it conducts with The Wall Street Journal. (The book says that it is largely ignoring global rankings.)

“Despite their different focuses and methodologies, most of these publications [including Times Higher Education/The Wall Street Journal] have regularly given the highest billing to the same cast of Ivy-plus characters,” Diver writes.

Ellie Bothwell, the rankings editor of Times Higher Education, said her publication had decided not to comment.

U.S. News released this statement: “We haven’t seen the book, but we know students and their families find significant value in our rankings. We strive to provide them with data and information to help make important decisions, using the rankings as one factor in their college search. As always, we continuously welcome feedback.”