News
'Manipulating,' Er, Influencing 'U.S. News'
ATLANTA -- Higher education's love-hate relationship with college rankings was on full display here this week at the annual forum of the Association for Institutional Research, where -- despite the continuing campaign by some campus presidents to marginalize rankings -- campus number crunchers were treated/subjected to at least a half-dozen sessions on the subject.
As is common at this gathering of self-described wonks, some of the sessions (like the one on using Microsoft Excel to adjust raw rankings data) were practical to the point of being arcane, but most aimed to defend, dissect or debunk them. Officials from U.S. News & World Report and the Times Higher Education Supplement were on hand to tell the men and women who in many cases are responsible for submitting information about their campuses how their rankings were evolving and what might be coming down the road, for instance. In another session, researchers from Michigan State and Pennsylvania State Universities examined a decade's worth of U.S. News results to show how little change there was in colleges' performance in the rankings' controversial "reputational" score, and that what movement did occur was tied mostly to changes in the selectivity of their student bodies.
One session, however, revealed more than any other the extent to which the rankings, for all the protestations to the contrary, influence colleges' behavior. A presentation by Catherine Watt, the former institutional researcher and now a staff member at Clemson University, laid bare in a way that is usually left to the imagination the steps that Clemson has (rather brazenly) taken since 2001 to move from 38th to 22nd in U.S. News's ranking of public research universities. (Note: This paragraph has been altered to correct Watt's current position at Clemson.)
This was no accident. When President James F. Barker took over the South Carolina institution in 2001, he vowed in his initial interview to move Clemson into the top 20 (a distinction that many research universities covet, but few can achieve, given that most of those already in the top 20 aren't eager to relinquish their spots). Although many people on the campus were skeptical, Clemson has pursued the goal almost single-mindedly, seeking to "affect -- I'm hesitating to use the word 'manipulate,' " Watt said -- "every possible indicator to the greatest extent possible." She added: "It is the thing around which almost everything revolves for the president’s office."
That statement was among the first at Watt's session that provoked murmurs of discomfort (and more) from the audience -- there would be many more as she described the various steps Clemson had taken to alter its profile in order to improve its U.S. News standing. Watt, director of the Alliance for Research on Higher Education at Clemson's Strom Thurmond Institute of Government and Public Affairs, subtly made clear her own discomfort with Clemson's approach.
The easiest moves, she said, revolved around class size: Clemson has significantly increased the proportion of its classes with fewer than 20 students, one key U.S. News indicator of a strong student experience. While Clemson has always had comparatively small class sizes for a public land-grant university, it has focused, Watt said, on trying to bump sections with 20 and 25 students down to 18 or 19, but letting a class with 55 rise to 70. "Two or three students here and there, what a difference it can make," she said. "It's manipulation around the edges."
Clemson has also transformed its admissions standards, more or less ceasing to admit full-time, first-time undergraduates who are not in the top third of their high school classes, and constantly re-assessing its SAT average throughout the admissions cycle, Watt said, so that admissions officials know whether they "have to increase the SAT score in the next round" of students.
Bringing about other changes has been harder, Watt continued, as she described what she called the "more questionable aspects of what we’ve done." The university has ratcheted up the faculty salaries it reports to U.S. News by about $20,000, which it has achieved by actually increasing spending (paid for largely through increased tuition) and by altering the way it relays the data to the magazine's editors. Clemson folded its benefit payments into the average faculty pay figure it reports to U.S. News, requiring the institutional research office to produce several different definitions of faculty pay for U.S. News, the American Association of University Professors and other surveys, Watt said. (Clarifying Clemson's approach after the panel for a reporter and an interested Robert Morse, director of data research for U.S. News's college rankings, Watt said that the university had added benefits to its faculty salary reporting to U.S. News after previously having failed to do so, as the magazine requires. So its jump came not from double counting or including information that it should not have, but from playing catchup.)
In reporting institutional financial information to the magazine, she said, Clemson runs "multiple definitions to figure out where we can move things around to make them look best" in the rankings. Academic expenditures are emphasized and administrative overhead minimized wherever possible, within reason, she said. The university has encouraged as many alumni as possible to send in at least $5 to help bring up their giving rate, and hired a firm to find disconnected alumni.
And to actual gasps from some members of the audience, Watt said that Clemson officials, in filling out the reputational survey form for presidents, rate "all programs other than Clemson below average," to make the university look better. "And I'm confident my president is not the only one who does that," Watt said.
Taken together, the changes have had an impact on numerous U.S. News indicators: the proportion of freshmen who were in the top 10 percent of their high school class has risen to 42 from 34 percent; student to faculty ratio has dropped to 14:1 from 16:1; the retention rate of freshmen has climbed to 89 from 82 percent and the graduation rate to 78 from 72 percent. And as those last few results show, Watt said, many of the changes Clemson has made have helped students.
But many of the administrators and data analysts in the audience were clearly troubled by Watt's description of Clemson's approach, especially as she pointed out that the university has grown more exclusive (fewer than 10 percent of its undergraduates are first-generation college students) and has "favored merit over access in a poor state," sending tuitions rising."To me it’s a little unsettling what you’re doing," said one audience member. "You had a perfectly good institution" before.
“We have been criticized for not fulfilling the mission of a public land-grant institution,” Watt responded. But "we have gotten really good press. We have walked the fine line between illegal, unethical, and really interesting.”
"You're pandering," said another, who was especially troubled by the idea that the university would produce multiple versions of its salary data. "It can be confusing, uninformative, hard to keep straight, and I've always been very vigilant about not doing that. It's a very dangerous activity," the administrator said. "How can you possibly justify doing that?" he asked Watt.
"To do anything else is not an option," she said.
"You mean you're just following orders?" the questioner asked.
"Yes." (A spokeswoman for Clemson did not respond to e-mail messages late Tuesday seeking a response to that assertion.)
Not everyone was troubled. "I love" your approach -- "it's pragmatic and forthright," said another audience member. "It's so common [for colleges] to ignore U.S. News, and yet when they drop, they complain."
Watt admitted at several points in the presentation that she was trying to contain her own "cynicism" about her institution's approach, but she also recounted a comment that a colleague had made to her at breakfast Tuesday, when they were discussing her upcoming session and Clemson's aggressive approach to rising in the rankings. "People don’t have this as their official vision," she quoted her colleague as saying, "but by God it’s their unofficial vision."
