News
'U.S. News' Sees Drop in Participation
Even though many colleges will boast today about their placement in the annual rankings by U.S. News & World Report, more colleges than ever are declining to participate in the survey that makes up the single largest part of the magazine's formula.
Only 46 percent of colleges returned the "reputational" survey, where presidents rate similar institutions. This peer survey is particularly controversial because it is viewed as unscientific and likely to reward colleges that had great reputations in the past. But it makes up 25 percent of the magazine's formula -- more than the statistics gathered on such factors as graduation rates, faculty resources and so forth.
This year's 46 percent participation rate is down from 51 percent last year, 58 percent the year before that, and 67 percent a few years prior. The drop this year was particularly steep for liberal arts colleges -- to 44 percent from 55 percent. Much of the opposition to the magazine's rankings has come from that sector of higher education.
Robert Morse, who leads the rankings project, said in an interview that while participation saw a decrease, "it's still pretty high for a mail survey like this, so the results are still high enough to be statistically significant." Morse declined to say whether there was some level to which participation could fall to make the results invalid. "We're not concerned," he said.
So why has participation been dropping so rapidly over the last several years -- and doesn't that mean something when it's the largest factor in the formula? Morse said that the magazine hasn't done "a scientific survey to determine why the results have slipped," and said that "there could be several factors."
One factor, he said, might be "survey fatigue." Another factor may be competing campus priorities. "Certainly there has been chaos on campus," he said. "People are worried about other things than the U.S. News survey -- there's the Higher Education Act, and how are we are going to finance our incoming class."
Asked whether one factor might be the protests organized by the Education Conservancy, Morse said he could not comment. The Education Conservancy is an anti-rankings group that encouraged college presidents to sign a letter pledging not to fill out the survey on other colleges and not to use rankings in promotional materials.
Lloyd Thacker, founder of the group, said he was "encouraged" to see fewer colleges helping the magazine with its rankings. While he said he has never thought that the survey of presidents had validity, he questioned how the magazine could say it was so important -- and express no concern about such a drop in participation. "If they say that the reputational survey should account for 25 percent of ranking, how can they say a drop of that percentage over a few years doesn't affect the validity?"
The magazine's view that the fall in participation doesn't mean anything "is just another example of how they are not interested in educational validity -- but in selling magazines," Thacker said.
For his part, Morse predicted that two new features this year -- a list of "up and coming" colleges and a survey of high school counselors -- would respond to criticisms of the rankings and may well encourage more colleges to participate next year. "It could reverse itself," he said.
For the high school counselors survey, the magazine turned to counselors at the high schools that U.S. News has ranked as the best public high schools in the United States. Half were sent the survey for national universities and half the survey for liberal arts colleges. The results show that high school counselors (at least those evaluating universities) like the Ivies. In terms of participation, 800 high schools received the university survey and 26 percent responded, while of the 800 high schools that received the liberal arts survey, 27 percent responded.
The "up and coming" category, Morse noted, was a direct response to criticism that the rankings reward longstanding reputations over colleges that have recently made great strides. The methodology for this category was 100 percent based on the presidential survey, with no objective data added. Presidents, while filling out the main survey, were also asked to identify up to 10 colleges that they believed were making significant improvements.
The colleges were ranked based on which ones received the most points. Here, institutions that aren't normally at the top of U.S. News lists did appear: George Mason University, for example, topped the university list, while Davidson College topped the liberal arts list.
This new category also provides an interesting test case for those colleges that pledged not to promote themselves. When the Education Conservancy's movement started, some defenders of the rankings suggested that the critics just had a case of sour grapes. After all, the colleges that signed the pledge were not the colleges that were turning up in the top five of various U.S. News categories, this line of attack went, so of course they favored less emphasis on the magazine's ratings.
But in the new U.S. News category of "up and coming" liberal arts colleges, two institutions whose presidents have signed the pledge were included in the top five: Ursinus College and Furman University. So are their presidents having second thoughts about the pledge now that they could have material with which to boast?
David Shi, president at Furman, said that he could answer with a "hearty No" on the question of whether his university would change its opinion of U.S. News rankings.
John Strassburger, president at Ursinus, said he was pleased that people are noticing the progress at the college, but that it was real accomplishment, not rankings, that mattered. So there will be no press release coming from Ursinus.
"I am concerned that all ordinal rankings can create a distinction where there is no real difference," he said via e-mail. "I think when parents look at Ursinus they see our first year program; they see it widely praised, they see one of the biggest and most successful Undergraduate Summer Fellows programs around, and they note we are one of 50 schools whose students win Watson Fellowships. So I do not plan to change our pledge regarding U.S. News, even while I am happy to think in this case that they got it right."
Other colleges -- whose presidents didn't sign the pledge -- started issuing press releases many hours before the rankings were officially released, at 12:01 this morning. Generally, those releases don't come from those on the top of the lists, but from those wanting reporters to know that the colleges were at the top of some subcategory or made a top 100 list. For those dying to know who is on the top of the research university category (usually the one that captures the most attention), some university based in Cambridge displaced one in New Jersey.
