You have /5 articles left.
Sign up for a free account or log in.
U.S. News & World Report may be on the verge of significant changes in its methodology.
The magazine has sent out surveys to 1,600 high school counseling offices asking them to evaluate colleges, and the results may be used in next year's rankings. Or not. U.S. News isn't deciding how to use the high school data until after the results come in. In addition, the magazine is asking a series of new questions of college presidents, having them identify "up and coming" colleges, inviting them to offer suggestions on changing the relative weights the magazine uses in its rankings, and giving them a series of possible additional measures and asking which should be added to the methodology.
Taken together, these changes could result in significant changes to the magazine's controversial and much debated methodology. And the magazine -- following a year in which more college presidents criticized it and fewer participated in its surveys -- is reaching out to educators for advice. But already the possible changes are themselves causing more controversy. Some high school counselors are calling for a boycott of the survey, and a leading critic of the rankings says that the possible changes show once again how flawed the rankings are.
"I think these changes reveal that they are acknowledging the lack of precision and the impropriety attached to this instrument," said Lloyd Thacker, president of the Education Conservancy. He said that at a time that the failings of the rankings have been exposed, U.S. News is seeking to involve more people "to give this the appearance of authority."
The new survey of high school counselors is going to the college advising offices at the 1,600 high schools that are the winners of another of the magazine's rankings, "America's Best High Schools." Half of the high schools will be asked to rank the 266 institutions that U.S. News considers "national liberal arts colleges." The other half of counselors will be asked to rank the 262 institutions that the magazine considers "national universities." The rankings will be on a five-point scale. Counselors will be told to rank only colleges they think they know something about, but the basic format is subjective in the same way the magazine's presidential survey is subjective, allowing different people to rank in different ways.
Robert Morse, who runs the magazine's college rankings division, said that high school counselors have over the years asked the magazine why their views were never sought, so the magazine decided it was time to ask them. Morse said that the magazine would publish the results in some way, and might -- but might not -- add the results to the actual rankings of colleges.
Many critics of U.S. News have charged in the past that when the magazine changes its methodology, it usually does so only in ways that keep certain colleges (Harvard, Yale, Princeton and Stanford Universities; and Amherst, Swarthmore and Williams Colleges) at the top, because many in the public expect them there. Asked if it wouldn't reinforce that criticism for the magazine to conduct a major national survey and decide whether to use the results from it only after those results are in, Morse reiterated that the magazine was trying to be responsive. He also said that the magazine wanted to be sure it received good response rates from different parts of the country.
On Tuesday, as word of the high school survey reached the discussion list of the National Association for College Admission Counseling, some were pointing out that what many high school counselors have been saying for years isn't that they want to be asked to rank colleges, but that they view the U.S. News rankings as counterproductive to their efforts to help students make thoughtful choices.
One counselor posted a message making fun of the magazine's claim that its readers needed the knowledge of counselors. "I suggest that everyone who gets one of these things sends it back with a letter that says we are refusing to participate in this ridiculous exercise. I do have knowledge that is extremely valuable to U.S. News readers -- Don't buy this magazine."
Another counselor suggested a draft pledge for counselors to sign in which they disavow the new survey and say: "I believe that ranking colleges in the way that U.S. News has done over the past years has contributed greatly to a climate of anxiety that has exploited the very children I have given my professional life to serve, guide, and educate."
Morse, of U.S. News, said he hoped that counselors wouldn't be influenced by the views of colleagues he characterized as having "anti- U.S. News attitudes."
The magazine also has just sent out this year's survey to college presidents asking them to rank institutions. This year's survey also includes additional questions:
- Presidents will be asked to name up to 10 colleges they think are "up and coming." Morse said that this is a response to complaints that the magazine's reputational survey -- among the most controversial parts of the rankings -- is slow to respond to improvements by colleges.
- Presidents will be asked for advice on changes the magazine should consider in its methodology.
- Presidents will be given a list of possible additional categories on which colleges could be ranked, and which could be used in the methodology for best colleges. These items include: ethnic diversity, admission yield, economic diversity, total endowment per student, learning outcomes, job placement data, and spending efficiency.
Morse said that the magazine has not decided whether it will release information on what the presidents say.
Why consider all these changes? Morse said that the effort was "partly in response" to the criticism of college presidents, some of whom have pledged not to fill out the magazine's surveys any more. But more broadly, Morse said that "we want to make what we are doing better."
Thacker said he was unimpressed by the effort. The rankings play a role that is "educationally meaningless" and that distorts the decisions of applicants and colleges, he said. But by considering all of these methodology changes, Thacker said, the magazine was admitting what a "fake science" its work is. The magazine, if it changes its methodology, will be admitting the lack of validity of rankings it pushed on high school students and their families for years, he said.
"The magazine is just trying to perpetuate what it has as more people ask questions," he said.
Further, he said that changes in methodology can't fix a central problem: "The problem with the rankings is that they are rankings. I do not believe you should ordinally rank a college experience."
