April 4, 2007

Another Rankings Fiasco at ‘U.S. News’

Portland State University’s engineering college has been transformed into “a national and international academic and research institution.” The excellence of the college “illustrates how state investments in higher education can increase programmatic capacity.” The university’s electrical engineering department is so good that it’s in a “top 10″ listing with such institutions as the Massachusetts Institute of Technology, Stanford University and the University of California at Berkeley. “We knew PSU engineering was significantly under-ranked. But Top 10? Wow! It made my day. Go PSU!”

Looking for a job?

See all 169 new postings

Browse all job listings:

Faculty: 2,694
Administrative: 2,046
Executive: 153
FEATURED EMPLOYERS

Those quotes are all from a press release that the Oregon university rushed out Friday, upon learning that the 2008 U.S. News & World Report guide to graduate programs had ranked Portland State No. 9 in electrical engineering. You won’t find the press release on the Portland State Web site any more. It turns out that Portland State doesn’t make the Top 10 — or the top 70 for that matter.

The stand-alone guidebook that U.S. News sent to newsstands nationwide last week (the version designed for sale throughout the year and purchased by college counseling centers, not the weekly magazine) turns out to have included Portland State and the University of Texas at Arlington incorrectly in the top 10 for electrical engineering — when their rankings are actually so low that the magazine doesn’t give them out. Portland State’s No. 9 slot really belongs to Carnegie Mellon University. Texas-Arlington’s tie for the No. 10 spot really belongs to another University of Texas campus — the flagship, in Austin.

It was Austin that first notified U.S. News that something might be wrong when officials at the university were confused about how the institution’s top-ranked electrical engineering program could have been overlooked.

How did Portland State and Arlington get their incorrect ranks?

Robert J. Morse, director of data research for U.S. News, blamed technology errors by the company hired by the magazine to conduct its surveys for the graduate rankings. “There was an error in data output,” he said.

Morse stressed that the error was uncovered before the magazine listings of the rankings or the magazine’s Web site went live, so those listings are correct. Next week’s issue of the magazine will contain a correction about the rankings that appear in the stand-alone guide, he said.

The rankings in question are based entirely on a reputational survey conducted of department chairs in the field, who were asked to rank the programs on a scale of 1-5, with 5 being the best score. To be ranked in the top 10 in electrical engineering, a department needed a score of at least 4.1 this year. Morse said he would not reveal the scores of the two universities incorrectly listed in the top 10, but they are below 2.5, since that is the lowest rank published by U.S. News. Eight universities’ electrical engineering departments tied for No. 73 on the list with that score.

One irony of the mistake is that a common criticism of the reputational surveys used by U.S. News is that they tend to favor programs that have existed and done well for a long time. Institutions like Portland State and UT-Arlington — that are respected but don’t have decades of experience as research powerhouses — are very unlikely to see their departments have a rapid rise in a reputational survey.

So how is it that U.S. News didn’t ask questions when Portland State displaced Carnegie Mellon (not to mention Princeton University and the University of California at Los Angeles, among others) and UT-Arlington replaced UT-Austin? Morse said that the magazine relied on the computer printouts it received. “It’s bad that we did this. There’s no excuse,” he said. Morse characterized the mistake as “not a big deal” in a big picture way, but acknowledged that it probably was to the universities involved.

UT-Arlington didn’t issue any announcement about the situation and a spokesman said Tuesday that he was just learning now about its meteoric rise and fall in the electrical engineering rankings.

Portland State had to rush out an e-mail to its press list, advising recipients to ignore the earlier release and to contact Morse at U.S. News to find out why it happened.

Joan Barnes, assistant vice president for communications at Portland State, said that educators there were “disappointed at this unexpected turn of events,” but not discouraged. “We’re redoubling our efforts to serve Oregon with confidence that increased national recognition will follow our success.”

— Scott Jaschik

Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.

Advertisement

Comments