Do-It-Yourself Ranking

The National Research Council says its new ratings methodology makes it impossible for any department to claim any ordinal rank. That didn't stop Penn State's philosophers.
October 25, 2010

The new doctoral rankings from the National Research Council have left many academics confused and/or frustrated. One of the issues that have received the most attention is that the NRC used two methodologies (which yielded different results) and that the results for each methodology were reported as ranges, not as a single ranking. NRC officials have said repeatedly that the system was designed to allow departments to report two ranges to reflect their relative status, but no more.

But that didn't stop Pennsylvania State University's philosophy department. It announced on its website that it has a rank -- and a good one, at No. 11.

Here's what the site says: "Penn State’s philosophy department has been highly ranked by the National Research Council (NRC) in its long-awaited national review of Ph.D. granting universities. Out of 90 Ph.D. granting philosophy departments in the United States, Penn State is ranked #11 in terms of overall quality (top 12%) using survey-based weights that evaluated departments’ faculty, students, and program as a whole."

The claim is particularly striking because NRC officials even went so far in briefing reporters on the rankings as to say that when only one department in a discipline is listed as possibly being in the top spot, even such departments can't claim to be No. 1, because the rankings are designed to reflect a confidence level of only 90 percent.

Notably, Penn State's philosophy department wasn't in such a position. When the NRC ran its two methodologies on philosophy, one gave Penn State a range of 6th through 15th (for which 11th would at least be possible). But the other gave a range of 23rd through 54th.

So how is it that one department among all those with doctoral programs can assign itself a specific rank?

Shannon W. Sullivan, chair of philosophy at Penn State, acknowledged via e-mail that the NRC never gave Penn State a rank of No. 11. But she noted that the website took the NRC data and produced lists of departments in various disciplines, and that in one such list, based on one of the NRC's methodologies, Penn State is the 11th institution on the page. She said that language on her department's website referencing "survey-based weights that evaluated departments’ faculty, students, and program as a whole" was from the site's explanation, and noted that Penn State's site links to that other site for further information.

At the same time, Sullivan said that "in order to make it clearer," she planned to have her department's website altered to state that it is based on one NRC methodology and that the NRC gave a range, not a precise ranking.

While Sullivan said it was true that "the NRC tried to avoid giving specific ratings," she added that "we're finding that many of those using the surveys want them." And she said that she didn't consider the No. 11 ranking misleading, "nor would I find it misleading for our peer departments to use a similar shorthand for their own range."

Penn State's boast has been noticed in the philosophy world. The blog Leiter Reports, by Brian Leiter, a University of Chicago professor who writes frequently about rankings in philosophy, law and other fields, featured pointed criticism of Penn State last week, and Leiter's post has circulated among philosophers. The post calls Penn State's statement "complete misinformation," adding: "Penn State appears to have simply concocted its own even odder weighting of a selection of the underlying data, and is now passing that off as a qualitative evaluation. Amazing."


Back to Top