The New Carnegie Classifications

If you want to know how influential the Carnegie Classifications are, consider that there are still college administrators all over the country who talk about turning their institutions into "Carnegie I's" -- even though that category (once considered the "top" ranking for research universities) hasn't existed for years.

For those who want to aspire to actual classifications, they will be posted today on the Web site of the Carnegie Foundation for the Advancement of Teaching -- with significant changes not only in who is where, but in the categories themselves. In the first update since 2000, research institutions are evaluated in a new way; for the first time ever, community colleges are not lumped together; and in a move that is raising tempers on more than a few campuses, the foundation has made it much more difficult for colleges to qualify for exceptions in which they are listed in one category even though they meet the criteria for another.

The classifications were first published in 1973, designed for researchers who sought to compare similar institutions or foundations that wanted to support a particular sector. While some institutions have always complained about not being in one category or another, the Carnegie system has been well respected. Carnegie never set itself up to be kingmaker among colleges, but the rankings have grown in influence as other groups, less well respected in the education world, have piggybacked off the rankings.

For example, the U.S. News & World Report rankings -- obsessed over by many colleges though they are generally viewed as educationally dubious -- are based on Carnegie's classifications. And while U.S. News will keep the status quo for its next set of rankings, even the chance that colleges might find themselves in new U.S. News groupings has some admissions officers terrified.

Carnegie has pushed over the years, with mixed success, to have the classifications used in ways that are consistent with the foundation's philosophy. For instance, the foundation is a leader among those in higher ed for arguing that traditional research alone is not the best measure of quality, but its classifications have been used by many to imagine a pyramid of the thousands of colleges and universities with the most prestigious few dozen research universities on the top. In its current revisions, the foundation is trying a number of approaches to discourage superficial use of the system.

In a largely cosmetic change, the foundation has changed the order of its categories so that they are based on enrollment. As a result the associate degree categories lead the lists. More substantively, the foundation in November released an entirely different categorization system to operate alongside the traditional system. The traditional system, now called the "basic" classifications, is the update being released today.

The new system uses a variety of measures (what is taught? to whom? in what setting?) to group colleges and provide analysis about them. While institutions end up in only one basic category, they can end up in multiple categories of the new system -- alongside institutions with which the basic system would never group them.

In the basic system, this year's changes largely attempt to make the various divisions more meaningful and accurate, said Alexander C. McCormick, a senior scholar at Carnegie who directs the classification projects.

McCormick said that the most significant change in the basic system was its analysis of research universities, which are shifting to three categories this year: research universities with very high research activity, research universities with high research activity, and doctoral/research universities. The 2000 classifications had only two categories (extensive and intensive) and previous editions had four.

Universities are placed in one of the three categories based on a formula that considers a variety of factors, including research and development spending (both for science and engineering and other areas), numbers of postdocs and non-faculty research staff members with doctorates, doctoral degrees conferred, and a per capita calculation to judge the relative importance of research within institutions. This multiple-factor formula marks a significant departure for Carnegie, which until 2000 based its calculations on federal research spending and in 2000 based categories on doctoral degrees conferred.

McCormick said that the new system came about in part by looking at flaws in previous methodologies. The system used prior to 2000, by focusing on federal research dollars, ended up equating research with certain kinds of science research, he said. "By looking at federal dollars, you are privileging fields where it costs a lot to produce knowledge and by looking at schools that rely a lot on federal funding, you are favoring large institutions," McCormick said.

At the same time, he said, the 2000 system ignored measures of research productivity -- an odd exclusion for a system of categorizing research universities. The combination of measures allows certain institutions to reach higher categories than they would otherwise -- while others fall out. McCormick said that the per capita figure helps institutions that are relatively small, but that have areas of research expertise with the kind of excellence that attracts grants and doctoral students.

Dartmouth College and Montana State University are good examples, he said, of institutions that make the "very high" category that didn't make the equivalent category in 2000. In a similar way, he said, Claremont Graduate University makes the "high category," while it would have been quite low under previous systems because of its small size and because many of its research strengths are outside the sciences.

Among those institutions that were in the "extensive" category in 2000 and that are not in the "very high" category this year are: Auburn, Clemson, Georgia State, Howard, Fordham, Mississippi State and Old Dominion Universities.

Many of those who will be most critical of the new system in the weeks ahead will, not surprisingly, come from institutions that feel that they were bumped out of a category like "very high" research. One close observer of research university rankings who doesn't have that bias is Kermit L. Hall, president of the State University of New York at Albany, an institution that made the "extensive" category in 2000 and the "very high" category this round.

Hall said that, on the whole, the new system seems to be an improvement, doing a better job at making clear "the diversity of higher education." He said he was particularly pleased to see new tools for understanding community colleges. With regard to research universities, he said that he was satisfied that the system counts non-science funds and the number of postdocs -- categories that he said help an institution like Albany and that provide "a fuller and better picture of who is doing what.'

The biggest problem with the new system for research universities, he said, was that in striving to include more measures it has also become "more complicated than its predecessors to digest." Previous systems were "readily understood" and struck many people as "intuitively correct," so there is a danger in having a complex system, he said.

McCormick of Carnegie acknowledged that flaw and said it was something that foundation officials were continuing think about. "The virtue of what we did in the past was that it was very transparent and very straightforward," he said.

While other research university officials shared that concern, officials at several of what are widely considered top institutions applauded the shifts. Many believed that the 2000 system rewarded quantity over quality in doctoral programs, and may even have had the impact of encouraging institutions to expand Ph.D. programs that shouldn't have been expanded.

Over all, here is how the institutions in the research university categories in 2000 fared in 2006 (with the caveat that a few institutions continue to appeal their status, providing new or corrected data to Carnegie). The category numbers don't add up for 2006 because a few institutions are still being classified or have moved out of the research university category.

Number of Institutions by Carnegie Category, 2000 and 2006

Another major change for the classifications was the use of subdivisions with community colleges. As community colleges have grown in number, enrollments and complexity, many educators have considered it unfortunate that Carnegie provided no ways to group them.

The system Carnegie decided to use is based largely on the work of Stephen G. Katsinas, director of the Education Policy Center at the University of Alabama at Tuscaloosa, and David E. Hardy, director of research at the center.

The system divides institutions based largely on size and location (where not otherwise noted, categories are for public institutions):

Associate Degree Categories in New Carnegie System

McCormick said he thought it was very important for Carnegie to explicitly recognize the range of two-year institutions. "I think symbolically, this says something about the importance of community colleges that we think they are important enough to differentiate."

He said that he thought the revised system could be used to group similar institutions well, but he said that he was disappointed not to find a good way to divide community colleges by their relative emphasis on pre-transfer vs. vocational programs. "I think the lack of data we have there is probably the single biggest gap on available information and we know that community colleges vary on that spectrum," he said.

Sara McPhee, a research associate at the American Association of Community Colleges, said that she applauded the move to having multiple categories for two-year institutions. "Small rural community colleges do end up having different missions and different offerings from large urban community colleges," she said.

McPhee said she was also pleased that Carnegie included in the community college categories institutions that are primarily two-year institutions, but offer a few four-year degree programs. (The cutoff used by Carnegie is 10 percent of degrees -- institutions that exceed that total would go in a bachelor's category.) The issue is an important one for community colleges nationally as the question of offering four-year degrees has become controversial. Those institutions that have done so have generally offered a relatively small number of degrees, so the Carnegie system will not "bump them" from the associate area, McPhee said.

Another change made by Carnegie -- largely for reasons of fairness -- is generating controversy. This change makes it much more difficult for institutions to be listed in one category even though they meet the criteria for another category. In the 2000 classification, there were more than 100 institutions with exemptions -- with the largest number of these institutions being places that "identified strongly as undergraduate liberal arts colleges, but had appreciable graduate education," McCormick said. At the research university level, institutions want to move "up," but for places whose image revolves around being a liberal arts college, moving "up" to a master's category is not seen as a promotion at all.

McCormick said that he understood the colleges' desire to remain identified in one way, but that it creates a problem to have "published criteria" that don't apply to all institutions. He also noted that more liberal arts colleges were in fact adding master's programs and that the value of a classification system depends on it reflecting such a trend.

This year, only about a dozen exemptions are being made -- although lobbying to gain exemptions was going on Friday and will continue (Carnegie typically makes some adjustments after the release of classifications, when institutions present arguments based on such facts as errors in data reviewed). For liberal arts colleges with selected master's programs, McCormick said that the classification system was applied with only a few exceptions, such as when colleges could show that their graduate programs took place at a different campus.

With the stricter enforcement of the rules, institutions such as Bryn Mawr and Smith Colleges, and Wesleyan University -- all of which formerly appeared in baccalaureate categories -- find themselves in master's categories. Middlebury College escaped such a fate on Friday, arguing that its graduate programs are either in the summer or off-site.

Robert Morse, who directs data research at U.S. News, said that the Carnegie revisions are too late to use for the next rankings from his magazine, which will be published in August. He said that there was a "significant likelihood" that the magazine would use the new Carnegie system for future editions, but that no decision could be made until the magazine's staff could study the revisions. Morse said he had heard about the concerns of some liberal arts colleges, and that U.S. News believed the end of Carnegie's exemptions could have a real impact in that category. But Morse said that the colleges had yet to contact the magazine, and he did not know what would be decided if they do.

For those upset about where they are being classified, McCormick noted that one of the ideas behind the system released in November was to ensure that colleges would not need to be identified solely by any one set of criteria. Colleges that have a strong emphasis on undergraduate liberal arts education, but that don't end up in that basic category, can use the other system to group themselves with similar institutions, he noted.

In fact, by giving colleges multiple tools for comparison, Carnegie is "giving up some of our authority in telling people what the official groupings are," he said.

Of course, there will also be colleges thrilled about where they ended up -- and McCormick isn't happy about the way they respond either. Categories are meant to be descriptive -- without a hierarchy among them, he said.

"The most shameless hyping of Carnegie Classifications is when an institution issues a press release saying that we have said that they are in the top 4 percent or something of colleges and universities nationwide," McCormick said. "That's just patently absurd," he said. "It's outrageous spinning and misrepresentation."