Alexander C. McCormick writes about the problem with 'U.S. News' rankings that nobody talks about.
A lot of ink has been spilled over the years about “America’s Best Colleges,” the annual rankings published by U.S. News & World Report. We read periodic complaints that the rankings improperly rely on inputs, resources, and a reputation survey of questionable validity. The rankings are in the news again, with fresh complaints about punitive actions against institutions that withhold data, and a fresh call for collective acts of resistance against the rankings juggernaut.
Surprisingly, a fundamental aspect of the rankings has gone relatively unquestioned -- the composition of the comparison groups. Because the groups are derived from the Carnegie Foundation for the Advancement of Teaching’s classification of colleges and universities, and because I direct that work, I have a special interest in the question of who gets compared to whom. It turns out to be a questionable use of the classifications that can have significant consequences.
The U.S. News rankings framework asserts that there are two kinds of higher education institutions -- universities and colleges -- and they compete in national or regional markets. Consequently, there is not one set of rankings, but ten: national universities and (national) liberal arts colleges, four regionally differentiated sets of master’s universities, and four regional sets of comprehensive colleges. To a considerable extent this reflects the reality of the higher education admissions market. It also makes good business sense: multiple rankings mean multiple sets of “top” performers who will be happy with the results and proclaim their distinction, and who will be eligible to display the America’s Best Colleges award badge on their Web sites and in their promotional materials.
But it leads to big definitional challenges. What’s a university? What’s a college? What’s national and what’s regional? There is no consensus, to be sure. Name alone is not sufficient for distinguishing universities and colleges: Boston College and the College of William and Mary, with their extensive graduate offerings and research orientation, are clearly universities, while exclusively-undergraduate Illinois Wesleyan University and McMurry University are clearly colleges in the conventional sense of the term. Considering the presence or absence of graduate degrees is also unsatisfactory: most would not list Bryn Mawr, Lewis & Clark, Middlebury, or Williams Colleges among universities, graduate programs notwithstanding. Defining national and regional is even trickier.
So what do they do? Easy -- they rely on the Carnegie Classification! (Now that there are several classifications, I should be specific: they use the basic classification, meaning the latest version of Carnegie’s “legacy” classification.) Any institution classified among doctorate-granting universities is deemed to be a national university, and all baccalaureate colleges -- arts and sciences are (national) liberal arts colleges. All master’s universities and the remaining baccalaureate colleges are ranked by region.
It’s reasonable to use the Carnegie Classification to define universities and colleges, though there are cases where the distinction is far from clear-cut. But there’s no basis for inferring national versus regional focus, because it’s not a factor in the classification criteria. So it should come as no surprise that the national and regional lists contain a great many inconsistencies and bizarre placements.
Let’s consider some examples. I’ll take a simple, straightforward approach to defining regional and national, using freely available data on the geographic origin of entering students. Among the roughly 270 institutions that U.S. News will label national universities in next year’s rankings (because they award at least 20 doctoral degrees annually), 82 draw students from fewer than 30 states or territories (for example, Cleveland State University, Illinois State University, University of La Verne, and University of Massachusetts-Lowell -- fine institutions, but undeniably regional in focus). By contrast, among about 660 universities that will be ranked regionally (because they don’t award doctorates, or award fewer than 20 per year), more than 100 draw from at least 30 states or territories (including Hampton University, Ithaca College, James Madison University, and Villanova University).
It’s even worse with respect to the colleges. Of almost 300 that will be ranked nationally because a majority of their graduates major in the arts and sciences, more than 120 enroll students from fewer than 20 states or territories (e.g., Ave Maria College, Burlington College, Penn State Abington, and the University of Wisconsin-Parkside), while about 100 of the 460-odd regional colleges have greater geographic diversity.
If we substitute a measure of admissions selectivity as the metric for regional or national focus, we get similarly bizarre contradictions: national universities and colleges with open or near-open admissions, and regional ones that are quite selective.
This isn’t just a technical issue. Ask any president of a regionally- or locally-focused college that -- due to students’ majors -- gets ranked against wealthier, far more selective colleges that truly compete nationally. If she calls U.S. News to request a correction, the answer will be, in essence: “Our comparison groups are based on the Carnegie Classification, so if you don’t like it, call them.” So she calls the Carnegie Foundation (read: me) to explain that hers is a regional institution. But because that’s not the basis for classification, and because misuse of classifications should not drive classification decisions, she doesn’t get satisfaction from us, either. She -- and her college -- are caught in the middle: stuck in the wrong comparison group, with potentially serious consequences for recruitment and fund raising as well as board and alumni relations, and no recourse.
If U.S. News thinks the national-regional distinction is important, they should define it. It’s preposterous to use degree offerings (for universities) and the proportion of majors in arts and sciences (for colleges). I told them so last November, and demonstrated how they could do it better. They heard me out, but ultimately decided that consistency (or convenience) is more important than accuracy and rationality. By continuing to rely on the Carnegie Classification, they avoid the tough job of defining their terms -- and in so doing, they can dodge a whole category of complaints by shifting responsibility to a third party.
If they aren’t willing to do it right, they should either stop making the distinction or they should adopt a much more flexible stance for institutions that legitimately challenge their comparison group placement.
Bio
Alexander C. McCormick is a senior scholar at the Carnegie Foundation for the Advancement of Teaching.
Read more by
Inside Higher Ed Careers
Browse Faculty Jobs
Browse Administrative Jobs
Browse Executive Administration Jobs
Topics
College Pages
Popular Right Now
College of New Rochelle announces that it will likely close this year
Scientists rally around Vanderbilt professor whose tenure bid appeared to hit roadblocks as Me Too a
How to write an effective diversity statement (essay)
Almost a week of no internet at Amherst College
Moves to kill UT Knoxville's Sex Week met with questions
New initiative to help single moms succeed in community colleges
Georgia Tech's bid to be the leader in university innovation (opinion)
Essay on writing academic book reviews
Essay about how white male students dominate discussions sets off debate at Dickinson and beyond
Opinions on Inside Higher Ed
Inside Higher Ed’s Blog U
Sign Up / Sign In
With your existing account from...
{* loginWidget *}With a traditional account...
{* #signInForm *} {* signInEmailAddress *} {* currentPassword *} {* /signInForm *}Newsletter enrollment
Existing account found
We have found an existing account for the email address.
Please sign in to update your newsletter preferences.
{* /signInForm *}Newsletter opt-out
Edit your newsletter preferences
Account Info
Sign In
Welcome back, {* welcomeName *}!
{* loginWidget *}Sign In
Welcome Back
Account Deactivated
Your account has been deactivated.
Account Reactivation Failed
Sorry, we could not verify that email address.
Email Verification Required
You must verify your email address before signing in. Check your email for your verification email, or enter your email address in the form below to resend the email.
{* #resendVerificationForm *} {* signInEmailAddress *}Almost Done
Please confirm the information below before signing in.
{* #socialRegistrationForm *} {* firstName *} {* lastName *} {* displayName *} {* emailAddress *} {* optInIHE *} {* optIn3rdParty *} {* agreeToTerms *}Almost Done
Please confirm the information below before signing in. Already have an account? Sign In.
{* #registrationForm *} {* firstName *} {* lastName *} {* displayName *} {* emailAddress *} {* newPassword *} {* newPasswordConfirm *} {* optInIHE *} {* optIn3rdParty *} {* agreeToTerms *}Thank You for Registering
We have sent a confirmation email to {* emailAddressData *}. Please check your email and click on the link to verify your email address.
Create New Password
We'll send you a link to create a new password.
{* #forgotPasswordForm *} {* signInEmailAddress *}Create New Password
We've sent an email with instructions to create a new password. Your existing password has not been changed.
Create New Password
This email address was registered with a social account. We've sent an email with instructions to create a new password, which can be used in addition to your linked social providers.
{| current_emailAddress |}
{| foundExistingAccountText |} {| current_emailAddress |}.
{| existing_displayName |} - {| existing_provider |} : {| existing_siteName |} {| existing_createdDate |}
{| existing_provider_emailAddress |}
Sign In to Complete Account Merge
Resend Verification Email
Sorry, we could not verify that email address. Enter your email below, and we'll send you another email.
{* #resendVerificationForm *} {* signInEmailAddress *}Change Password
Create New Password
We didn't recognize that password reset code. Enter your email below, and we'll send you another email.
{* #resetPasswordForm *} {* signInEmailAddress *}Create New Password
We've sent you an email with instructions to create a new password. Your existing password has not been changed.
Edit Your Account
Profile Photo
Linked Accounts
Link Your Accounts
This allows you to sign in to your account using that provider in the future.
Password
Deactivate Account
Change Password
Deactivate Your Account
Are you sure you want to deactivate your account? You will no longer have access to your profile.
{* deactivateAccountForm *} {* /deactivateAccountForm *}
Expand commentsHide comments — Join the conversation!