Even before the midnight embargo lifted last night, colleges started sending out announcements about how they fared in this year's annual rankings by U.S. News & World Report. The boasting -- at least sometimes by colleges that criticize the rankings -- is a tradition each August.

We'll let you go to the magazine itself to find out who is No. 1, but can assure you that there are no shockers. What may be more newsworthy is the annual fight over participation in the survey portion -- seen by many as the least valid part of the rankings.

U.S. News said that 48 percent of all institutions responded to the reputation survey that can be filled out by presidents, provosts, admissions deans or others and that counts for the largest portion of formula used in the rankings. That's up two percentage points from last year. Among liberal arts institutions, this year's 46 percent participation was also up two points. In both cases, these upticks still don't make up for a lot of lost ground -- just a few years ago the national participation rate was 67 percent.

As a result, U.S. News can argue that the declines have ended, while its critics respond that the response rate is settling in at less than half -- even as new reports surface that some of those who do fill out the form do so in ways that game the system by ranking competing universities low, or that give the process of filling them out short shrift.

Also of note this year, U.S. News is making some methodology changes in its consideration of testing -- in ways that are frustrating to those who want colleges to question the idea that they need standardized tests. And also Wednesday, an alumni group that advocates for a traditional curriculum released its grades of prominent colleges -- and angered those who don't believe a traditional curriculum is the best approach in 2009.

What the Participation Rates Mean

Colleges have long complained that U.S. News oversimplifies a choice that should be individual and encourages colleges to share values of the magazine (that selectivity in admissions is key, for example) that don't reflect the values of many colleges. In response, the magazine has argued that it promotes accountability -- and many college administrators admit, for example, that they have paid more attention to retention issues since the magazine made that count.

The Education Conservancy, a group promoting the decommercialization of college admissions, has worked with some liberal arts college presidents to have them pledge to boycott the survey portion of the rankings and to pledge not to promote their institutions with their rankings. The thinking has been that if fewer colleges participate, the weaknesses of the peer assessment (and the rankings as a whole) will be more apparent.

Robert Morse, who runs the rankings for the magazine, called the participation rate shift "a move in the right direction" that may suggest that "maybe the boycott is losing steam."

He also said he wasn't worried by the report in Inside Higher Ed Wednesday that noted a number of college presidents and chancellors who either seem to be gaming the system or paying no attention to how their peer surveys are filled out. He noted that the report found only a minority of presidents engage in these practices, and he pointed to safeguards in place at U.S. News, such as tossing the highest and lowest ratings for institutions.

Morse also said that the way presidents engaged in "strategic voting" actually affirmed one of the magazine's arguments to institutions -- that "it's in their best interests to respond, and we're going to do it anyway, so why not participate."

Others see the surveys that have become public as more evidence of gaming the system. James A. Dearden, a professor of economics at Lehigh University, last year was among the authors of a paper called “The University Rankings Game: Modeling the Competition Among University Rankings," in America Statistician. Dearden said that surveys of this type "always create the incentive to give our peers lower ranks," and so the question is whether there are sufficient safeguards against those administrators who "don't do the right thing."

From a game theory perspective, he said, administrators "have no incentive to do the right thing and so many are going to manipulate the surveys to game the system."

Lloyd Thacker, founder of the Education Conservancy, said that the two-point gain wasn't that significant, and that what matters is the fact that the most weight is given to a survey that isn't filled out by even half of the institutions. "It's just as ludicrous as it's ever been, and if the best they can do is two points, they haven't done much."

Methodology and Standardized Testing

U.S. News made two changes in methodology related to standardized testing -- only one of which is mentioned in the main methodology article in the magazine's issue being released today. Until this year, U.S. News has measured standardized test scores by the averages submitted by enrolled students of the most popular test -- the SAT or the ACT -- on a given campus. Morse said that given increased use of the ACT on campuses that have been SAT dominant, the magazine wanted to count all scores, not just those of the most popular test.

The other methodology change may be more controversial. U.S. News has consistently defended the importance of including test scores in its methodology even as more colleges have ended their requirements that applicants submit test scores. And colleges that don't value SAT scores say that the magazine isn't just providing consumer data, as it claims, but is taking a side in an education debate over the role of standardized testing. Others, meanwhile, have criticized the colleges going test-optional by saying that they are trying to game the rankings, since the scores they continue to receive and report will come from higher scorers, not lower scorers.

U.S. News has until this year dealt with the issue by what Morse called "a slight dinging" of the test averages of any college that fails to submit scores for at least 50 percent of those who enroll. This year, U.S. News gave that ding to any college that didn't submit test scores for at least two-thirds of those enrolling. Morse said that this was about fairness to colleges that report scores for all incoming students, and said that the shift did result in "a small number" of additional colleges getting the "dinging."

The shift could be significant because most SAT-optional colleges report that a majority of applicants continue to submit scores even if they don't have to. But raising the bar will bring the U.S. News punishment for test-optional policies to a level that could affect many institutions over time -- or that might make some think twice about going test optional in the first place.

Said Thacker: "Who is U.S. News to tell colleges what's right for them in terms of how they admit kids and holding them hostage based on this? Who says there is better education at colleges that require everyone to submit SAT scores? They are being bullies."

Robert Schaeffer, public education director for the National Center for Fair & Open Testing, a group that criticizes the use of standardized testing, said that "shifting the requirement to 66 percent may hurt some test-optional colleges where 35 percent to 45 percent do not submit, a reasonably common figure. Of course, no one will be able to tell how much impact the so-called 'slight ding' will have because of the secret way U.S. News operates."

Schaeffer added: "This change illustrates two of the many flaws in the U.S. News formula -- the rankings criteria and weights are completely arbitrary, subject to the whim of Bob Morse et al., and the process is generally not transparent." (While U.S. News did not draw attention to this change, Morse provided it when asked for any methodology changes taking place.)

Going Forward

What the future will hold for these issues is unclear. U.S. News has continued -- as it has for years -- to invite colleges to suggest improvements in the process (a move that the magazine's critics view cynically). The magazine has been in discussions, for example, with the National Association for College Admission Counseling about that group creating a committee that could discuss areas of concern with the magazine.

David Hawkins, director of public policy and research at NACAC, said that the magazine first approached the group a few years ago, and that after much discussion the two entities were close to agreeing on the format the body would take. "We want to be sure that this is not perceived as a mechanism through which U.S. News might obtain a NACAC endorsement," he said. "Our goal is to have a way to raise concerns that relate to rankings."

One area Hawkins cited was the relationship between standardized testing policies and the rankings. NACAC released a report last year that -- while not calling on all colleges to abandon testing -- said that they should carefully examine whether they needed testing requirements. The report suggested that many colleges, upon closer examination, would find that they do not have educational reasons to maintain the requirements.

Hawkins said that U.S. News did not ask his opinion about the methodology change that will hurt some test-optional colleges.

Speaking generally, he said, "our members want to know U.S. News's knowledge of the ramifications of their methodology and their sense of responsibility for having created certain consequences."

As U.S. News moves on its fronts, so too does the Education Conservancy. Last year it unveiled a prototype for something of an anti-rankings Web site, where colleges would post information that relates to education styles and values in a highly individualized way, without rankings. Thacker said that the organization is in discussions with several entities that he declined to name that would be the types that could support and possibly host the venture.

And for Those Who Like Requirements...

While U.S. News does not critique curricular requirements, the American Council of Alumni and Trustees has no such hesitation. The organization issues regular reports that lament the lack of specific survey requirements in selected disciplines. The ACT on Wednesday unveiled "What Will They Learn?," a Web site billed as "a guide to what college rankings don't tell you. The site assigns colleges letter grades based on whether they require survey courses in composition, literature, foreign language, U.S. history, economics, mathematics and science.

The Web site explains its mission this way: "This Web site asks a simple question about today's students: What will they learn? Specifically, will they gain the knowledge and skills they need to compete in the global marketplace, lead our nation thoughtfully, and be lifelong learners? Many college guides and ranking systems measure institutions' prestige and reputation, but no guide has looked at what students are actually required to learn. That's what we are doing here."

ACTA ends up giving A grades to only seven colleges: Baylor University, City University of New York's Brooklyn College and Hunter College, Texas A&M University, the U.S. Military Academy, the University of Arkansas at Fayetteville and the University of Texas at Austin.

Many institutions known for academic rigor fare poorly in the study. Swarthmore College, to name but one, gets an F, even though Swarthmore has some of the requirements that the council endorses. But the group has determined, for example, that composition courses that also relate to academic subjects other than writing don't count, so Swarthmore gets no credit there. Dartmouth earns a C overall and gets credit for its composition requirement, but its literature requirement is rejected because "it may be fulfilled with niche courses such as 'Bob Dylan' or 'The Graphic Novel' -- a course about comic books."

ACTA is correct that Dartmouth doesn't have a survey requirement in literature, but those reading its brief description might not know that the long list of courses that do meet its requirements extends beyond cultural studies and include courses in Chaucer, the 19th century English novel, American poetry, Shakespeare and Milton, among many others. A college spokesman said: "Dartmouth does have a literature requirement -- one course. There are a number of ways to satisfy this requirement and many of our students take more courses in literature than just the one course requirement over their four years at Dartmouth."

Anne Neal, president of the alumni council, stood by her view that what should count is only "a comprehensive literature survey that can't be avoided." Neal said that most colleges boast about their general education programs and her organization was holding them accountable.

But does she really believe that Harvard University is as bad as the D grade it received? Neal is writing tuition checks for her daughter there, and she didn't indicate plan to suggest a quick transfer to an A-rated institution. "The fact that a well known school does poorly in general education is not a reflection on other aspects of that school," she said.

Neal also said that many universities that do poorly in her grading system in fact offer good survey courses for those who want to take them.

"The mere fact that the university does not have a structured curriculum does not mean that the student is left helpless. Students can do it on their own," she said. "But no 18 year old should have to create from scratch a coherent general education."

She added that at some of the universities that are well regarded by the public, but that do poorly in her grades, "there are top ranked students who come in at the top of their game and they educate each other, but we are asking about what is the value of the school once they get there."

Some experts on general education were disturbed by the ACTA analysis of curricular needs through a list of survey requirements.

Carol Geary Schneider, president of the Association of American Colleges and Universities, said that two of the most important curricular trends in recent years have been to connect the curriculum to the real world and to connect various subjects that students study. The ACTA survey appears "indifferent," she said, to the idea that "what students most need to take with them to college is the ability to integrate and apply their knowledge and skills in new settings. They need to have they been involved in high impact practices," she said.

The ACTA standard that composition can be taught only outside of disciplines ignores decades of work having professors from many disciplines stress writing, so that writing is taught rigorously and throughout a student's college years, not just as a single course, she said. And while she said she applauded foreign language instruction, she said that it was at least as important to be teaching world history and cultures.

"It seems nice to say that you can get everyone into a small set of courses, but that is not today's reality," she said. Schneider applauded the ACTA for "having the courage of their outdated commitments."