Study questions whether marketing materials influence rankings
Marketing materials by law schools -- designed to help them in the rankings -- are widely called "law porn." These materials have been widely criticized for years, yet most law schools mail them out each rankings season, trying to out-boast and out-glossy their competitors.
Leaving aside the wisdom of playing the rankings game, do such efforts succeed in pushing various law schools up a few notches? A new study released on the Social Science Research Network suggests that they don't.
For the study, Larry Cunningham, associate dean for student services and associate professor of legal writing at the law school of St. John's University, in New York, had all of his colleagues at the law school save every piece of marketing they received from any other law school between June and December of 2011. Fall is the high season for this type of marketing because it's when U.S. News & World Report collects data for the "reputational survey" part of its law school rankings, which counts for 25 percent of a school's rank.
Unlike rankings of undergraduate colleges, for which presidents are the sole job title surveyed, law school reputations are judged by U.S. News through surveys of law schools' deans, deans of academic affairs, chairs of faculty appointments, and the most recently tenured faculty members -- meaning that there are many people to target.
At St. John's in the fall of 2011, law school faculty members and administrators received 427 pieces of marketing material -- coming from 125 of the 202 accredited law schools in the United States. The materials were then divided into categories, such as postcards (large or small), letters, brochures and magazines of various types. On average, he found that law schools sent out just over two pieces of marketing material, but some sent quite a bit more. One law school sent 16 pieces of marketing, one sent 10, three sent 9 and five sent 8.
Cunningham then looked at movement in the rankings from the 2011-12 to 2012-13 rankings by U.S. News to see if there were patterns that could be identified. And he could find no statistically significant relationship between law schools sending out materials and going up in the rankings.
For only one type of marketing -- magazines that are about legal topics but that are not alumni magazines -- did Cunningham find any evidence that use may have a positive impact on rankings growth. And he cautioned that he was hesitant to attribute rankings gains there to the magazines. Because these publications are expensive to produce, the use of this type of magazine may reflect law school wealth that could affect all kinds of areas -- such as faculty recruiting, financial aid and admissions efforts -- that could have an impact on a law school's ranking.
Cunningham's findings have generated considerable discussion among law school officials. Some have questioned -- as this blog post does -- whether a single year is a valid way to measure the impact of marketing, given that law school rankings don't change much from year to year.
In an interview, Cunningham said he would like to see a study done over a longer time frame, but he noted that many law schools appear to hope for real gains year to year, and that he doubts they will get them with the marketing they are doing.
Cunningham's study stops short of saying that law schools should stop the marketing -- although he does note many of the choice digs that professors have gotten in at these marketing materials. Among the highlights (besides "law porn"): "beneath the dignity of scholars," "undignified and misleading," "futile" and "crap."
What law schools need to do, Cunningham said, is be honest about their goals for marketing, and then to do a real cost/benefit analysis. "If the goal is really to invite people to a speaker series, that's great, but why are you sending it out after the fact?" he asked.
If, however, the goal is to influence rankings, he suggested that a realistic assessment might lead to different decisions. "I think what law schools should do is pause and give some serious consideration to what are the goals they are trying to achieve, and whether this is likely to accomplish those goals," he said.
"If you are producing this stuff for seeing an immediate jump in rankings, it's not worth it."
Asked if he thought there would be similar findings in looking at materials designed to shift undergraduate rankings, Cunningham said that he has "no reason to expect you would see any other result."