You have /5 articles left.
Sign up for a free account or log in.
iStock
Survey results released by the Brookings Institution last week attracted considerable attention with their suggestions that many college students don't understand the First Amendment and that a significant minority of students (19 percent of all students and 30 percent of male students) said it would be acceptable for a student group to use violence to prevent a controversial speaker from speaking.
The findings, particularly on tolerance for violence, were widely discussed, including in a brief article in Inside Higher Ed. A column in The Washington Post called the survey results "chilling." On social media, many people expressed sadness, with one tweet suggesting that "James Madison weeps." Conservative websites ran articles with headlines like “Illiberal Liberals on Campus -- It’s Worse Than You Thought” and “Antifa and Its Left-Wing Allies Are Winning the Battle for Free Speech on College Campuses.”
But are the findings valid?
The Guardian on Saturday drew attention to a fact that was not noted in the Brookings write-up of the survey. The study was an "opt-in" survey and was not in fact a representative sample of college students. The study itself, available here, says that the survey was online and doesn't say that it was opt-in. And language in the survey write-up suggests that it was based on a representative sample. The Brookings article says, "To the extent that the demographics of the survey respondents (after weighting for gender) are probabilistically representative of the broader U.S. college undergraduate population, it is possible to estimate the margin of error in the tables above. For a confidence level of 95 percent, the margin of error is between approximately 2 percent and 6 percent -- the margin of error is smaller for the categories with larger numbers of respondents." Those not paying close attention to the "to the extent …" part of the sentence might think that a real margin of error was claimed.
In the Guardian article, experts on polling were quoted about the combination of not revealing that the survey was opt-in with the language on margin of error. Their conclusions were quite negative, with phrases such as "junk science" and "really not appropriate" and charging the author with "trying to overstate the quality of his survey."
Then there is the question of timing. The Brookings write-up does clearly note that the study was conducted between Aug. 17 and Aug. 31. But no mention is made that this means the survey opened just a few days after white nationalists rallied at the University of Virginia, with many shouting a Nazi chant; a counterprotester was killed -- apparently by one of the white nationalists who drove his car into a group -- the next day. The events in Charlottesville, Va., stunned and angered students (and others) nationwide.
The scholar who produced the survey and the Brookings article is John Villasenor, a nonresident senior fellow at Brookings and a professor of electrical engineering, public policy, and management, and a visiting professor of law, at the University of California, Los Angeles.
In an interview with Inside Higher Ed, he acknowledged that the survey was opt-in and that there could be no assurance that the sample was representative. He said that the makeup of survey participants in various ways (party identification, gender, attendance at various kinds of colleges) pointed to what he said was the likely reliability of his data. But he said that nothing in his article was untrue, even if he didn't include that the survey was opt-in. "I was very careful with language," he said.
He also acknowledged that many experts in the field and journalists tend to pay little attention to opt-in surveys.
Asked if he regretted not including that detail, he said he always engages in "Monday morning quarterbacking" after he publishes articles, but said he wasn't willing to say he had made a mistake.
Villasenor said he plans to submit a more scholarly version of his article to a law review, and that he did plan to note the opt-in nature of the results in that piece.
The Brookings study is not the first to find that many college students today favor limits on free speech, especially when it comes to what students view as "hate speech" that demeans groups of people. Gallup and the Knight Foundation released such a study last year, although that study did not have questions about student tolerance of violence as a response to controversial speakers. The Gallup/Knight project includes a methodology that explains how students were interviewed and why those in that study make up a representative sample.
As to whether Charlottesville might also raise questions about the findings, Villasenor said that could be possible. But he said that it would be "speculative" to conclude that Charlottesville influenced the results, so he opted not to note the timing issue.
