News, Views and Careers for All of Higher Education
Dec. 28, 2005
A few years ago, Cary Nelson submitted an essay to PMLA — the flagship journal of the Modern Language Association — the dealt in part with a little-known strategy used by the Soviets in World War II: sending shells at German soldiers that contained poetry designed to encourage them to surrender.
The article explored more broadly the use of poetry in the war, but one of his outside reviewers urged rejection of the article, saying that he thought it was a spoof. While Nelson got to see the review (as part of the process of getting rejected by PMLA), he doesn’t know who wrote it and he couldn’t contact this person to show that he wasn’t writing a spoof.
Nelson, the Jubilee Professor of Liberal Arts and Sciences and professor of English at the University of Illinois at Urbana-Champaign, used the example in a presentation on anonymity Tuesday night at the annual meeting of the MLA. Anonymous reviews, he said, are producing boring journals, discouraging risky work, and making the tenure process unfair and unworkable. Others who spoke at the meeting defended anonymous reviews as a good thing or a necessary evil.
For Nelson, the story of his article has a happy ending. American Literary History published it and it is now the first chapter of a book in progress. But Nelson said that he is convinced that anonymous reviews are damaging the field.
Evaluating work without identifying who the author is promotes “agency without identity,” Nelson said, and encourages the idea of literary scholarship as a profession that “assumes we will take no risk in our actions.”
Knowing the authors or tenure candidates you are reviewing may expose bias (and encourage people where appropriate to recuse themselves), while keeping them secret allows for “bad faith readings.” He cited one tenure case where an outside reviewer did not reveal that he had made a sexual overture to the candidate he was reviewing (and been rebuffed). This outside reviewer apparently equated the candidate’s rejection of him as evidence of bad taste generally and wrote a highly critical (and unfair) review, Nelson said. While this conflict of interest was uncovered, “secret assassination” is common in tenure reviews, and candidates are helpless, he said.
In scholarship, Nelson argued, the anonymous system encourages authors to try to appeal to what they think will work to get into a given journal, and discourages a scholar from taking a bold risk — or a reviewer from applauding such a risk.
Michael T. Thurston, an associate professor of English at Smith College, said he organized the discussion after an informal gripe session at a previous MLA where everyone in the group talked about how they had been victims of the “cloak of anonymity.”
But not all panelists agreed that anonymity was all bad.
John N. Duvall is a Purdue University professor who is editor of Modern Fiction Studies, which like PMLA uses a “double blind” system: Reviewers don’t know whose work they are reading and authors don’t know who is evaluating their work.
Duvall said he believes that there is a direct relationship between that policy and the diversity of institutions that have authors in the journal. Over the last four issues, he said, the journal has published four pieces by scholars at top research universities in literary studies: Cornell University (twice), the University of California at Irvine and Yale University. Far more pieces in the journal come from places like the Universities of Kansas and Kentucky, the State University of New York at Old Westbury, Roosevelt University and Texas A&M University at Corpus Christi. And of the four pieces from top universities, Duvall said, two were from graduate students.
A generation or two ago, people might have assumed that all the best ideas came from professors at a small set of institutions. But Duvall said that the job market being what it is, there are lots of highly talented people doing very important work all over the place. “There is no correspondence between the quality of work and where one lands a tenure track job,” he said.
While Duvall backed anonymous reviewing, he also talked about problems with book publishing, where the norm for many university presses is to keep the reviewers secret, but not the authors. There are too many cases, Duvall said, of reviewers writing negative evaluations to “settle a grudge.” He said that this is a particular problem when you edit a collection of pieces by different authors, meaning that someone in the group surely will have offended just about any reviewer.
Duvall said that he ran into this when he was the editor of a collection, and one anonymous reviewer blasted the collection, but focused all of his anger on one essay. Duvall figured out (he thinks) who the reviewer was, and it turned out that the person and the essay’s author had been on opposite sides of an ugly tenure lawsuit. When Duvall raised this with the publisher, the officials there would not confirm his suspicions, but did agree to send the collection out to two more reviewers, and it has been smooth sailing for the project ever since.
Jeffrey Di Leo, editor of the journal Symploke, said that he thinks it is “a little bit deceptive” to suggest that double-blind reviews are really blind to all. Editors pick reviewers, he said, and they pick them for reasons. “We can choose readers for acceptance and we can choose readers for rejection,” he said.
Di Leo, interim dean of arts and sciences at the University of Houston-Victoria, also said that picking reviewers with biases in mind is not necessarily a bad thing. He spoke of his concerns about the “cults” of supporters of various literary theorists. When dealing with a submission about such a figure’s ideas, Di Leo said, he’ll intentionally seek someone from “inside the cult” and someone from outside, to have both perspectives.
Another question panelists considered was why so much power in academe goes to anonymous reviewers — with so little questioning of the system.
Don Howard Bialostocky, a professor of English at the University of Pittsburgh, compared the use of anonymity in academic careers to the way society evaluates judges. Noting the scrutiny of nominees to the Supreme Court, Bialostocky asked the audience to imagine a world in which judicial opinions were unsigned, in which Supreme Court justices asked questions from behind a screen to protect their identities, and in which those confirming or rejecting a nominee did so in complete secrecy. Academics would be outraged, he suggested, and yet many of them unquestioningly will revise a paper or book to satisfy an anonymous reviewer whose credentials they know nothing about.
Where you stand on anonymity may depend in part on the impact you think it has had on your career and publications you care about. This being an MLA meeting there was much bashing of the PMLA. Nelson said that when it shifted to anonymous reviewing to “level the playing field,” it became “a level journal, with few of the highs and lows that are in the best journals.” Others (who didn’t necessarily agree with Nelson’s call to end anonymity) agreed, calling the journal “horribly boring,” among other things.
But just down the hall, at a session to welcome graduate students to the MLA, the younger members of the profession were being told that the PMLA (and the prestige that goes to those who publish there) was open to them. During a recent five-year period, as many graduate students as full professors had their work published there (and assistant professors beat both of those categories).
In an interview, Rosemary G. Feal, executive director of the MLA, said that she sees those statistics and is pleased that the journal’s reviewers “don’t feel any obligation to publish the big names,” and that she can truly tell scholars who aren’t big names or even medium names that their work will be judged “on the material, not the person.” (Feal also noted that many special forums in the PMLA involve top scholars being solicited to write, so a mix of criteria are in use. “Dull is in the eye of the beholder,” she said.)
Back at the session on anonymity, one other issue in play is the fear factor. Duvall of Modern Fiction Studies said that he couldn’t get the best reviewers if they believed that their names would become public. In some areas, he said, the best reviewer of a piece is a junior scholar in a field, who may be later reviewed for tenure by the author of a piece under consideration.
Others said it was time for people to show more courage about being open about criticism. Di Leo, whose background is in both English and philosophy, said that in the latter field, “you just dis people,” and there aren’t huge fallouts. In English, there is “less of that sharpness,” so people who criticize feel that they must be protected.
Likewise, Di Leo said that people should complain not about authors of critical reviews, but about the editors who blindly follow the advice they receive. He said that there are cases where he receives two negative reviews for an article, but publishes it anyway. Editors need to remember that this is an option.
Michael Bérubé, a professor at Pennsylvania State University, was in the audience at the session and said that a few years ago at the MLA, he ran into the author of a book that he had harshly criticized in a public book review. The two are unlikely to become friends, Bérubé said, and the moment was “awkward,” but he said that’s all it was, and that scholars with tenure need to be less nervous about saying things — with their names attached — that might disagree with other scholars.
Nelson said that ultimately, it’s a matter of honesty. He said that on the day of his tenure vote in his department, he ran into a senior faculty member in the hallway, who “congratulated me, patted me on the back, and then went to the meeting to try to get me fired.”
While faculty members almost always have the right to keep their tenure votes secret, Nelson said that they don’t have to, and that he always tells a department member when he plans to vote against them.
Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.
Advertisement
To CJ
The cost to reviewers is already pretty high. The benefits are close to zero. What happens if the number of reviewers greatly diminishes.
Max, at 1:04 pm EST on December 28, 2005
Even the US Supreme Court issues per curiam decisions. When I was in graduate school, I was taught that Germany’s interest in protecting the notion of the certainty of the law was reflected in a policy of only announcing the decison of the court and never announcing the vote or concurring and/or dissenting opinions. I don’t know if this remains the process post-unification.
Kenneth Sherrill, at 3:30 pm EST on December 28, 2005
[Gentle [Men and Ladies]]: I cannot imagine a case where I would read, think about, and write a review of a paper submitted to a journal I read for in which I would not simply affix my name to the review for the author to see. I don’t need to know who the author is, but I feel the author should know who reviewed the paper. The important discussions then can take place between the judicious editor of the journal and the author — the editor arbitrating between his reviewers and the authors. The extremely important arbiter here is the editor of the journal, who in many cases can serve to counteract any shenanigans by authors OR by reviewers. The responsible editor always has the right (and I’d say the duty) to send the paper out to other reviewers, in addition to the two or three original reviewers. In a sense, the authors of submitted works are a more vulnerable powwer position, and for this reason, I feel that reviewers should reveal their names. Reviewers can be wrong in their interpretation and/or understanding of a paper, and for this reason, multiple reviewers are needed and a truly responsible editor is also an obvious sine qua non. Scientists who overly personalize their science and the reactions to their science by others in the field should be banned from the earth. Of course, then, there may be no more science because it is usually the case that science is done by humans, and humans are often very frail entities.
Hugh Buckingham, Professor at Louisiana State University, at 3:37 pm EST on December 28, 2005
Cary Nelson is right: secrecy in manuscript evaluation and promotion case does permit abuse of at least two kinds—salting publication and promotion dossiers with people who will produce the result the chair or editor wants, or poisoning them with hostile reports by error or by choice.
But I’m not convinced by Nelson’s anecdotes that those abuses are at all common, or that they are the reason for dullness in academic writing. Neither am I convinced that going to full transparency would make things any better. Rather, going to full transparently would, in all likelihood substitute a new set of (probably less managable) problems.
I’ll be anecdotal, too. Several times when I was editor of Journal of American Folklore I received angry letters from authors of rejected manuscripts demanding the names of the reviewers who’d written negative evaluations of their work. I’d write back, “Tell me what you disagree with in the evaluation?” Only rarely did I get anything specific; most often it was on the order of, “That reader must be biased else why would he or she say my manuscript was not worthy of publication in JAF?” What might they have done with the names, had I supplied them? Commenced an interesting correspondence with the reviewer on the key issues of mutual interest? Or would they have waited for payback when that person came through with a proposal on an NEH panel or an subsequent editor sent the rejectee a manuscript by the rejector? How many people, however well-intentioned, are able to look at someone whose evaluation cost them a major publication or promotion and think, “What an objective and fair scholar that person is?”
The dangers are as great for senior as for junior people. How many authors of rejected article or book manuscripts think, “They’re right, I did not do a good enough job,” and how many think, “What idiots they are” or “I knew that sonofabitch hated me"?
It’s not reviewers who accept or reject manuscripts; editors do that. It’s not letter of recommendation writers who vote on tenure; faculty members do that. Maybe part of the solution to the problem Nelson raises is to push department chairs to look carefully into motives of letters that seem skewed, and for department members to look carefully at the jobs their chairs are doing in assembling those dossiers. Maybe members of professional organizations should insist on having editors capable of picking readers they or their editorial boards can be confident will review fairly, followed by an opportunity for rejected scholars to challenge objections for cause.
But reducing the pool of competent reviewers who can safely take part in evaluations —which full transparency surely will do—serves none of us well, with the possible exception of incompetent scholars who will be more likely to get published or promoted because the people most knowledgable about their failures will be far more likely to opt out of the conversation.
Bruce Jackson, SUNY Distinguished Professor at University at Buffalo, at 4:18 pm EST on December 28, 2005
To Max: If reviews are credited and published alongside the articles, benefits accrue to reviewers as well: it’s another opportunity to publish. Competent, insightful, constructive reviewers will receive recognition and credit; bad ones will be remembered as well.
More generally:I’m both amused and saddened that both sides in this discussion (myself included) invoke the emotional immaturity, insecurities, and petty retributiveness of our colleagues as reasons pro *and* con. I can see no way to know for certain whether openness in peer reviews would be more susceptible to abuse than secrecy, but there are some empirical clues laying about if we want to examine them.
For instance, every push toward more openness and transparency in higher ed has been opposed with variants of these same arguments, yet as far as I can see none of the successful initiatives has destroyed the fabric of scholarship. As just one example, my current department believes that graduate students should not be allowed to serve on search committees because it would prevent faculty from speaking honestly — but I trained in a department where students routinely served, served a couple of times myself, and never ran into (nor heard of) a committee that had such problems. In fact, the quality of hiring deliberation at my graduate department was intellectually superior to what I’ve witnessed at my present home (which is at least equally prestigious). At best, the presence of students on the committees brought a freshness and energy that they might otherwise have lacked; at worst, their presence curbed pettiness and encouraged faculty to justify their positions more thoroughly, lest they appear arbitrary or foolish to their students. I imagine faculty who wanted to backstab still found ways to do so when the students weren’t around, but I believe the increased openness of the process encouraged better behavior — and better results — all around.
Finally, there’s an issue of moral leadership: what kind of example do we set for students and society, and what kind of intellectual and moral authority can we claim, when even our most trivial deliberations continue to be cloaked in principles of guild secrecy that the rest of our society has largely abandoned? Employees in every major American corporation and even government agencies now get regular, face-to-face performance evaluations: it’s becoming a bit embarrassing to explain that we scholars are not mature enough to handle the same evaluative pressures as, say, greeters at the local discount store.
CJ, at 4:28 am EST on December 29, 2005
Blessings on PMLA for its policy of anonymous submissions: the queen & the beggar have the same chance.
Lucy McDiarmid, Cullman Center for Scholars & Writers, New York Public Library, at 2:36 pm EST on January 2, 2006
It’s hard to care about submission policies for the _PMLA_, which is in no way a respected journal. In fact, when my dissertation adviser retired, he wanted to donate his collected journals to libraries in countries less able to acquire scholarly materials. These libraries snapped everything up with one notable exception: No one (again — NO ONE) wanted the _PMLA_s. They were all discarded.
So whatever “prestige” some may believe attaches to the _PMLA_, you literally cannot give the damned things away, even to the most needy. Enough said.
Bad English, at 11:52 am EST on December 31, 2005
I found out about this piece through Buffalo Report, published online by Bruce Jackson, whom I have known since I was his student back in the 70s. I sent him an email on this piece. Here’s a slightly edited version of it:
I read the Inside Higher Ed piece on blind review, and your reply. I can’t say that I’ve got a strong opinion one way or the other. But I don’t think that open reviewing will have any marvelous effect, either.
I’ve given a lot of thought to the problem of “opening up” the academic world to innovation. The problem is that any workable solution has to be an institutional one that is workable by pretty much the same bunch of folks who run things now. And that’s tough. It’s all well and good so say “get rid of the idiots and the dullards.” But just who’s going to decide who those idiots are?
The problem with fostering innovation is that innovation is easily recognized in retrospect, but not in prospect. In prospect the innovators are mixed in with the cranks, crackpots, and merely brilliant exploiters of the status quo. And they’re all sure their stuff is the wave of the future.
What’s important is that all ideas have a chance. The web makes publication so cheap that that’s now possible. Of course, most of the stuff published on the web is crap, but it’s now widely available crap. Now it is possible, at least in principle, for a piece of crap to be discovered and refined into brilliance. And I think that in time institutions will evolve so as to allow things to sort them out.
During the summer I attended the Contra-Theory-fest held at The Valve and thought that was good stuff. I think some real intellectual work got accomplished over two weeks or so, and the record is there for all to see. In mid-January The Valve is going to host a discussion of Franco Moretti’s little book, *Graphs, Maps, Trees: Abstract Models for a Literary History.* That should be interesting. It’s a new mode of intellectual discourse.
With things like that happening, I think dickering around over getting reviewers to reveal their names is a waste of time. Given that there’s no reason to think the upside will outweigh the downside, I don’t see this as a hassle worth anyone’s time. Hmmm, looks like I’ve arrived at an opinion on the issue, doesn’t it? More generally, why fight old battles when there’s some new and more interesting ones around?
Bill Benzon, at 11:54 am EST on December 31, 2005
This thread reminds me of an incident years ago, when my husband first submitted his dissertation for book publication. One publisher turned it down on the basis of a rejection recommendation by an anonymous reviewer. My husband works in a small field, and he knew who this reviewer was. The recommendation for rejection was based not on the quality of the work but on a deep ideological divide between himself and this reviewer (one’s a socialist, one’s not). To add insult to injury, the same reviewer had the gall to CITE my husband’s dissertation a year later in his own work. Apparently, the reviewer also had a conflict of interest since he was in the midst of publishing a similar piece of work.
The book was successfully published last year, I’m glad to say.
Yet, despite this story, I am all for anonymous reviews. Yes, it’s harder to hold reviewers accountable. But I believe that two other steps in the process cover for this weakness: the use of three or more referees, to help isolate the out-of-line reviews that sank my husband’s submission, and a vigilant editor who can make the borderline decisions.
I am much more alarmed by the reliance by many editors on two referees or less, and by the sometimes astonishing workload of our editors, than I am by anonymous reviews.
Hoosier Prof, at 2:13 pm EST on January 5, 2006
So, who is this person ‘Student’, who keeps commenting on Inside Higher Ed pieces? I had originally thought s/he was a bamboozled sophommore, but how s/he claims to have had a dissertation advisor ... apparently in English.
At one point, I wondered whether ‘Student’ were a Horowitz (or similar) plant, since s/he comments on so many of these articles, and often seems to say more or less the same thing (whatever the topic of the article).
Anyway, in case Student is a real person trying to figure out what is going on: old collections of PMLA are hard to ‘get rid of’ because libraries often have them already, or have access to them via online databases. PMLA is classic, and standard; it’s not a rare or ‘niche’ publication.
When you donate something to a library, the acquisition isn’t free to them: they still bear the cost of processing, cataloguing, storage, and so on. It takes a well funded library to be able to afford these costs for a _second_ copy of something.
Leslie Bary, at 9:40 pm EST on April 1, 2006
Advertisement
or search for jobs directly.
About The American University in Cairo: Founded in 1919, AUC’s campus has moved to its new, state-of-the-art campus in New ... see job
Posting Description: The American Indian and Alaska Native Programs (AIANP) of the University of Colorado ... see job
Position Summary: The Sawyer Business School at Suffolk University seeks faculty candidates who can deliver ... see job
Join one of the finest regional universities in the nation. James Madison University, home to 18,000 + students, welcomes you ... see job
Essential Duties of the Position: The certified medical assistant’s major responsibilities will be assisting the physicians , ... see job
The University of Minnesota is a premier employer and a talent magnet attracting leading faculty and staff from around the ... see job
A career at Johnson County Community College is more than a job. We believe it’s important to invest in our employees and ... see job
Posting Description: The CSDMS Integration Facility seeks a software engineer with at least 3 years of ... see job
About The American University in Cairo: Founded in 1919, AUC’s campus located in Cairo, Egypt, has moved to a new, ... see job
Salem State College is an equal opportunity / affirmative action employer. Persons of color, women and persons with ... see job
Anonymous course evaluations
The place where current anonymity policies really bite is when it comes to course evaluations. Students are irresponsible because there’s no accountability, because they don’t realize the impact their reviews could have on untenured faculty or, less often, because they do know and are out to get them, because they feel they’re largely powerless and use these evaluations as an opportunity to assert themselves or whatever.
Ideally the policy across the board should be that the individual being evaluated should be anonymous but the evaluator should be known though obviously that isn’t always feasible and, when it comes to, e.g. reviews for publication it’s not a one-shot game: you may end up reviewing, or otherwise evaluating, someone who reviewed you.
But when it comes to course evaluations, where a student is negative on an instructor and so would want to be anonymous, it generally is a one-shot deal: if a students who don’t like the instructor won’t take another course with him. The whole bit of anonymous course evaluations also promotes the idea that students are both victims and consumers and faculty are vindictive, petty tyrants who will go after students that write negative evaluations.
H. E. Baber, Professor at University of San Diego, at 7:15 am EST on December 28, 2005