PubPeer Pressure

Restaurants, movies, plumbers -- the internet can be used to crowdsource reviews of just about anything. So why not scientific research? That’s the idea behind PubPeer, an online platform to discuss research that’s already been published in other journals -- kind of like a Yelp for Ph.D.s. It’s part of a growing post-publication peer review movement that seeks to democratize the traditional peer review process, which many scientists say is old-fashioned and doesn’t always ensure quality. Some research suggests the same.

Post-publication peer review sites also hope to make correcting research errors more simple and immediate; instead of commenting on the initial article in the specific journal in which it appeared -- if commenting is enabled, that is -- peers can comment on the article in one clearinghouse for maximum visibility. At the same time, post-publication peer review sites generally have terms of use that require professional discourse. To that end, some require users to post comments under their real names.

PubPeer, however, revels in anonymity. Its creators have identified themselves only as “a diverse team of early-stage scientists in collaboration with programmers who have collectively decided to remain anonymous,” in order to avoid “personalizing the website” and potential “negative effects on their scientific careers.” It also allows commenters, who must have an institutional email address, to post anonymously, including under monikers such as “Peer 1” or “Peer 2.”

It's a model that’s worked since its inception about two years ago, with some success. Most notably, Cell said it was investigating a high-profile paper on stem cells last year after a PubPeer commenter noticed potential image duplications in the researchers’ work, Science Insider reporter. The scientists involved stood by their results, but did publish a correction based on the PubPeer observation.

But now PubPeer is facing what it says is its first legal battle related to its anonymity policy: Fazlul Sarkar, a cancer researcher at Wayne State University in Michigan, says he lost out on a job at the University of Mississippi due to anonymous comments posted on the site. So he’s suing the commenters for defamation and other alleged offenses, and subpoenaing PubPeer to get their names. Experts say that’s a relatively common legal tactic in cases involving user-generated review sites, since online platforms are protected from defamation claims against commenters by Section 230 of the Communications Decency Act. But the legal territory is still new for post-publication peer review sites, and Sarkar's case already has attracted lots of attention in academic circles.

Sarkar vs. John and/or Jane Doe(s)

According to the lawsuit he filed this month in a Michigan court, Sarkar, a tenured longtime professor at Wayne State who has published hundreds of articles, received anticipated terms of an offer from Mississippi in September 2013. Details included a distinguished professorship with tenure in the department of pharmacology and a $350,000 annual salary. He was interviewed extensively over the next several months and, in March 2014, received a formal offer. That offer was confirmed by the provost in April and approved by the Board of Trustees of State Institutions of Higher Learning in May, according to the suit. Several days after the board vote, Sarkar submitted his resignation to Wayne State and prepared to sell his home, for a start date of August 1.

But in late June, Sarkar received a letter from Larry Walker, director of the National Center for Natural Products Research at Mississippi’s Cancer Institute, rescinding his offer.

Walker allegedly wrote that he and at least two other colleagues had received a series of emails forwarded anonymously, all containing comments posted on PubPeer about Sarkar’s work. The comments alleged that Sarkar had used the same images in different experiments, suggesting falsified research.

“At this point, we cannot go forward with an employment relationship with you and your group,” Walker wrote. “With these allegations lodged in a public space and presented directly to colleagues here (I am not sure of the scope of anonymous distribution), to move forward would jeopardize our research enterprise and my own credibility.”

Indeed, such a discussion of Sarkar's work had appeared on PubPeer. Some commenters invited others to compare an illustration from one 2006 article with another, similar illustration published in a separate journal that year. An unregistered submission links to another page where a commenter says that Sarkar used “the same blot to represent different experiments. I guess the reply from the authors would be inadvertent errors in figure representation.”

Another unregistered submission reads: “You might expect the home institution to at least look into the multiple concerns which have been raised.” Later, another unregistered commenter – possibly the same person, Sarkar says -- reveals that a misconduct claim was filed at Wayne State in October 2013. Sarkar says posting that kind of information, along with allegations of research misconduct, are clear violations of PubPeer's commenting policy, so the comments never should have appeared on the site.

It's true that PubPeer tells users: "PLEASE [emphasis theirs] don't accuse any authors of misconduct." The reason? "Firstly, we are scientists," the moderators say. "We should only work with data and logic. Our conclusions must be verifiable. On PubPeer, that means that the facts must be restricted to those accessible to other readers. ...Comments based upon personal knowledge or hearsay are unacceptable for the same reason -- you CAN'T say 'My friend used to work in the lab and said they cheated all the time.' "

PubPeer also cites legal concerns about defamatory comments, saying it's "a foregone conclusion" that a lawsuit from a "disgruntled" party will arise at some point.

Sarkar says another comment about him on the website is an “attack on his character” and an “invitation” for Mississippi and other bodies to investigate him. Comments in that thread include: “Just letting you know that the award for doing what he/she allegedly did is promotion [to] a prestigious position at a different institution. Strange.” And, referencing his federal funding: “Why isn’t the [National Institutes of Health] and [Defense Department] investigating? The problems came to light only because they were gel photos. What else could be wrong? Figures, tables could be made up as well.”

The suit acknowledges that PubPeer is protected by Section 230, so long as the commenters aren't also the moderators. Sarkar also says that PubPeer did respond to some of his requests to take defamatory posts down, but only after the most damage was done. He does not deny the claim that he was reported for research misconduct at Wayne State but says that he was “never found responsible” for it, and he says he’s voluntarily published a relatively small number of errata, for about 2 percent of his papers.

PubPeer via email said its moderators could not discuss the case. But they posted a page on their website saying it was their first legal challenge, and offered some commentary -- including an invitation for Sarkar to post data that disproves any unflattering comments.

“In terms of legal tactics, one intriguing point is that regulations have almost certainly obliged the researcher to retain the original data for some of the papers commented on PubPeer,” the page says. “That data has the potential to resolve all issues immediately.”

But in the event that the case does proceed to court, PubPeer moderators said, “it should contribute to the exploration of the poorly defined interface between the law of defamation and the scientific process. The case may have to resolve what is perhaps the most vexed legal issue in post-publication peer review today: if features of published data are suggestive of misconduct, is it defamatory to draw attention to those features?

Implications for Post-Publication Peer Review

PubPeer also says that it may not have identifying data for the commenters, but that it’s re-evaluating its “data retention policy," to err on the side of retaining less data about commenters in the interest of preserving “open discussion.”

Sarkar, who is back at Wayne State working on a one-year contract without tenure, referred questions to his lawyer, Nicolas Roumel.

Roumel said he and his client are not advocating for censorship and “recognize there are many anonymous web sites and people enjoy free speech rights." But when “when comments cross the line and become defamatory, a person should have the right to know who made the comments so that they may be held accountable," he said. That’s why they’ve subpoenaed PubPeer for the names of the Sarkar's biggest critics.

Anonymous websites “should exercise their screening processes with utmost caution and responsibility, given that an unproven allegation on the internet can affect people’s careers and lives so profoundly,” Roumel added. That’s especially true because an anonymous person commenting “may have a suspect motive or personal ax to grind.”

Roumel also said he was “looking into” the tenure issues involved in Sarkar’s case -- namely that he had tenure at Mississippi and apparently lost it based on anonymous comments on a website. Whether Wayne State ever investigated Sarkar for research misconduct is unclear; a spokesman for Wayne State said he couldn’t comment on a personnel matter with legal implications. Roumel said didn't answer that question directly, either, but said research misconduct "is a legally precise term," and inadvertently or "unintentionally using an incorrect illustration is not research misconduct."

Tom Eppes, a spokesman for Mississippi, said that Sarkar was given the opportunity to explain the research-related allegations “before going forward with employment.” But Eppes said the professor “never responded or otherwise provided assistance in addressing the questions.”

Eppes said Sarkar was never employed by the university or conferred tenure, but minutes from the state Board of Trustees meeting on May 15 show that the board voted to approve his tenured appointment at an annual salary of $350,000.

Andrew Crocker, a legal fellow for the Electronic Frontier Foundation, said he couldn't comment directly on the Sarkar case, as his organization was involved. But said EFF sees “a lot” of defamation cases in which plaintiffs sue anonymous commenters in order to ask the intermediary for their identities. Post-publication peer review is relatively new territory, though, he said, since many cases involve businesses objecting to posts on platforms such as Yelp.

Still, Crocker said there’s a well-developed body of the law that governs “how and when plaintiffs can get that information,” and the bar is high, especially for public figures. Anonymity is rooted in the free speech “tradition,” Crocker said, so a plaintiff must show that the comments are defamatory -- generally untrue and harmful -- and that there are “compelling” reasons to reveal the commenters’ names.

So while anonymity is a right that’s important to the foundation, it understands that it’s not “absolute," Crocker said. Unfortunately, not everyone knows that it has limits -- even on the internet.

In any case, Sarkar's case is likely to prove instructive for post-publication peer review -- which will continue to grow, said Michael Eisen, a professor of genetics and science blogger at the University of California at Berkeley. 

“I definitely think we'll see more of [sites like PubPeer], and I think this is a very good thing -- we need more and robust systems for post-publication review so that we can get past the idea, prevalent today, that pre-publication peer review is an effective means to assess the validity and impact of the published literature,” he said via email.

But the future of anonymity on such sites is a “is a trickier question," he said.

“Everything is cleaner if people engaging in post-publication review are not anonymous -- it allows not only the original authors, but also readers of reviews to contextualize them more effectively, and it serves as a fairly obvious defense against certain forms of abuse,” Eisen said. “However, anonymity also affords people, especially younger, less well-established scientists and people who for other reasons feel less secure in their positions, the opportunity to be honest without fear of retribution or other negative effects on their careers.”

Sites that allow anonymous commenting have to make sure the system doesn’t “break down,” however, Eisen said. “The best and maybe only way to do this is to allow reviewers to be anonymous to people reading the reviews, but not anonymous to people organizing the review process -- much as is done with anonymous peer review of papers and grants today.” Under that system, grant applicants don't know who is reviewing them, but an editor or grants administrator does, “so some degree of trust in the process is maintained.”

John DuPuis, a science librarian and blogger at York University in Canada, also said post-publication review is here to stay. Still, he said, it remains "pretty controversial and it's hard to know how things will fall out in terms of community norms around anonymity and such." Sarkar's case is "part of the process of feeling out the limits," he guessed.