Measured Innovation in Peer Review
Conversations about the future of academic publishing often revolve around the pros and cons of open peer review. Would a new mechanism for vetting research that relies on the wisdom of crowds, rather than a select few editors and reviewers, lead to a scholarly renaissance or to chaos?
Now several publishers are trying to find a balance. Drawing from both the traditional peer review and open-access models, PeerJ and Rubriq are looking to use the architecture of the Web to build community-oriented platforms that are accessible and empowering, yet stable and habitable — walled gardens, but with windows that open from the outside.
For PeerJ — launched in May by Peter Binfield, the former publisher of the popular open-access journal PLoS One — that means giving researchers a membership stake in the publications. Instead of charging authors for each paper they submit, or charging readers who want to view the final versions, PeerJ asks authors to pay membership fees — between $100 and $300 for a "lifetime" membership — that allow them to submit multiple papers while also obligating them to review papers submitted by other members (at the risk of having their membership lapse).
While only members can submit articles, anybody can view them. “It’s a wholly new model in the open-access space,” says Binfield. The site, which will begin taking submissions late next month, has already accumulated “hundreds” of paid subscribers, he says.
Rubriq, meanwhile, aims to create a layer of peer review within the drafting process. Instead of providing a platform for authors to solicit free, fragmental feedback from a crowd of readers, Rubriq gives them the option of paying for highly regimented reviews, complete with a rubric and standard scoring metrics, from three “independent” reviewers — with a guaranteed one-week turnaround, which is much faster than the review process at traditional journals. Authors are slated to pay around between $500 and $700 to have a paper reviewed. Reviewers will be paid $100 per paper.
The objective is not to circumvent the traditional publishing pathways, but to make them more efficient. By offering a layer of peer review prior to the submission process, Rubriq aspires to use its assessment metrics (still in development) to match articles to particular outlets, “like a dating service for journals and authors,” says Keith Collier, the co-founder.
“There’s a huge problem in the marketplace,” says Collier. “The problem is the time that gets really sucked out of the research community every year simply because of our inefficient publishing model.… It shouldn’t take this long, and it shouldn’t be this painful.”
Neither PeerJ nor Rubriq actually uses “open” peer review, aiming largely to make traditional peer review more efficient rather than replacing it with a more radically progressive system.
“These experiments do rely on fairly traditional mechanisms,” says Kathleen Fitzpatrick, the director of scholarly communication for the Modern Language Association. But they also gravitate toward some of the more participatory, community-oriented ideas for which Fitzpatrick and others have advocated.
In her 2011 book Planned Obsolescence, Fitzpatrick envisioned a paperless regime of post-publication peer review, where getting published is easy but getting visible is harder, and research rises to the top based on whether a preponderance of readers decide it is worthy. Authors might also use “the crowd” to help improve their papers, says Fitzpatrick.
“We’re not necessarily going full swing in that direction,” says Binfield.
Nor is it clear whether Binfield and Collier will swing and miss. PeerJ’s membership model prompted some skepticism when it was unveiled earlier this year. Writing on the popular academic publishing blog Scholarly Kitchen, Phil Davis said the logistics of maintaining an active, reliable membership could prove difficult.
“Initially, when the community is small, it may be very difficult to enlist a competent member-reviewer for a submitted manuscript,” wrote Davis, who is an independent consultant specializing in academic publishing. “PeerJ will need to go outside to the rest of the author community for review help, and this may be difficult to do if the quality of the submitted manuscripts are poor.”
Rubriq, which is still in its beta phase, is similarly untested.
But Fitzpatrick says that even if these companies are not facilitating open peer review, necessarily, she finds it encouraging that they are trying to use technology to make the current system easier on academics.
“We need more experiments moving in more different directions before we figure out what different communities of practice want and need,” she says.