Is it working now?

We have seen a great number of examples of how well peer review has failed in the past few years. Michael Bellesiles, Ward Churchill, and a large number of faculty in the so-called Duke Gang of 88 seem to have “gamed” the peer review process. It is going to need to have some reforms if the public is going to take it seriously.

However, the problem with simply discarding peer review is that the reading public loses any shred of promise that the work has been judged to be solid work by reputable members of the profession. Maybe one way around this is to recognize that signed reviews of work give some way of checking up on both the author and the members of the profession who either agree or disagree with his work. Accountability, in a word. Blind peer review seems to have become problematic, though in theory it is a good approach. Perhaps a compromise would be to keep the reviewers names closed for two years, then reveal them. Again, this may not be popular with the folks who like anonymity so they can be brutally frank about what they find wrong with a work, but it also gives accountability. the author knows if he is being reviewed by impartial people, or those who may have an ax to grind.

That latter point raises another issue — review theoretically is blind, but the editors may be sending work to people who are likely to be favorably disposed toward it, or hostile toward it, and the author has no way of knowing.

Orson Buggeigh, at 9:20 am EDT on July 26, 2007

Peer review cont.

I agree with Prof. Hollander’s comment, but there are still universities that refuse to accept on-line journal publication, even when peer-reviewed. As a stake-holder in such a journal, I urged a paper-presenter to submit to the journal, and was told that Brown University does not count on-line journals even when peer-reviewed. This is obviously second-hand so I would welcome correction if my information is incorrect.Another Modern Languages reader

LM, at 11:00 am EDT on July 26, 2007

Here is a speculation from a completely different angle.

Peer review works best when the author and the reviewers agree on the general theoretical and methodological framework to be applied, so that reviews address the technical strength of the work. Where this agreement fails, reviewing often takes the form of upholding one framework against alternatives. I stopped submitting most of my work to peer review long ago because the responses I got were of the form, “I do not accept your approach to the discipline.” I still serve as a reviewer for various journals, and I try to be scrupulous in this respect, giving the approach of the author the benefit of the doubt (even when I disagree) and looking only for issues of accuracy, consistency, awareness of the relevant literature, etc. I don’t think this attitude is very common.

Now to the point: as long as doctrinal disagreements apply to a minority, it is the minority that must suffer the consequences. Increasingly, however, deep divisions over methodology and theoretical perspective have arisen among the mainstream of the economics profession. A poster child for this development is the story related by David Card concerning the reception of his work with Alan Krueger on the minimum wage.

Of course, most economists must continue to publish in prominent journals in order to gain tenure, promotion and professional esteem. The most suitable journals for this in economics are the prominent general-interest periodicals like the AER, JPE, QJE etc. It is possible, however, that economists whose professional status is secure might choose (at the margin) to forego potentially pointless peer reviews (that doctrinal disagreement issue) in some portion of their applied work.

Incidentally, one argument against the time-to-publication argument advanced in the NBER paper, is that one can publish both to a website or working paper series and to a peer reviewed journal. This is quite common, in fact.

ps: One of my most-cited articles was rejected several times (in different versions) by peer reviewers on doctrinal grounds (no issues of faulty math, interpretation or literature review were raised), but it has served its purpose very nicely on one of the working papers websites.

(As you may have guessed, I am an economics misfit.)

Peter Dorman, at 1:10 pm EDT on July 26, 2007

Peer-Reviews

in any field of expertise and knowledge, people have to have a set of crieria / standards to pass, fulfill and meet, whereby a fair plane is made available in academic research. peer reviews of pioneering research studies and articles is just that — an endorsement that others who are contemporary experts in the field are able to critique the works of the relatively less experienced researchers. of course, researchers and peer reviewers can also be colleagues who have the same level of expertise that either one of them would like to share with the general public and among themselves. the parity is ideal when the ultimate goal is the scientific inquiry and its objective for greater good of maximum amount of people (generalization) and not an exclusionary method of experimenting and ‘declaring’ that what one has found is the truth for everybody.

as people strive to be unbiased in their research and openly discuss their research in the professional circles, peer reviews ought to ideally be seen as opportunities for professional research writing’s growth and refine ness. instead, it is seen as a boring, tedious and at a times unfair process. research has no reason for being considered plausible and effective, unless it’s impact is evident, and once the research is done in a given field, this process of analyzing and critiquing the results and assessing the potential publicity success by way of peer reviewing, is in fact a rewarding experience of consistently improving one’s already written research work. therefore, the process of peer reviewing deserves to be seen in a positive light.

Research, at 4:35 am EDT on July 27, 2007