This is one of the mysteries that I have yet to figure out. Obviously, publishing is very important, and reviewing is the main factor deciding whether a submitted paper will be published. But somehow the reviewers receive almost no credit (except for token ones like conference mugs or listed names on the proceedings).
As a result, it should not be a surprise that many people spend much less efforts in reviewing than in publishing. And it is hard for me to imagine this asymmetry in efforts would not cause any problems, or least would not prevent the peer review systems from being better or more effective.
A simple solution is to publish good reviews, *under the reviewers consensus*, for both positive and negative opinions towards the relevant papers. The reviews could appear alongside the published papers in the proceedings, or at least in the digital library.
I could see several advantages of such an approach. First of all, it would make the recognition between papers and reviews more symmetric, and thus gives the reviewers more incentives to do a good job. Since the reviewers could opt for not having the reviews published, there should be no concern about integrity of the review process. The reviews could also provide more useful information for the papers, in the form of digests or opinions. Since reviewers are often among those who read the papers most carefully and thoroughly, their comments could provide additional angles for the relevant papers, which by nature reflect the authors (mostly) one sided opinions. This also makes the review process more transparent, producing a better sense of fairness and letting out some hints on why the papers are accepted.
Of course, reviews for rejected papers would not be published, but the symmetry between publishing/reviewing remains so I do not see any major problems here. One side effect would be that people might steer away from reviewing apparently bad papers even more so than they would now. But I would say that if the committee finds it hard to recruit reviewers for a particular submission, it might as well be a good reason to reject the paper outright; if a paper cannot even attract reviewers, how likely is that it would attract readers once published?
Brilliant idea! To be fair, I would suggest author should be able to decide weather to publish the review content (along with rebuttal content) or not though.
Comment by Tim — March 5, 2010 @ 7:30 pm |