Re: IACR publication reform - bad reviewers
Posted by: cbw
Date: 29 May 2013 15:56
> One concern is that the review process of the unique
> submission must be more reliable than what we have so
> far. I have seen many bad reviews in IACR conferences
I agree. Two possible remedies come to mind.
First, we could use a "rebattle-phase" were authors can comment quickly (1 week or so) on the reviews before they are discussed. I think it will help to identify badly written reviews - in particular reviews with factual flaws. To my knowledge, this was successfully implemented in some security conferences.
The second thing is "reviewing the reviews". Here, authors comment on how helpful / fair reviews are (and also assign numerical grades). Sure - the responds will be biased: If my paper got accepted, I like the review(er) and vice versa. However, this bias can easily be identified and eliminated given a few reviews. Hence, this could be built automatically into the reviewing system. Bad reviewers simply don't get requests for reviews anymore.
I don't have a strong opinion (yet) which would be the best solution, but I would be very grateful for other suggestions / comments :-)