Dear Serge,

> One concern is that the review process of the unique
> submission must be more reliable than what we have so
> far. I have seen many bad reviews in IACR conferences
> recently.

I agree. Two possible remedies come to mind.

First, we could use a "rebattle-phase" were authors can comment quickly (1 week or so) on the reviews before they are discussed. I think it will help to identify badly written reviews - in particular reviews with factual flaws. To my knowledge, this was successfully implemented in some security conferences.

The second thing is "reviewing the reviews". Here, authors comment on how helpful / fair reviews are (and also assign numerical grades). Sure - the responds will be biased: If my paper got accepted, I like the review(er) and vice versa. However, this bias can easily be identified and eliminated given a few reviews. Hence, this could be built automatically into the reviewing system. Bad reviewers simply don't get requests for reviews anymore.

I don't have a strong opinion (yet) which would be the best solution, but I would be very grateful for other suggestions / comments :-)

Best,
Christopher

Hi,

I would implement both things. A rebattle phase where you can try to convince the reviewer that something he's said is wrong and a review rating where you can assess whether the review was helpful.

If you also add a fairness rating, then it should be weighted by the PC to protect tough but fair reviewers.

Cheers,
Simon Hoerder