I think Nigel's proposal can go a long way towards solving the problem we currently have with refereeing overload and the resulting bad quality of reviews we suffer from now.

It can also help increasing bandwidth, which I think we need, NOT because I think we should accept lower quality papers, but because we need more bandwidth just to keep the bar in the same position.

Now, a major potential problem is resubmission: the proceedings of the IACR should maintain a good reputation, but might at the same time become essentially the only entry point to publication in the IACR system. So papers will get rejected, but I don't think we want rejection to necessarily be a death warrant on a paper. So if you can resubmit, who decides if the revision is substantial enough to merit a new review? and how to arrive at that decision? This is also the problem Dan points out in another post.

I would propose a combination of several things: first, you can only resubmit after some time has past since last rejection. Claiming you added something substantial in two weeks is probably bogus anyway. Second, you must specify in a dedicated field in the resubmission form what you changed. If this is not clearly written or is quickly found to be wrong or insufficient, you get rejected without a full review. Third, it is clearly communicated that attempts to attack this, e.g., by changing only the title and submit as a new paper will seriously damage the authors' reputation if found out.

- Ivan

From my point of view, this should be handled with the "maybe" option: If something is so bad that it should not be resubmitted, it's a clear "reject". Otherwise, it becomes a "maybe".

If an author does something funny like just changing the title of his article, s/he should be banned for a couple of month, preferably a full year from submitting to Proc/IACR.

Best,
Christopher

Ivan points out an important issue and suggests a very reasonable solution. The ban on resubmission could be one year.

Indeed, the database folks have already solved these problems in the same way, i.e., see the description of the reviewing process of PVLDB and their FAQ.

PVLDB submissions -- [www.vldb.org]
PVLDB FAQ -- [www.vldb.org]


Christian

Hello,

To see the effect of pooled reviewing I have looked up some submission numbers from [icsd.i2r.a-star.edu.sg] They give the average number of papers and accepted papers for all IACR venues. From there we get:
* around 640 submissions to the flagship conferences (110 accepted)
* around 430 submissions to the workshops (120 accepted)

The big question is: how many of these are re-submits?

Let's assume that for each paper accepted there are two that get favourable reviews - so the authors are "implicitly invited" to resubmit (hopefully after making changes). The problem is more pressing for the flagship conferences as their submission dates are roughly spread around the year while a PKC paper cannot go to FSE or vice versa.

So let's fix 200 re-submissions for the flagship conferences. For the workshops, I would also put 100-200 - but for a different reason: When a paper gets rejected from a flagship conference (with not so favourable reviews), the authors may decide to resubmit for a workshop.

Hence, we talk about 300-400 papers here that would not need additional reviews if we use pooled reviewings. Which basically means we eliminate the equivalent of one conference plus one workshop PC :-)

I know it's all guesswork - but impressive guesswork, isn't it ;-)

Based on this, we would have a total of 600-700 papers / year to review under the new regime. Even if we accept 50% of all papers (strictly upper bound), we "only" increase the bandwidth from 230 papers to 350. Which is not too bad from my point of view.

Again - it's only guesswork. For example, I would very much like to have more reliable figures on the number of re-submits then the "educated guess" from above.

Best,
Christopher

Dear Colleagues,
Here are my points:

1. IACR should keep JoC (journal) and C, EC, AC, PKC, FSE, CC, CHES (conferences/workshop) as competitive as they are. With more submissions and same acceptance rate, the number of accepted papers will indeed increase. Say with 300 submissions (will soon happen in C, EC, AC), 20% acceptance rate (60 accepted) should be sufficient.

2. Agree with Nigel: We should not kill the more or less competitive conference which are not under the IACR banner (e.g., SAC, CT-RSA, ACISP, Indocrypt). These conferences provide the diversity. In fact, IACR may present a list of favourite conferences which are no under its umbrella.

3.
(i) Putting the papers in a pool by the authors marking their interest in conferences and

(ii) allowing the PC chairs/members to select from that (given that anonymity will in any case will not be preserved)

will be a disaster.

By themselves the PC Chairs and Members are extremely busy. How will they get time to select the papers from a pool of say 600? A few good papers by new authors will never be noticed from a large pool of bad papers. Author names will become more important than the content of the paper.

4. For resubmission: For IACR conferences and workshops we should have the following.

Any resubmission (means say less than 50% additional material over the earlier work) to any IACR conferences/workshops should list the IACR conferences/workshops where the papers were rejected
earlier. Extra pages must be allowed to add the previous comments and answers.

I believe this is already allowed now, but due to page constraints this is not added by the authors. Please allow additional pages for this.

The conferences which are rejecting the paper may also encourage re-submission. That tag may be useful for future re-submission.

5. Each conference should have more PC Chairs (say 1 each from different areas, compared to editorial board members of JoC) and PC members (should be around 50) for a better review quality. Given 300 submissions, 100 of which are really competitive, there should be two levels of reviews. We cannot afford 3/4 reviews for bad papers. There should be a standard format that should politely say the authors that we are not interested in these papers and we are not reviewing them. Potential papers may be reviewed more seriously, expecting that these will obtain a resubmit decision to other places if rejected.

In my opinion, the current mode does not need any big change. IACR publication channels are competitive and let it be. There are enough places to publish moderate cryptology related works. I have very few publications in IACR publication channels. My personal experience is my papers that are rejected from these places deserve to be rejected.

We should try to provide better reviews to the authors and less load on the serious reviewers.

Best regards,
Subho