FAQ for Paper Submission and Commitment and for Reviewers/Action Editors for ARR w.r.t. ACL 2022

FAQs About Accepted Papers:

1) Can I change the title of my paper?

Yes, you can; this will be enabled shortly in Open Review.

2) Can I change the author list of my paper, e.g., add an author, remove an author, or change the author order?

It is not allowed to add an author to a paper accepted to ACL 2022 or to the Findings of ACL 2022 (this is because adding an author can result in conflicts of interests with the assigned reviewers, meta-reviewers, and SACs). However, removing an author or changing the author order is allowed; this will be enabled shortly in Open Review.

3) My paper was accepted. Can I now release an arXiv version before the camera-ready deadline?

Yes, you can publish your accepted paper on arXiv.

FAQs About Paper Commitment:

1) How to interpret my Scores? Should I Commit to ACL 2022?

TL:DR ACL 2022 Program Chairs and Senior Program Chairs will consider the papers holistically and not only on average scores.

We encourage authors to look carefully at the content of the reviews rather than just judging solely based on the numerical review scores when deciding to commit their papers to ACL 2022. Here is why.

First of all, there are multiple scores in the review forms, and acceptance/rejection decisions are not done based on a single numerical score. ACL 2022 is looking for a diverse program across 23 different areas, including the Special Theme.

Moreover, the numerical scale used by the ACL Open Review (ARR) May-November is different from what was used recently for *ACL conferences (in fact, the *ACL scales and the meanings of the scores have been changing as well: 4-scale, 5-scale, 6-scale). In the past, a score of 3 was for borderline papers, but in the ARR review form, the labeled borderline score is 2 (rather than 3). This means that the absolute numerical scores that you have received from ARR might be lower than what you are used to in previous years. Moreover, we want to emphasize that we are looking beyond the scores, as we want a strong and diverse program across different areas. Thus, the ACL 2022 Program Chairs and the Senior Area Chairs will consider the papers and the comments of the reviewers/meta-reviewers as well as the comment to the Senior Area Chair if the authors choose to provide one.

Note also that the scores for the ARR meta-review have a different meaning than those for the ARR reviews.

Here is the review scale used by ARR for May-November:

*Overall Assessment

5 = Top-Notch: This paper has great merit, and easily warrants acceptance in a *ACL top-tier venue.

4.5

4 = Strong: This paper is of significant interest (for broad or narrow sub-communities), and warrants acceptance in a top-tier *ACL venue if space allows.

3.5

3 = Good: This paper is of interest to the *ACL audience and could be published, but might not be appropriate for a top-tier publication venue. It would likely be a strong paper in a suitable workshop.

2.5

2 = Borderline: This paper has some merit, but also significant flaws. It does not warrant publication at top-tier venues, but might still be a good pick for workshops.

1.5

1 = Poor: This paper has significant flaws, and I would argue against publishing it at any *ACL venue.

Here is the meta-review scale used by ARR for May-November:

*Overall Assessment

5 = The paper is largely complete and there are no clear points of revision

4 = There are minor points that may be revised

3 = There are major points that may be revised

2 = The paper would need significant revisions to reach a publishable state

1 = Even after revisions, the paper is not likely to be publishable at an *ACL venue

Note that both the ARR review and meta-review forms changed in December, but this does not affect ACL 2022, as the December submissions to ARR are not eligible for commitment to ACL 2022.

Note that you can also revise your paper and re-submit to ARR (https://aclrollingreview.org/authors) even if you commit one version of your paper with reviews and meta-reviews to ACL 2022. See item 3) below.

2) Can I commit my paper with reviews and meta-reviews to ACL 2022 and also to revise and to resubmit to ARR in future cycles?

Yes. Check the message by ARR: https://aclrollingreview.org/choices/

However, make sure that your resubmission to ARR is a revised version of your paper, not the exact same version that you are committing to ACL 2022. Check this blogpost from ARR for more detail: https://aclrollingreview.org/resubmissions/

3) Can I revise my paper based on the reviews and the meta-reviews and then commit this revised version directly to ACL 2022?

No. You can commit to ACL 2022 only the exact version of the paper that has reviews and meta-reviews (and in case of multiple versions, this should be the latest version that has reviews). However, you can revise the paper, addressing the reviewers' comments and submit a revised version to ARR in a subsequent cycle (even if a version of the paper is committed to ACL 2022). Here is more information on how to submit a revised version to ARR https://aclrollingreview.org/authors

4) How can I commit my paper to ACL 2022?

For ACL 2022, we have a two-step submission process, where authors first submit their paper to the ACL Rolling Review (ARR), and then once the paper has received reviews and a meta-review, in a second step, the authors need to commit it to ACL 2022 via the ACL 2022 Commitment site. The deadline for committing a paper to ACL 2022 is January 15, 2022 11:59pm, UTC-12 (i.e., Anywhere On Earth timezone). We have prepared an ACL 2022 Authors’ Guide to OpenReview to support authors in this process.

When committing to ACL 2022, you will need to provide a link to your paper in ARR that contains the latest version of the paper with the associated reviews, and meta-review. You will also need to select a track (including the special Theme track), as ACL 2022 has tracks, while ARR does not. Finally, you will be allowed to submit optional comments to the ACL 2022 Senior Area Chairs (SACs).

5) Which Link Should I Put in the ACL 2022 Commitment Form?

When you commit your paper, you need to put a link to the page that shows the reviews and the meta-review.

● Please, do NOT put a link to the PDF or to the list of your papers.

● Examples (all links are fake; this is to show the format):

○ Good (link to the paper reviews and the meta-review)

■ https://openreview.net/forum?id=srA4-MNqTX

○ Bad (link to the PDF):

■ https://openreview.net/pdf?id=srA4-MNqTX

○ Bad (link to all papers by an author):

■ https://openreview.net/forum?id=srA4-MNqTX& referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3Daclweb.org%2FACL%2FARR%2F2021%2FOctober%2FAuthors%23your-submissions)

See also the ACL 2022 Authors’ Guide to OpenReview.

6) What if my paper went through several rounds of review in ARR?

If your paper has multiple versions (revisions), the latest version that has reviews and a meta-review should be committed to ACL 2022.

7) Can I submit an author response in the ACL 2022 commitment form?

The ACL 2022 commitment form allows authors to submit optional comments to the ACL 2022 Senior Area Chairs (SACs). These comments are mainly to raise potential concerns about objective misunderstandings by the reviewers and/or by the Action Editor about the technical aspect of the paper that the authors believe will help the SACs in their decision making process. Note that these comments will only be visible to the SACs, and they will NOT go to the ARR reviewers, nor will they be seen by the ARR Action Editor. Responding to ARR reviewers and/or to the ARR Action Editor should be handled in a response letter if the authors decided to do a resubmission in ARR, which is a completely different process from committing a paper to ACL 2022.

8) What is allowed in a comment to the Senior Area Chairs?

If you choose to send comments to the SACs, note the following:

* Your response should contain no more than 3,000 characters in total.

* Reviewing is double-blind; thus, do not include any information in the response that can identify you or your co-authors.

* If a reviewer or the Action Editor has expressed uncertainty about an issue, or is making an incorrect assumption, or has misunderstood a point in the paper, you can address these concerns in your comments.

* Please note that this process is not intended to strengthen the arguments in your paper: For instance, you should not try to “correct” your paper in any way, e.g., the comments should not be used to report new results obtained after the ARR reviewing period.

* If your paper is accepted, you will still have the opportunity to make a revision before sending the camera-ready version of your paper.

9) What is the anonymity period for ACL 2022?

ACL 2022 has a commitment anonymity period for papers committed to ACL 2022: it starts one month before the commitment deadline (i.e., on December 16, 2021 as the commitment deadline is January 15, 2022), and it ends on the date of notification (February 23, 2022). Note that this ACL 2022 commitment anonymity period is separate from the submission anonymity period for ARR reviewing (yet, both are mentioned in the ARR CFP guidelines): for ARR, the reviewing anonymity period starts one month before submission to ARR and ends either when the authors receive their reviews or when they withdraw their paper (see the ARR CFP). From the ARR CFP page: “A paper can only be (re)submitted to ARR if no deanonymized preprint has been posted in the month prior to submission. Also, a paper can only be submitted to a venue if no deanonymized preprint has been posted in the month prior to submission. Submissions will be rejected if not properly anonymized.” Be careful that the anonymity periods for ARR reviewing and for ACL 2022 commitment can overlap (again from the ARR CFP page): “So, if you submit to the June deadline and get reviews in July, then on July 21st your paper is no longer under review. If you then post a deanonymized preprint on July 22nd, you can’t resubmit to ARR or to a conference in the August round but you can in September.” From an ACL 2022 perspective, you need to comply (i) with the ARR reviewing anonymity, and (ii) with the ACL 2022 commitment anonymity. Both of these are as per the ARR policy.

10) Can I submit/commit my paper to another ARR conference/workshop in parallel to ACL 2022?

As per the ARR Multiple Submission Policy, ACL 2022 will not consider any paper that is under review in a journal or another conference at the time of commitment, and committed papers must not be submitted/committed elsewhere during the ACL 2022 commitment period. This policy covers all refereed and archival conferences and workshops (e.g., IJCAI, SIGIR, NAACL, etc.), including all events registered in the ACL Rolling Review: at any given time, a paper could be committed to no more than one ARR event. We have made sure that the notification date for ACL 2022 (February 23, 2022) allows for a subsequent commitment to ACL 2022 workshops (which all have a submission/commitment deadline on February 28, 2022), and also to NAACL 2022 (whose commitment deadline is March 2, 2022). Moreover, we will not consider any paper that overlaps significantly in content or results with papers that will be (or have been) published elsewhere. Authors committing more than one paper to ACL 2022 must ensure that their submissions do not overlap significantly (>25%) with each other in content or results.

11) I submitted my paper for the November 15 ARR deadline; will I get my reviews and a meta-review in time to commit my paper to ACL 2022?

The ARR reviewing for November is ongoing, and the authors will receive their reviews by January 10, 2022, which would be in time for the commitment deadline for ACL 2022, which we have now pushed to January 15, 2022. Here is the ACL 2022 timeline:

● Last deadline for submission to the ACL Rolling Review: November 15, 2021;

● Anonymity period for ACL 2022 starts: December 16, 2021;

● Commitment deadline for ACL 2022: January 15, 2022;

● Notification of acceptance: February 23, 2022;

● Deadline to withdraw papers: February 26, 2022;

● Camera-ready due: March 15, 2022.

FAQ About Paper Submission

1) If I submit my paper to the ACL Rolling Review by November 15, 2021, is there a guarantee that my paper will be considered for ACL 2022 (i.e., will I get reviews and meta-reviews in time, so that I would be able to commit my paper to ACL 2022 by the January 15, 2022 deadline)?

Yes. The ACL 2022 Program Co-chairs are working very hard to guarantee that all submissions sent by the November 15, 2021 deadline in ARR would be guaranteed to have reviews and meta-reviews, and thus could be considered for ACL 2022 if the authors decide to commit to ACL 2022. See our ACL 2022 Program Co-Chairs Blog Post 2, which gives more detail on what we are doing to help ARR scale up for the expected huge number of submissions in November, 2021.

2) Do ALL authors of a paper need to have a complete OpenReview profile?

Yes. The setup of OpeReview for ARR is such that you cannot submit your paper unless ALL co-authors of your paper have an OpenReview profile. Similarly to how all authors had to have an up-to-date Global Profile in START for EMNLP 2021, all authors of a paper submitted to ARR (and thus potentially eligible for ACL 2022) need to have an OpenReview account and a complete profile. This will be mandatory for the November 15 submissions, so plan ahead!

3) How do I create an OpenReviewProfile?

To create an Open Review account (OpenReview ID) please go to https://openreview.net/login and click on New User. For reviewers/ActionEditors: Make sure you use or add the email address that you used to accept the Reviewer or the Action Editor invitation. For authors: make sure that your OpenReview account (ID) is the one associated with the email address you used for your paper. Once created, an OpenReview account ID/handle would look like this: ~John_Smith1.

Once you have created an OpenReview account, you will receive an email to activate it. Clicking on that link will enable you to complete your profile. It is very important that you fill in all the required information since this profile will be used for many things, including reviewer assignment and for computing COIs.

Note: If you already have a OpenReview profile, go to your OpenReview account, and then to Profile/Edit Profile and make sure ALL the fields below are filled. Make sure your email address with which you accepted the reviewer/Action Editor invite or, if an author, that the email address on your paper is associated with your OpenReview profile.

What to fill in in the OpenReview profile (for matching and COI handling)

a) Personal Links: please include

i) Semantic Scholar URL and

ii) DBLP URL, also click on ADD DBLP papers to profile (when adding papers from DBLP you can import all your papers or you can control what you would like to be used as representative of your expertise: these papers are used for calculating affinity scores for reviewer assignments).

Note: your Google Scholar is not used for matching. What really matters is your Semantic Scholar and the papers you explicitly import from DBLP. Also, your declared conflicts of interest.

b) Education and Career

c) Advisor and Relationship

d) Expertise (please include meaningful keywords about your area such as dialog, information extraction, deep learning, etc rather than generic NLP).

4) Do all authors need to agree to review?

Yes, we ask all authors to be available to review for November if asked by ARR. This is not a commitment for year-long reviewing, just for November and only if asked. Moreover, if an author was already a reviewer or an action editor for ARR in previous months, that load will be taken into account. If for personal or other reasons, an author who is not an ARR reviewer or an action editor is unable to review for November, they would need to provide a convincing rationale for this when invited.

5) How do I indicate that my submission has to be considered for the ACL 2022 Special Theme track?

Please write “ACL 2022 Theme Track” in the field Preferred Venue.

6) If I am re-submitting a paper, do I need to keep the same length (4 pages for short and 8 pages for long)?

Yes. For the resubmission to ARR, you will have to keep the same number of pages as for the original submission. Once you commit the paper at ACL 2022 and the paper is accepted, you will be able to add an extra page for the camera-ready version of your paper. NB: You can submit a response to reviewers with your resubmission, and that is a separate file.

FAQ for Reviewers/Action Editors w.r.t. ACL 2022

7) If I am an Action Editor or Reviewer for ARR, can I say I am also a Reviewer and Area Chair for ACL 2022?

Yes, if you have been an Action Editor and Reviewer for ARR for June through November submissions, you are effectively also an Area Chair or Reviewer for ACL 2022. However, you need to make sure you have a complete OpenReview profile to do that (see item FAQ 8 below).

8) Do I need to have an OpenReview profile to be a Reviewer or an Action Editor for ARR (and thus for ACL 2022)?

Yes. ARR uses the OpenReview platform and relies on your OpenReview Profile for initial reviewer assignments and for computing COIs (similarly to how the Global Profile was used in the START system). It is essential that you have an OpenReview account and that you complete your profile in order to be able to serve as a reviewer or as an action editor for ARR, and thus ACL 2022. See also the answer to FAQ 3) “How to create a complete OpenReview profile” above.

9) I am not available to review year-round for ARR, but I would be happy to review for the November submissions or at other specific times. Can I do that?

Yes. If you were invited to be a reviewer for ARR and you would like to review for the November 15, 2021 cycle please accept by November 11, 2021, and then fill in your OpenReview profile (see FAQ 3 above); then, you can fill in the ARR Unavailability form (e.g., if you are unavailable for some periods of time during the year, you can fill that in and you will not be asked to serve for the submission cycles where your duties overlap with those dates). Similarly, if you would like to volunteer, you should fill in the ARR Reviewer Interest Survey by November 11, 2021 and if invited, you should accept right away and fill in your OpenReview profile as discussed above.

OTHER FAQ

10) How is ACL 2022 planning to handle the large scale in November 2021?

See our ACL 2022 Program Co-Chairs Blog Post 2.