ACL 2022 PC Chairs Blog Post 2: Preparations for November 15, 2021 Submissions

Please note that there have been changes to the important dates, and in particular the commitment deadline for ACL 2022 has been moved from January 7, 2022 to January 15, 2022 (23:59, UTC-12).

As part of the ACL plan to implement the ACL Rolling Review (ARR) for major *ACL conference submissions, the ACL 2022 team has been working very closely with the ARR Editors-in-Chief (EiCs). The scale of our conferences requires a massive collaboration effort, which includes the ACL 2022, the NAACL 2022, and the ARR teams, as well as the entire *ACL community, especially during this initial adoption period where the workflows are being adjusted. In this post we describe some of the actions taken for this effort, and also discuss some important preparatory checks for all authors and reviewers for the forthcoming deadline, including ensuring their OpenReview profiles are complete (see also our FAQ for Paper Submission and Reviewer/ActionEditor w.r.t ACL 2022).

To facilitate the coordination and the communication between ARR and the conference teams, on October 13, 2021 one of the ACL 2022 PC chairs joined ARR as a Guest Editor-in-Chief (EiC), and on October 22, 2021 the other two PC chairs joined as EiCs as well. In this new role, we have been helping to handle the submissions from September, October, and November. The NAACL 2022 PC chairs will later join as guest EiCs for the months of December and January.

The ARR team and the ACL 2022 PC chairs are currently working together to integrate the ARR and some of the conference workflows to ensure scaling up, maintaining quality and timely processing of submissions for November, and thus guaranteeing that all papers submitted by the November 15, 2021 deadline can be considered for ACL 2022 if the authors decided to commit to ACL 2022 by January 7, 2022. This requires making sure we have all reviews and meta-reviews ready in time. Below, we describe some of the activities that we have undertaken to prepare for the expected large submission volume that is typical of *ACL conferences and what we need from the community to make ACL 2022 a success!

1. Recreating the typical *ACL hierarchy in ARR for the November submission cycle to handle the larger number of expected submissions. Similarly to other conferences, ARR relies heavily on automatic assignments of papers to reviewers and to Action Editors (AEs). While the matching algorithm typically works very well, particularly if the reviewers and the Action Editors have complete profiles, there are instances where manual checking and refinement is needed, which so far has been done by the EiCs (for AE assignments) and by the AEs for reviewer assignments. Unfortunately, this would not be currently feasible for November, when we expect 2,500-3,000 submissions. To fix this, we have invited the ACL 2022 Senior Area Chairs (SACs) who are not currently Action Editors (AEs) to become Guest SACs in ARR for the November submission cycle and to help check the assignments of AEs and reviewers to papers. Effectively, this means that for the November submissions, we will have in ARR the full 4-level hierarchy that is typical for an *ACL conference: reviewers, Action Editors (roughly equivalent to Area Chairs), Senior Area Chairs (from ACL 2022), and Editors-in-Chief (which includes the original ARR EiCs and the ACL 2022 PC chairs).

2. Increasing the number of reviewers and AEs. The ACL 2022 PC chairs have worked with the ACL 2022 SACs to recruit qualified Action Editors and reviewers for ARR. This resulted in about 2,300 suggested reviewers and 200 AEs. Overall, ARR has so far recruited over 3,400 reviewers and over 470 Action Editors (AEs), but there is a need to further increase this number to be able to deal with the expected number of submissions for the November 15 deadline, while keeping the load for the reviewers and the AEs relatively low.

○ We count on your help. If you are not an ARR reviewer yet, you can volunteer to be a reviewer using the Reviewer Interest Survey and if you have already received an invitation to join ARR as a reviewer (email subject: “[ACL Rolling Review] Invitation to serve as Reviewer”) or as an Action Editor (email subject “[ACL Rolling Review] Invitation to serve as Action Editor”), please make sure to accept it quickly. If you received an invitation for both roles (reviewer and AE), you need to accept only one of them, but you are encouraged to help in both roles if you can (the load of both roles will be taken into account). Note that this is a year-long commitment for ARR, but you can fill in your unavailability using this form (e.g., if you are unavailable for some periods of time during the year, you can fill that in and you will not be asked to serve for the submission cycles where your duties overlap with those dates).

○ Additionally, we ask all authors who submit for the November deadline in ARR to be available to review for November if asked by ARR (authors will need to agree to this as part of the paper submission form). Note that this is not a commitment for year-long reviewing, just for November, and only if you are asked! Moreover, if you are already serving as a reviewer or an AE for ARR with the general year-long commitment, or if you are serving as a SAC in ARR (see above), this will not add any additional reviewing responsibilities to you; however, you need to agree to this in the November submission form anyway.

3. Making sure all authors have OpenReview profiles. It is important that all authors who submit a paper have a properly filled OpenReview profile (including imported papers from DBLP, Semantic Scholar, and conflict of interests) at paper submission time for proper paper matching. Note that a paper cannot be submitted to ARR unless all authors have an OpenReview profile; plan for this in advance! See also our FAQ.

4. Identifying crucial missing information about reviewers and AEs. Many of the already recruited reviewers and AEs do not have OpenReview profiles or have profiles that lack the relevant information that is used for automatic matching of papers to reviewers and AEs (required are Semantic Scholar and importing DBLP papers, as well as filling COIs, and interests), which possibly leads to suboptimal assignments. Email reminders have been sent to ask for OpenReview profiles to be completed with relevant information. We urge anyone who has accepted to be a reviewer or an Action Editor for ARR (and thus, implicitly but effectively to be a reviewer or an area chair for ACL 2022), to make sure they have an OpenReview account and a complete profile by November 7, 2021 (see how to have a properly filled OpenReview profile and read also our FAQ).

5. Preparing tutorials and guides for new OpenReview users. OpenReview is being used as the custom platform for ARR. In the meantime, the ACL PC chairs have been helping ARR prepare visual guides to explain better how AEs and reviewers can do their job in the OpenReview interface. These will be shared by ARR in the coming days and we will link them in our FAQ and in our blog as well.

6. Recruiting an Editorial Manager/Assistant. We are delighted to announce that Sara Goggi from the Institute for Computational Linguistics, National Research Council, Pisa, Italy, is joining ARR as an Editorial Assistant. Sara has ten years of experience as an editorial assistant for the Language Resources and Evaluation journal and she will be helping to streamline the communication with AEs and reviewers, and to monitor the timeline of the reviewing process. The ACL 2022 team helped to recruit Sara and we are grateful to the ACL Exec for their support in this process.

7. Special Theme papers are handled by Special Theme AEs recommended by the Special Theme SACs. Authors who want to submit papers for the ACL 2022 special Theme on “Language Diversity: from Low Resource to Endangered Languages” need to mention the ACL 2022 Theme Track when selecting the venue on the ARR submission form.

8. Identifying emergency AEs and reviewers. In addition to preparing for scaling up for November, we have been assisting with identifying emergency AEs and reviewers for the September submissions. Several submissions to ARR in September did not have AEs assigned by October 15. The ACL 2022 PC chairs have been helping to recruit qualified emergency AEs. We are trying to have as many papers with reviews as possible before November 15 (in case authors want to revise and resubmit). We have promised that one advantage for early submission to ARR is to allow for in-time revisions, but delays might not make this possible in some cases. However, ARR has decided to release incompletely processed submissions for August and September and to allow authors to revise and resubmit an incompletely processed submission to a future ARR cycle. If authors choose to do so, their rebuttal (free form) should only address the reviews for the initial/previous submission that have been received by the time of resubmission. The main issue was that until September, the ARR workflow and system had issues that led to many invitations to review papers being declined. The main issue was that until September, the AAR workflow and system had issues that led to many invitations to review papers being declined.

9. For the November 15 submissions, reviewers and action editors who have committed to review for ARR will receive direct assignments, which they will be expected to complete. Unavailability of reviewers and AEs, for those with a year-long commitment to ARR, should be declared using the ARR Unavailability form.

The ACL 2022 team is committed to contribute to this huge effort of implementing ARR reviewing for our conferences as mandated by the ACL (you can read more about it here and in our previous blog post), but we also want to bring awareness that given the scale of our conferences, it may take some time for this process to mature. We would like to thank the ARR team for their tremendous effort in the past several months and going forward. We are also grateful for all the constructive feedback and support from the community, especially to our SACs, and to the ARR EiCs, AEs, and reviewers who have been working to make this happen.

Other activities

10. Keynote Speakers: We have amazing keynote speakers, which we will announce in the next blog post.

11. Surprise 60th Anniversary activities: We are also planning some surprise activities for the 60th anniversary of ACL in coordination with the Diversity and Inclusion chairs. Again, to be announced in a future blog post.

Stay tuned!

Smaranda, Preslav, and Aline

ACL 2022 Program Co-Chairs