Tutorials at ACL 2022

Schedule

​

Tutorials will be held on Sunday, May 22, 2022.

​

​

​

​

​

Details

T1: A Gentle Introduction to Deep Nets and Opportunities for the Future [introductory, afternoon]

https://github.com/kwchurch/ACL2022_deepnets_tutorial

Kenneth Church, Valia Kordoni, Gary Marcus, Ernest Davis, Yanjun Ma, Zeyu Chen

​

​

​

​

​

​

​

​

The first half of this tutorial will make deep nets more accessible to a broader audience, following “Deep Nets for Poets” and “A Gentle Introduction to Fine-Tuning.” We will also introduce, GFT (general fine tuning), a little language for fine tuning deep nets with short (one line) programs that are as easy to code as regression in statistics packages such as R using glm (general linear models).  Based on the success of these methods on a number of  benchmarks, one might come away with the impression that deep nets are all we need. However, we believe the glass is half-full: while there is much that can be done with deep nets, there is always more to do.  The second half of this tutorial will discuss some of these opportunities
 

Suggested readings:

• Deep Nets for Poets (Church et al., 2021)

• A Gentle Introduction to Fine-Tuning (Church et al., 2021)

T2: Towards Reproducible Machine Learning Research in Natural Language Processing [introductory, morning]

​

https://acl-reproducibility-tutorial.github.io/

Ana Lucic, Maurits Bleeker, Samarth Bhargav, Jessica Zosa Forde, Koustuv Sinha, Jesse Dodge, Alexandra Luccioni, Robert Stojnic

​

​

​

​

​

​

​

While recent progress in the field of ML has been significant, the reproducibility of these cutting-edge results is often lacking, with many submissions lacking the necessary information in order to ensure subsequent reproducibility. Despite proposals such as the Reproducibility Checklist and reproducibility criteria at several major conferences, the reflex for carrying out research with reproducibility in mind is lacking in the broader ML community. We propose this tutorial as a gentle introduction to ensuring reproducible research in ML, with a specific emphasis on computational linguistics and NLP. We also provide a framework for using reproducibility as a teaching tool in university-level computer science programs. 

​

T3: Knowledge-Augmented Methods for Natural Language Processing [cutting-edge, morning]

https://github.com/zcgzcgzcg1/ACL2022_KnowledgeNLP_Tutorial/

Chenguang Zhu, Yichong Xu, Xiang Ren, Bill Yuchen Lin, Meng Jiang, Wenhao Yu

​

​

​

​

​

​

​

​

Knowledge in NLP has been a rising trend especially after the advent of large scale pre-trained models. NLP models with attention to knowledge can i) access unlimited amount of external information; ii) delegate the task of storing knowledge from its parameter space to knowledge sources; iii) obtain up-to-date information; iv) make prediction results more explainable via selected knowledge. In this tutorial, we will introduce the key steps in integrating knowledge into NLP, including knowledge grounding from text, knowledge representation and fusing. We will also introduce recent state-of-the-art applications in fusing knowledge into language understanding, language generation and commonsense reasoning.

​

T4: Non-Autoregressive Sequence Generation [cutting-edge, afternoon]

https://nar-tutorial.github.io/acl2022/

Jiatao Gu, Xu Tan

​

​

​

​

​

​

​

​

Non-autoregressive sequence generation (NAR) attempts to generate the entire or partial output sequences in parallel to speed up the generation process and avoid potential issues (e.g., label bias, exposure bias) in autoregressive generation. While it has received much research attention and has been applied in many sequence generation tasks in natural language and speech, naive NAR models still face many challenges to close the performance gap between state-of-the-art autoregressive models because of a lack of modeling power. In this tutorial, we will provide a thorough introduction and review of non-autoregressive sequence generation, in four sections: 1) Background, which covers the motivation of NAR generation, the problem definition, the evaluation protocol, and the comparison with standard autoregressive generation approaches. 2) Method, which includes different aspects: model architecture, objective function, training data, learning paradigm, and additional inference tricks. 3) Application, which covers different tasks in text and speech generation, and some advanced topics in applications. 4) Conclusion, in which we describe several research challenges and discuss the potential future research directions. We hope this tutorial can serve both academic researchers and industry practitioners working on non-autoregressive sequence generation. 

T5: Learning with Limited Text Data [cutting-edge, morning]

https://github.com/diyiy/ACL2022_Limited_Data_Learning_Tutorial

​

Diyi Yang, Ankur P Parikh, Colin Raffel

​

​

​

​

​

​

Natural Language Processing (NLP) has achieved great progress in the past decade on the basis of neural models, which often make use of large amounts of labeled data to achieve state-of-the-art performance. The dependence on labeled data prevents NLP models from being applied to low-resource settings and languages because of the time, money, and expertise that is often required to label massive amounts of textual data.  Consequently, the ability to learn with limited labeled data is crucial for deploying neural systems to real-world NLP applications. Recently, numerous approaches have been explored to alleviate the need for labeled data in NLP such as data augmentation and semi-supervised learning. This tutorial aims to provide a systematic and up-to-date overview of these methods in order to help researchers and practitioners understand the landscape of approaches and the challenges associated with learning from limited labeled data, an emerging topic in the computational linguistics community.  We will consider applications to a wide variety of NLP tasks (including text classification, generation, and structured prediction) and will highlight current challenges and future directions.

​

T6: Zero- and Few-Shot NLP with Pretrained Language Models [cutting-edge, afternoon]

https://github.com/allenai/acl2022-zerofewshot-tutorial

Iz Beltagy, Arman Cohan, Robert L. Logan IV, Sewon Min, Sameer Singh

​

​

​

​

​

​

​

​

The ability to efficiently learn from little-to-no data is critical to applying NLP to tasks where data collection is costly or otherwise difficult. This is a challenging setting both academically and practically---particularly because training neutral models typically require large amount of labeled data. More recently, advances in pretraining on unlabelled data have brought up the potential of better zero-shot or few-shot learning (Devlin et al., 2019; Brown et al., 2020). In particular, over the past year, a great deal of research has been conducted to better learn from limited data using large-scale language models. In this tutorial, we aim at bringing interested NLP researchers up to speed about the recent and ongoing techniques for zero- and few-shot learning with pretrained language models. Additionally, our goal is to reveal new research opportunities to the audience, which will hopefully bring us closer to address existing challenges in this domain.

Suggested readings:

• Language Models are Few-Shot Learners (Brown et al., 2020)

• It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (Schick and Schütze, 2021)

• Multitask Prompted Training Enables Zero-Shot Task Generalization (Sanh et al., 2021)

• FLEX: Unifying Evaluation for Few-Shot NLP (Bragg et al., 2021)

T7: Vision-Language Pretraining: Current Trends and the Future [cutting-edge, afternoon]

https://vlp-tutorial-acl2022.github.io/

Aishwarya Agrawal, Damien Teney, Aida Nematzadeh

​

​

​

​

​

​

​

In the last few years, there has been an increased interest in building multimodal (vision-language) models that are pretrained on larger but noisier datasets where the two modalities (e.g., image and text) loosely correspond to each other (e.g., Lu et al., 2019; Radford et al., 2021). Given a task (such as visual question answering), these models are then often fine-tuned on task-specific supervised datasets. (e.g., Lu et al., 2019; Chen et al.,2020; Tan and Bansal, 2019; Li et al., 2020a,b). In addition to the larger pretraining datasets, the transformer architecture (Vaswani et al., 2017) and in particular self-attention applied to two modalities are responsible for the impressive performance of the recent pretrained models on downstream tasks (Hendricks et al., 2021).

In this tutorial, we focus on recent vision-language pretraining paradigms. Our goal is to first provide the background on image–language datasets, benchmarks, and modeling innovations before the multimodal pretraining area. Next we discuss the different family of models used for vision-language pretraining, highlighting their strengths and shortcomings. Finally, we discuss the limits of vision-language pretraining through statistical learning, and the need for alternative approaches such as causal representation learning.

T8: Natural Language Processing for Multilingual Task-Oriented Dialogue [cutting-edge, morning]

https://github.com/cambridgeltl/ACL2022_tutorial_multilingual_dialogue

Evgeniia Razumovskaia, Goran Glavaš, Olga Majewska, Edoardo Ponti, Ivan Vulić

​

​

​

​

​

​

​

​

Recent advances in deep learning have also enabled fast progress in the research of task-oriented dialogue (ToD) systems. However, the majority of ToD systems are developed for English and merely a handful of other widely spoken languages, e.g., Chinese and German. This hugely limits the global reach and, consequently, transformative socioeconomic potential of such systems. In this tutorial, we will thus discuss and demonstrate the importance of (building) multilingual ToD systems, and then provide a systematic overview of current research gaps, challenges and initiatives related to multilingual ToD systems, with a particular focus on their connections to current research and challenges in multilingual and low-resource NLP. The tutorial will aim to provide answers or shed new light to the following questions: a) Why are multilingual dialogue systems so hard to build: what makes multilinguality for dialogue more challenging than for other NLP applications and tasks? b) What are the best existing methods and datasets for multilingual and cross-lingual (task-oriented) dialog systems? How are (multilingual) ToD systems usually evaluated? c) What are the promising future directions for multilingual ToD research: where can one draw inspiration from related NLP areas and tasks?

​

​