Avsnitt
-
In this episode, we talk with Kayo Yin, an incoming PhD at Berkeley, and Malihe Alikhani, an assistant professor at the University of Pittsburgh, about opportunities for the NLP community to contribute to Sign Language Processing (SLP). We talked about history and misconceptions about sign languages, high-level similarities and differences between spoken and sign languages, distinct linguistic features of signed languages, representations, computational resources, SLP tasks, and suggestions for better design and implementation of SLP models.
-
This episode is the third in our current series on PhD applications.
We talk about what the PhD application process looks like after applications are submitted. We start with a general overview of the timeline, then talk about how to approach interviews and conversations with faculty, and finish by discussing the different factors to consider in deciding between programs.
The guests for this episode are Rada Mihalcea (Professor at the University of Michigan), Aishwarya Kamath (PhD student at NYU), and Sanjay Subramanian (PhD student at UC Berkeley).
Homepages:
- Aishwarya Kamath: https://ashkamath.github.io/
- Sanjay Subramanian: https://sanjayss34.github.io/
- Rada Mihalcea: https://web.eecs.umich.edu/~mihalcea/
The hosts for this episode are Alexis Ross and Nishant Subramani. -
Saknas det avsnitt?
-
This episode is the second in our current series on PhD applications.
How do PhD programs in Europe differ from PhD programs in the US, and how should people decide between them? In this episode, we invite Barbara Plank (Professor at ITU, IT University of Copenhagen) and Gonçalo Correia (ELLIS PhD student at University of Lisbon and University of Amsterdam) to share their perspectives on this question.
We start by talking about the main differences between pursuing a PhD in Europe and the US. We then talk about the application requirements for European PhD programs and factors to consider when deciding whether to apply in Europe or the US. We conclude by talking about the ELLIS PhD program, a relatively new program for PhD students that facilitates collaborations across Europe.
ELLIS PhD program: https://ellis.eu/phd-postdoc (Application Deadline: November 15, 2021)
Homepages:
- Barbara Plank: https://bplank.github.io/
- Gonçalo Correia: https://goncalomcorreia.github.io/
The hosts for this episode are Alexis Ross and Zhaofeng Wu. -
This episode is the first in our current series on PhD applications.
How should people prepare their applications to PhD programs in NLP? In this episode, we invite Nathan Schneider (Professor of Linguistics and Computer Science at Georgetown University) and Roma Patel (PhD student in Computer Science at Brown University) to share their perspectives on preparing application materials.
We start by talking about what factors should go into the decision to apply for PhD programs and how to gain relevant experience. We then talk about the most important parts of an application, focusing particularly on how to write a strong statement of purpose and choose recommendation letter writers.
Blog posts mentioned in this episode:
- Nathan Schneider’s Advice on Statements of Purpose: https://nschneid.medium.com/inside-ph-d-admissions-what-readers-look-for-in-a-statement-of-purpose-3db4e6081f80
- Student Perspectives on Applying to NLP PhD Programs: https://blog.nelsonliu.me/2019/10/24/student-perspectives-on-applying-to-nlp-phd-programs/
Homepages:
- Nathan Schneider: https://people.cs.georgetown.edu/nschneid/
- Roma Patel: http://cs.brown.edu/people/rpatel59/
The hosts for this episode are Alexis Ross and Nishant Subramani. -
In this episode, we discussed the Alexa Prize Socialbot Grand Challenge and this year's winning submission, Alquist 4.0, with Petr Marek, a member of the winning team. Petr gave us an overview of their submission, the design choices that led to them winning the competition, including combining a hardcoded dialog tree and a neural generator model and extracting implicit personal information about users from their responses, and some outstanding challenges.
Petr Marek is a PhD student at the Czech Technical University in Prague.
More about the Alexa Prize challenges: https://developer.amazon.com/alexaprize
Technical report on Alquist 4.0: https://arxiv.org/abs/2109.07968 -
What can NLP researchers learn from Human Computer Interaction (HCI) research? We chatted with Nanna Inie and Leon Derczynski to find out. We discussed HCI's research processes including methods of inquiry, the data annotation processes used in HCI, and how they are different from NLP, and the cognitive methods used in HCI for qualitative error analyses. We also briefly talked about the opportunities the field of HCI presents for NLP researchers.
This discussion is based on the following paper: https://aclanthology.org/2021.hcinlp-1.16/
Nanna Inie is a postdoctoral researcher and Leon Derczynski is an associate professor in CS at the IT University of Copenhagen.
The hosts for this episode are Ana Marasović and Pradeep Dasigi. -
In this episode, we talk with Lisa Beinborn, an assistant professor at Vrije Universiteit Amsterdam, about how to use human cognitive signals to improve and analyze NLP models. We start by discussing different kinds of cognitive signals—eye-tracking, EEG, MEG, and fMRI—and challenges associated with using them. We then turn to Lisa’s recent work connecting interpretability measures with eye-tracking data, which reflect the relative importance measures of different tokens in human reading comprehension. We discuss empirical results suggesting that eye-tracking signals correlate strongly with gradient-based saliency measures, but not attention, in NLP methods. We conclude with discussion of the implications of these findings, as well as avenues for future work.
Papers discussed in this episode:
Towards best practices for leveraging human language processing signals for natural language processing: https://api.semanticscholar.org/CorpusID:219309655
Relative Importance in Sentence Processing: https://api.semanticscholar.org/CorpusID:235358922
Lisa Beinborn’s webpage: https://beinborn.eu/
The hosts for this episode are Alexis Ross and Pradeep Dasigi. -
In this episode, we talk to Shunyu Yao about recent insights into how transformers can represent hierarchical structure in language. Bounded-depth hierarchical structure is thought to be a key feature of natural languages, motivating Shunyu and his coauthors to show that transformers can efficiently represent bounded-depth Dyck languages, which can be thought of as a formal model of the structure of natural languages. We went on to discuss some of the intuitive ideas that emerge from the proofs, connections to RNNs, and insights about positional encodings that may have practical implications. More broadly, we also touched on the role of formal languages and other theoretical tools in modern NLP.
Papers discussed in this episode:
- Self-Attention Networks Can Process Bounded Hierarchical Languages (https://arxiv.org/abs/2105.11115)
- Theoretical Limitations of Self-Attention in Neural Sequence Models (https://arxiv.org/abs/1906.06755)
- RNNs can generate bounded hierarchical languages with optimal memory (https://arxiv.org/abs/2010.07515)
- On the Practical Computational Power of Finite Precision RNNs for Language Recognition (https://arxiv.org/abs/1805.04908)
Shunyu Yao's webpage: https://ysymyth.github.io/
The hosts for this episode are William Merrill and Matt Gardner. -
We discussed adversarial dataset construction and dynamic benchmarking in this episode with Douwe Kiela, a research scientist at Facebook AI Research who has been working on a dynamic benchmarking platform called Dynabench. Dynamic benchmarking tries to address the issue of many recent datasets getting solved with little progress being made towards solving the corresponding tasks. The idea is to involve models in the data collection loop to encourage humans to provide data points that are hard for those models, thereby continuously collecting harder datasets. We discussed the details of this approach, and some potential caveats. We also discussed dynamic leaderboards, a recent addition to Dynabench that rank systems based on their utility given specific use cases.
Papers discussed in this episode:
1. Dynabench: Rethinking Benchmarking in NLP (https://www.semanticscholar.org/paper/Dynabench%3A-Rethinking-Benchmarking-in-NLP-Kiela-Bartolo/77a096d80eb4dd4ccd103d1660c5a5498f7d026b)
2. Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking (https://www.semanticscholar.org/paper/Dynaboard%3A-An-Evaluation-As-A-Service-Platform-for-Ma-Ethayarajh/d25bb256e5b69f769a429750217b0d9ec1cf4d86)
3. Adversarial NLI: A New Benchmark for Natural Language Understanding (https://www.semanticscholar.org/paper/Adversarial-NLI%3A-A-New-Benchmark-for-Natural-Nie-Williams/9d87300892911275520a4f7a5e5abf4f1c002fec)
4. DynaSent: A Dynamic Benchmark for Sentiment Analysis (https://www.semanticscholar.org/paper/DynaSent%3A-A-Dynamic-Benchmark-for-Sentiment-Potts-Wu/284dfcf7f25ca87b2db235c6cdc848b4143d3923)
Douwe Kiela's webpage: https://douwekiela.github.io/
The hosts for this episode are Pradeep Dasigi and Alexis Ross. -
We invited members of Masakhane, Tosin Adewumi and Perez Ogayo, to talk about their EMNLP Findings paper that discusses why typical research is limited for low-resourced NLP and how participatory research can help.
As a result of participatory research, Masakhane has many, many success stories: first datasets and benchmarks in African languages, first research on human evaluation specifically for MT for low-resource languages, etc. In this episode, we talked about one of them—MasakhaNER—in more detail.
The hosts for this episode are Pradeep Dasigi and Ana Marasović.
--------------------------
Tosin Adewumi is a PhD student at the Luleå University of Technology in Sweden. His Twitter handle: @tosintwit
Perez Ogayo is an undergrad student at the African Leadership University in Rwanda. Her Twitter handle: @a_ogayo
Masakhane is a grassroots organization whose mission is to strengthen and spur NLP research in African languages, for Africans, by Africans: https://www.masakhane.io/
Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages (Findings of EMNLP 2020): https://arxiv.org/abs/2010.02353
MasakhaNER: Named Entity Recognition for African languages (AfricaNLP Workshop @ EACL 2021): https://arxiv.org/abs/2103.11811 -
We invited Lisa Li to talk about her recent work, Prefix-Tuning: Optimizing Continuous Prompts for Generation. Prefix tuning is a lightweight alternative to finetuning, and the idea is to tune only a fixed-length task-specific continuous vector, and to keep the pretrained transformer parameters frozen. We discussed how prefix tuning compares with finetuning and other efficient alternatives on two tasks in various experimental settings, and in what scenarios prefix tuning is preferable.
Lisa is a Phd student at Stanford University. Lisa's webpage: https://xiangli1999.github.io/
The hosts for this episode are Pradeep Dasigi and Ana Marasović. -
How can we build Visual Question Answering systems for real users? For this episode, we chatted with Danna Gurari, about her work in building datasets and models towards VQA for people who are blind. We talked about the differences between the existing datasets, and Vizwiz, a dataset built by Gurari et al., and the resulting algorithmic changes. We also discussed the unsolved challenges in this field, and the new tasks they result in.
Danna Gurari is an Assistant Professor as well as Founding Director of the Image and Video Computing group in the School of Information at University of Texas at Austin (UT-Austin).
Vizwiz project page: https://vizwiz.org/
The hosts for this episode are Ana Marasović and Pradeep Dasigi. -
We invited Jayant Krishnamurthy and Hao Fang, researchers at Microsoft Semantic Machines to discuss their platform for building task-oriented dialog systems, and their recent TACL paper on the topic. The paper introduces a new formalism for task-oriented dialog to effectively handle references and revisions in complex dialog, and a large realistic dataset that uses this formalism.
Leaderboard associated with the dataset: https://microsoft.github.io/task_oriented_dialogue_as_dataflow_synthesis/
Jayant's Twitter handle: https://twitter.com/jayantkrish
Hao's Twitter handle: https://twitter.com/hfang90 -
In this episode, Robin Jia talks about how to build robust NLP systems. We discuss the different senses in which a system can be robust, reasons to care about system robustness, and the challenges involved in evaluating robustness of NLP models. We talk about how to build certifiably robust models through interval bound propagation and discrete encoding functions, as well as how to modify data collection procedures through active learning for more robust model development.
Robin Jia is currently a visiting researcher at Facebook AI Research, and will be an assistant professor in the Department of Computer Science at the University of Southern California starting Fall 2021. -
We invited Nils Holzenberger, a PhD student at JHU to talk about a dataset involving statutory reasoning in tax law Holzenberger et al. released recently. This dataset includes difficult textual entailment and question answering problems that involve reasoning about how sections in tax law are applicable to specific cases. They also released a Prolog solver that fully solves the problems, and show that learned models using dense representations of text perform poorly. We discussed why this is the case, and how one can train models to solve these challenges.
Project webpage: https://nlp.jhu.edu/law/ -
We invited Alona Fyshe to talk about the link between NLP and the human brain. We began by talking about what we currently know about the connection between representations used in NLP and representations recorded in the brain. We also discussed how different brain imaging techniques compare to each other. We then dove into experiments investigating how hidden states of LSTM language models correlate with EEG brain imaging data on three types of language inputs: well-formed grammatical sentences, pseudo-word sentences preserving syntax but not semantics, and word-lists preserving neither. We talk about the kinds of conclusions that can be drawn from these correlations and conclude by discussing avenues for future work.
-
We invited Asli Celikyilmaz for this episode to talk about evaluation of text generation systems. We discussed the challenges in evaluating generated text, and covered human and automated metrics, with a discussion of recent developments in learning metrics. We also talked about some open research questions, including the difficulties in evaluating factual correctness of generated text.
Asli Celikyilmaz is a Principal Researcher at Microsoft Research.
Link to a survey co-authored by Asli on this topic: https://arxiv.org/abs/2006.14799 -
In this episode, Diyi Yang gives us an overview of using NLP models for social applications, including understanding social relationships, processes, roles, and power. As NLP systems are getting used more and more in the real world, they additionally have increasing social impacts that must be studied. We talk about how to get started in this field, what datasets exist and are commonly used, and potential ethical issues. We additionally cover two of Diyi's recent papers, on neutralizing subjective bias in text, and on modeling persuasiveness in text.
Diyi Yang is an assistant professor in the School of Interactive Computing at Georgia Tech. -
In this episode, we talked about Coreference Resolution with Marta Recasens, a Research Scientist at Google. We discussed the complexity involved in resolving references in language, the simplification of the problem that the NLP community has focused on by talking about specific datasets, and the complex coreference phenomena that are not yet captured in those datasets. We also briefly talked about how coreference is handled in languages other than English, and how some of the notions we have about modeling coreference phenomena in English do not necessarily transfer to other languages. We ended the discussion by talking about large language models, and to what extent they might be good at handling coreference.
-
We interviewed Sameer Singh for this episode, and discussed an overview of recent work in interpreting NLP model predictions, particularly instance-level interpretations. We started out by talking about why it is important to interpret model outputs and why it is a hard problem. We then dove into the details of three kinds of interpretation techniques: attribution based methods, interpretation using influence functions, and generating explanations. Towards the end, we spent some time discussing how explanations of model behavior can be evaluated, and some limitations and potential concerns in evaluation methods.
Sameer Singh is an Assistant Professor of Computer Science at the University of California, Irvine.
Some of the techniques discussed in this episode have been implemented in the AllenNLP Interpret framework (details and demo here: https://allennlp.org/interpret). - Visa fler
