BERT Rediscovers the Classical NLP Pipeline

Ian Tenney, Dipanjan Das, Ellie Pavlick

2019 · arXiv

HN score: 0
Mentions: 3
Readers: 3
Months: 3

From the abstract

Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks. We focus on one such model, BERT, and aim to quantify where linguistic information is captured within the network. We find that the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way, and that the regions responsible for each step appear in the expected sequence: POS tagging, parsing, NER, semantic roles, then coreference. Qualitative analysis reveals that the model can and often does adjust this pipeline dynamically, revising lower-level decisions on the basis of disambiguating information from higher-level representations.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

Show

jayalammarMay 25, 2023

Neutral mention · rule

This is a field I find fascinating. It's generally the research field of Machine Learning Interpretability. The BlackboxNLP workshop is one of the main places for investigating this and is a very popular academic workshop https://blackboxnlp.github.io/ One of the most interesting presentations in the last session of the workshop is this talk by David Bau titled "Direct Model Editing and Mechanistic Interpretability". David and his team locate exact information in the model, and edit it. So for example they edit the location of the Eiffel Tower to be in Rome. So whenever the model generates anything involving location (e.g., the view from the top of the tower), it actually…

Read the full comment

describes Rome Talk: https://www.youtube.com/watch?v=I1ELSZNFeHc Paper: https://rome.baulab.info/ Follow-up work: https://memit.baulab.info/ There is also work on "Probing" the representation vectors inside the model and investigating what information is encoded at the various layers. One early Transformer Explainability paper (BERT Rediscovers the Classical NLP Pipeline https://arxiv.org/abs/1905.05950) found that "the model represents the steps of the traditional NLP pipeline in an interpretable and localizable way: POS tagging, parsing, NER, semantic roles, then coreference". Meaning that the representations in the earlier layers encode things like whether a token is a verb or noun, and later layers encode other, higher-level information. I've made an intro to these probing methods here: https://www.youtube.com/watch?v=HJn-OTNLnoE A lot of applied work doesn't require interpretability and explainability at the moment, but I suspect the interest will continue to increase.

Original comment Show HN: Visual intuitive explanations of LLM concepts (LLM University)

KhoomeiKFeb 17, 2023

Neutral mention · rule

Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc as input, models seem to learn something like syntax. https://arxiv.org/abs/1905.05950 https://aclanthology.org/N19-1419/ https://arxiv.org/abs/1906.04341

Original comment We Found an Neuron in GPT-2

deskuMar 12, 2021

Neutral mention · rule

Here's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950

Original comment Self-supervised learning: The dark matter of intelligence

Keep following the thread.

Sparks of Artificial General Intelligence: Early experiments with GPT-4

Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.

Language models Psychology Mathematics

60 mentions14 months

3.5HN score

2020arXiv

Language Models are Few-Shot Learners

Tom B. Brown, Benjamin Mann, Nick Ryder +28

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.

Language models Machine learning

83 mentions35 months

3HN score

2025arXiv

Design Patterns for Securing LLM Agents against Prompt Injections

Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu +11

As AI agents powered by Large Language Models (LLMs) become increasingly versatile and capable of addressing a broad spectrum of tasks, ensuring their security has become a critical challenge.

Security & cryptography Language models

4 mentions4 months

2.5HN score

2023arXiv

Toolformer: Language Models Can Teach Themselves to Use Tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dessì +5

Language models (LMs) exhibit remarkable abilities to solve new tasks from just a few examples or textual instructions, especially at scale.

Language models

24 mentions5 months

2HN score