Roee Hendel, Mor Geva, Amir Globerson
2023 · arXiv
In-context learning (ICL) in Large Language Models (LLMs) has emerged as a powerful new learning paradigm. However, its underlying mechanism is still not well understood. In particular, it is challenging to map it to the "standard" machine learning framework, where one uses a training set $S$ to find a best-fitting function $f(x)$ in some hypothesis class. Here we make progress on this problem by showing that the functions learned by ICL often have a very simple structure: they correspond to the transformer LLM whose only inputs are the query $x$ and a single "task vector" calculated from the training set. Thus, ICL can be seen as compressing $S$ into a single task vector $\boldsymbolθ(S)$ and then using this task vector to modulate the transformer to produce the output. We support the above claim via comprehensive experiments across a range of models and tasks.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
It's technically deterministic, but it feels nondeterministic in chatbots since tokens are randomly sampled (temp > 0) and input is varied. Using the right prompt makes the model perform better on average, so it's not completely dumb. I like task vectors and soft prompts because I think they show how prompt engineering is cool and useful. https://arxiv.org/pdf/2310.15916 https://huggingface.co/docs/peft/conceptual_guides/prompting
>"In the in-context learning (ICL) paradigm, given a set of examples, the model has to learn the mapping from inputs to outputs. Prior research has demonstrated that LLMs implicitly compress this mapping into a latent activation, called the task vector" Related: In-Context Learning Creates Task Vectors: https://arxiv.org/abs/2310.15916 Function Vectors in Large Language Models: https://functions.baulab.info/ https://x.com/graceluo_/status/1852043048043319360
This is speculation but i don't think this information (how it came to be) is actually irrelevant information for the LLM. Your state, which transformer LLMs notably have no way of transferring is "just" the computation that preceded the prediction. I think LLMs use previous tokens to compute the likely state vector that preceded the tokens and make predictions that continue this computed state. So the 2 board positions at time x may be identical but the state preceding it or "strategy" was not and so the next move is not identical. This way also, you can make a decision based on the perceived competence of your opponent. These 2 papers kind of touch on what i've been…
thinking for a while now. They demonstrate task vectors and function vectors here for explicit in context learning tasks. In the absence of explicit tasks, perhaps state vectors would be more appropriate. In-Context Learning Creates Task Vectors - https://arxiv.org/abs/2310.15916 Function Vectors in Large Language Models - https://arxiv.org/abs/2310.15213
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Anders Sandberg, Eric Drexler, Toby Ord
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.