Adam Karvonen
2024 · arXiv
Language models have shown unprecedented capabilities, sparking debate over the source of their performance. Is it merely the outcome of learning syntactic patterns and surface level statistics, or do they extract semantics and a world model from the text? Prior work by Li et al. investigated this by training a GPT model on synthetic, randomly generated Othello games and found that the model learned an internal representation of the board state. We extend this work into the more complex domain of chess, training on real games and investigating our model's internal representations using linear probes and contrastive activations. The model is given no a priori knowledge of the game and is solely trained on next character prediction, yet we find evidence of internal representations of board state. We validate these internal representations by using them to make interventions on the model's activations and edit its internal board state. Unlike Li et al's prior synthetic dataset approach, our analysis finds that the model also learns to estimate latent variables like player skill to better predict the next character. We derive a player skill vector and add it to the model, improving the model's win rate by up to 2.6 times.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
6 selected excerpts · 6 total mentions
Relax. Anyone who's genuinely interested in the question will see with a few searches that LLMs can play chess fine, although the post-trained models mostly seem to be regressed. Problem is people are more interested in validating their own assumptions than anything else. https://arxiv.org/abs/2403.15498 https://arxiv.org/abs/2501.17186 https://github.com/adamkarvonen/chess_gpt_eval
> Can you please explain how we can discern that GPT-2 in this instance really built a model of the board? Read the article. It's very clear. To quote it: "Next, I wanted to see if my model could accurately track the state of the board. A quick overview of linear probes: We can take the internal activations of a model as it’s predicting the next token, and train a linear model to take the model’s activations as inputs and predict board state as output. Because a linear probe is very simple, we can have confidence that it reflects the model’s internal knowledge rather than the capacity of the probe itself." If the article doesn't satisfy your curiosity, you can continue…
with the academic paper it links to: https://arxiv.org/abs/2403.15498v2 See also Anthropic's research: https://www.anthropic.com/research/mapping-mind-language-mod... If that's not enough, you might explore https://www.amazon.com/Thought-Language-Lev-S-Vygotsky/dp/02... or https://www.amazon.com/dp/0156482401 to better connect language and world models in your understanding.
More details in the previous blog post: https://adamkarvonen.github.io/machine_learning/2024/01/03/c... > A 50 million parameter GPT trained on 5 million games of chess learns to play at ~1300 Elo in one day on 4 RTX 3090 GPUs. And from the paper: https://arxiv.org/abs/2403.15498 > The 25M parameter model took 72 hours to train on one RTX 3090 GPU. The 50M parameter model took 38 hours to train on four RTX 3090 GPUs. definitely inspiring :)
>They still make these errors on anything that is out of distribution. There is literally a post in this thread linking to a chat where Sonnet failed a basic arithmetic puzzle: https://news.ycombinator.com/item?id=47051286 I thought we were talking about actual arithmetic not silly puzzles, and there are many human adults that would fail this, nevermind children. >LLMs can match an example at exactly that trivial level because it can be predicted from context. However, if you construct a more complex example with several rules, especially with rules that have contradictions and have specified logic to resolve conflicts, they fail badly. Even if that were true (Have you…
actually tried?), You do realize many humans would also fail once you did all that right ? >They can't even reliably play Chess or Poker without breaking the rules despite those extremely well-represented in the dataset already, nevermind a made-up set of logical rules. LLMs can play chess just fine (99.8 % legal move rate, ~1800 Elo) https://arxiv.org/abs/2403.15498 https://arxiv.org/abs/2501.17186 https://github.com/adamkarvonen/chess_gpt_eval
LLMs playing chess isn't a big deal. You can train a model on chess games and it will play at a decent ELO and very rarely make illegal moves(i.e 99.8% legal move rate). There are a few such models around. I think post training messes with chess ability and Open ai et al just don't really care about that. But LLMs can play chess just fine. [0] https://arxiv.org/pdf/2403.15498v2 [1] https://github.com/adamkarvonen/chess_gpt_eval
>the model will learn what it can to minimize the error over the specific provided loss function, and no more. Change the loss function and you change what the model learns. You clearly do not really understand what it means to predict internet scale text with increasing accuracy. No more than that ? Fantastic LLMs do not just learn surface statistics. So many papers have thoroughly disabused this that i'm just not going to bother. This is just straight up denial. This havs been evidently shown in chess as well. https://arxiv.org/abs/2403.15498v2 You have no idea what you are talkin about. You've probably never even played with 3.5-turbo-instruct. That's how you can say…
this nonsense. You have your conclusion and keep working backwards to get a justification. >It's interesting that LLMs can reach the ELO level that they do (says more about chess than about LLMs) When you say this for everything LLMs can do then it just becomes a meaningless cope statement.
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Anders Sandberg, Eric Drexler, Toby Ord
The Fermi paradox is the conflict between an expectation of a high {\em ex ante} probability of intelligent life elsewhere in the universe and the apparently lifeless universe we in fact observe.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.