Sparks of Artificial General Intelligence: Early experiments with GPT-4

Just to further this, it's not just 'big names' that feel this way. Read this paper from a team at Microsoft Research: https://arxiv.org/abs/2303.12712 . These folks spent months studying properties of GPT-4, that paper is ~150 pages of examples probing the boundaries of the model's world understanding. There is obviously some emergent complexity arising from the training procedure.

You might find this video[1] on why "GPT is smarter than you think" interesting. They're older, but the "Sparks of AGI" paper[2] and video[3] are also well worth your time. [1] - https://m.youtube.com/watch?v=wVzuvf9D9BU [2] - https://arxiv.org/abs/2303.12712 [3] - https://m.youtube.com/watch?v=qbIk7-JPB2c

The recent post from Stephen Wolfram[1] is pretty good as an introduction, but I haven't seen any super comprehensive material that tries do dissect all the interesting behaviour we see in the really big llms. For that just reading the relevant papers themselves has been pretty fruitful for me. Some of them are actually very well written, even if you aren't used to reading scientific papers. I can recommend the Sparks of AGI paper[2] and the toolformer paper[3]. Obviously there's much more out there, those three things are a pretty good read. [1]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... [2]: https://arxiv.org/abs/2303.12712 [3]: https://arxiv.org/abs/2302.04761

I just went down a rabbit-hole, and happily I found the rabbit - an early example of this type of prompt (draw an unusual animal character in a vector graphics language). There is a paper and a 1 hour podcast resulting from a Microsoft evaluation of a pre-release of GPT4. One of the prompts (see pages 4,7,8 in the paper PDF) was to draw a unicorn in TikZ. (I'll leave to the historians the questions of whether this was the first, or whether Simon Willison may have been inspired by this). I remember hearing some of the podcast, and recalled the prompt about balancing on a nail, and the triangle forming the unicorn's tusk; this was clearly a big step beyond choosing the next word and blending images. https://arxiv.org/abs/2303.12712

> there’s zero chance any AI lab would train a model for such a ridiculous task. A lot of people here stated that this is a ridiculous metric, but no one seems to remember that it was introduced in the initial GPT report ("Sparks of Artificial General Intelligence: Early experiments with GPT-4" [1]) by Microsoft about 3 years ago. Shortly after that it was parroted by a network of booster accounts and became a thing every clueless AI hype peddler does to "test" models. 100% marketing, 0% science. [1] https://arxiv.org/pdf/2303.12712

Minor correction: "sparks of AGI" was in reference to GPT-4, which came out 5 months later in 2024. (https://arxiv.org/abs/2303.12712)

> alignment becomes adversarial against intelligence itself. It was hinted at (and outright known in the field) since the days of gpt4, see the paper "Sparks of agi - early experiments with gpt4" (https://arxiv.org/abs/2303.12712)

Yes, he published a paper claiming GPT-4 has "sparks" of AGI. What else is he known for in the field of computer science? https://arxiv.org/abs/2303.12712

Disagree. We know it _can_ learn out of distribution capabilities based on similarities to other distributions. Like the TikZ Unicorn[1] (which was not in training data anywhere) or my code (which has variable names and methods/ideas probably not seen 1:1 in training). IMO this out of distribution learning is all we need to scale to AGI. Sure there are still issues, it doesn't always know which distribution to pick from. Neither do we, hence car crashes. [1]: https://arxiv.org/pdf/2303.12712 or on YT https://www.youtube.com/watch?v=qbIk7-JPB2c

> Feynman, Bill Gates, Socrates, etc. One of these doesn't quite belong ;) But these AI researchers don't even understand these figures except as advertising reference points. The Socratic dialogue in the "sparks of AGI" paper https://arxiv.org/abs/2303.12712 has nothing whatsoever to do with Socrates or the way he argued. Fourteen authors and not a single one seemed to realize there's any possible difference between a Socratic dialogue and a standard hack conversation where one person is named "Socrates."

The gpt4 model was able to draw a unicorn and given a program that drew a unicorn, modify it so the unicorn was rotated 90 degrees. https://arxiv.org/abs/2303.12712

That talk[1] (also paper[2]) covering some of this by the Microsoft team involved with the unfettered GPT-4 model certainly shifted my views last year at a time when until then it was more common to see downplaying about LLMs based on prior gen models. Recently saw an AI safety specialist who also become more concerned about the rapidity of progress cite their research. [1] https://www.youtube.com/watch?v=qbIk7-JPB2c [2] https://arxiv.org/abs/2303.12712

*Further Reading*: - [GPT-4's decline over time (HackerNews)](https://news.ycombinator.com/item?id=36786407) - [GPT-4 downgrade discussions (OpenAI Forums)](https://community.openai.com/t/gpt-4-has-been-severely-downg...) - [Behavioral changes in ChatGPT (arXiv)](https://arxiv.org/abs/2307.09009) - [Zero-Shot Replication Effort (Github)](https://github.com/emrgnt-cmplxty/zero-shot-replication) - [Inconsistencies in GPT-4 HumanEval (Github)](https://github.com/evalplus/evalplus/issues/15) - [Early experiments with GPT-4 (arXiv)](https://arxiv.org/abs/2303.12712) - [GPT-4 Technical Report (arXiv)](https://arxiv.org/abs/2303.08774)

> What is speaking if not stringing together coherent words in a readable sentence? Haven't the latest LLMs shown us that a neural net trained to "just string together readable words" leads to at least simple intelligence? [1] > I would say that emotion is something more intangible It feels like we just keeping shifting the goal posts. > If I tell ChatGPT "I shot you dead!" and it says "ow!" back, nothing has transpired I agree. On the other hand, if I tell a bot "I'm going to turn you off now" and it tries to stop me, that implies it feels fear. [1] https://arxiv.org/abs/2303.12712

Your response does two things -- Dismisses the countless examples given, from me in this thread/my comment history, and many other people in many threads on this website and Reddit. Dismisses the pretense that certain people get access to unfiltered models under the guise of conspiracy. In the sparks of AGI paper from Microsoft, the researcher mentions the differences in private/test models versus the ones prepped for consumers. If you want a hard to ignore visual example, just look at the unicorn they drew for that paper and then look at https://gpt-unicorn.adamkdean.co.uk/. I hope you can add to these discussions rather than primarily be dismissive. This is not conspiracy, we do not know the intention behind the changes and I am not speculating on the intentions of the actors. Sparks of AGI: https://arxiv.org/pdf/2303.12712.pdf

Aggressive quantization by itself could explain the quality difference. Combined with stricter guardrails, I would certainly expect intelligence to go down. Check out the tikz unicorn drawing example from the paper "Sparks of Artificial General Intelligence: Early experiments with GPT-4" (pdf page 7) https://arxiv.org/pdf/2303.12712.pdf

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s [1]: https://arxiv.org/abs/2303.12712