Memory Augmented Large Language Models are Computationally Universal

I do. This is the newest thng i'm aware of: https://www.percepta.ai/blog/can-llms-be-computers But there were papers in 2023 with a different approach requiring external memory https://arxiv.org/abs/2301.04589 too

Transformers with memory can abstract recursively in theory. https://arxiv.org/abs/2301.04589. If it's just extra computation that's the kick then simply more tokens will suffice. You can even even implement a computation token if you wish. https://arxiv.org/abs/2310.02226 Frankly, the ability to truly abstract recursively is by no means necessary either. Humans can't actually do this. Either in every day practice (you will get bored, you will lose interest) or in theory (flawed memory, finite memory, you will die). Limited recursive abilities =/ trivial recursive abilities. The current state of the art is easily general enough by actual testable definitions/baselines.

LLMs (at least the ones with read/write memory) can exactly simulate the execution of a universal Turing machine [1]. AFAIK running such models will therefore entails the same fundamental security risks as ordinary software. [1] https://arxiv.org/pdf/2301.04589.pdf

Well I don't understand the nitroglycerin reference and of course "computationally universal" doesn't mean "sentient" but the point is when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training. There are some other papers out there arguing this from a theoretical angle too, but this is the one whose title I quoted: https://arxiv.org/abs/2301.04589

It does to some degree. Some people have shown that you can prompt GPT to behave like shell or python interpreter. Then you can "execute" some statements, of course to some degree of saneness. However I'm not sure if it does in this particular case where we ask it for explanation. Also see this paper: https://arxiv.org/pdf/2301.04589.pdf tldr if we equip LLM models with infinite memory it becomes Turing complete.

Not even going to bother with the weird leap of logic you've made here but Memory augmented LLMs are computationally universal - https://arxiv.org/abs/2301.04589

The significance of the paper is moreso the implications of how far ICL can take you rather than the ease/viability of the solution proposed. Sure there are better methods for arithmetic but arithmetic is extremely quantifiable with rigid steps. What happens when you step out of that kind of domain ? Like the above blog. or Code documentation. For example, you can paste new documentation in a gpt-4 instance and it will use it for your queries as if it trained on it. Basically Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589. and you kind of get the feeling of that from the previous paper.

The "predicting words" part of the language models is a building block, but the actual applications being built around the LLMs are using that predictive ability in very dynamic ways along with external memory, feedback and sophisticated control methods to attenuate and channel their behavior. And anyhow: > Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589