Toolformer: Language Models Can Teach Themselves to Use Tools

The recent post from Stephen Wolfram[1] is pretty good as an introduction, but I haven't seen any super comprehensive material that tries do dissect all the interesting behaviour we see in the really big llms. For that just reading the relevant papers themselves has been pretty fruitful for me. Some of them are actually very well written, even if you aren't used to reading scientific papers. I can recommend the Sparks of AGI paper[2] and the toolformer paper[3]. Obviously there's much more out there, those three things are a pretty good read. [1]: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... [2]: https://arxiv.org/abs/2303.12712 [3]: https://arxiv.org/abs/2302.04761

It does. This is the plugins methodology described in the toolformers paper which I've linked elsewhere[1]. The model learns that for certain types of problems certain specific "tools" are the best way to solve the problem. The problem is of course it's simple to argue that the LLM learns to use the tool(s) and can't reason itself about the underlying problem. The question boils down to whether you're more interested in machines which can think (whatever that means) or having a super-powered co-pilot which can help with a wide variety of tasks. I'm quite biased towards the second so I have the wolfram alpha plugin enabled in my chat gpt. I can't say it solves all the math-related hallucinations I see but I might not be using it right. [1] But here it is again https://arxiv.org/abs/2302.04761

It should be in line with the likes of this - https://arxiv.org/abs/2302.04761 or https://twitter.com/minosvasilias/status/1627076214639976449 I didn't say guardrail to mean anything particularly elaborate. The pre-prompt for Bing has instructions to avoid adversarial arguments.

This document seems to have been written before the toolformer paper[0], which fine tunes the model to use tools (e.g search) to retrieve information. [0]: https://arxiv.org/abs/2302.04761

> LLMs can't to my knowledge process a request into a lookup on say an actual database of facts at the moment or parse a request into API actions. They can: https://arxiv.org/abs/2302.04761 > In this paper, we show that LMs can teach themselves to use external tools

Bing chat already hooks into a web search tool. It shows "Searching for: ..." when it does that. This is kind of the point of Bing chat. edit: a paper about how to hook up LLMs to any external tool https://arxiv.org/abs/2302.04761

Since the GPT-4 paper didn't reveal any details of their model, it's possible that GPT-4 is already using some variant of toolformer (https://arxiv.org/abs/2302.04761).

He likely meant techniques such as ToolFormer[1] where the language model will output a "request" (in some text syntax) that another system can parse, run and report back. [1] https://arxiv.org/abs/2302.04761

That's called Toolformer (Facebook AI Research): https://arxiv.org/abs/2302.04761 Toolformer: Language Models Can Teach Themselves to Use Tools

Here you go, here's a bunch of papers that you have not read. If you had read them then you would know that the errors are predictable and therefore there are many measurable ways to make improvements. Toolformer: Language Models Can Teach Themselves to Use Tools: https://arxiv.org/abs/2302.04761 PAL: Program-aided Language Models: https://arxiv.org/abs/2211.10435 TALM: Tool Augmented Language Models: https://arxiv.org/abs/2205.12255 Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: https://arxiv.org/abs/2201.11903 Survey of Hallucination in Natural Language Generation: https://arxiv.org/abs/2202.03629

Hooking ChatGPT (and similar architectures) into applications is going to be the next big thing. It just takes time to hook up various systems. Eventually a vision module will be integrated and it will probably be able to "figure out" most systems. There's a lot of money to be made in hooking up these "brains" into apps. I hope people get on it, since it will be quite convenient. See: Act-1: https://news.ycombinator.com/item?id=32842860 Open-Assistant: https://github.com/LAION-AI/Open-Assistant Toolformer: https://arxiv.org/abs/2302.04761 Understanding HTML with LLMs: https://arxiv.org/abs/2210.03945

You can make small LLMs more performant than large ones like GPT-3 by fine-tuning on specific tasks or providing them tools to offload precise calculations: e.g. Toolformer: https://arxiv.org/abs/2302.04761 which uses APIs and functions to improve GPT-J beyond GPT-3 for various tasks

There's research being done on this: https://arxiv.org/abs/2302.04761 At its core using an LM alone to solve factual problems seems silly: It's not unlike asking Dall-E to draw DOT compliant road signs. I've gone at length at how unfortunate it would be if LMs start to get a bad rap because they're being shoehorned into being "Ask Jeeves 2.0" when they could be so much more.

They literally "do context", as they do in-context learning. Also, factuality can effectively be solved by chain-of-thought prompting[0] and the use of external tools[1], also see toolformerzero[2] for a simple example. [0]: https://arxiv.org/abs/2201.11903 [1]: https://arxiv.org/abs/2302.04761 [2]: https://toolformerzero.com/ [2]: https://toolformerzero.com/

In their current iteration the models are very neutered. It’s been demonstrated that GPT models are fairly good at choosing when to perform a task. Obviously lots of APIs and machinery is needed to actually perform tasks, but the heavy lifting “intelligence” portion can be almost entirely performed by our existing models. Some basic text based APIs that would quickly improve LLM utility: Calculators Database storage and retrieval Web access (already kind of done by bing) Shell scripting Thinking further into the future of multimodal models, it’s not hard to imagine this sort of thing could be extended to include image based APIs. Imagine a LLM looking at your gui and clicking on things. The sky’s the limit at that point. Checkout toolformer, they’ve got this mostly working with a much smaller model than gpt3.5. https://arxiv.org/abs/2302.04761

Relevant to this discussion is the Toolformer paper that was just released today from Meta AI. To me the exciting result from the paper is that it appears that much smaller models can be used to translate human intent into actions: https://arxiv.org/abs/2302.04761

Nothing, and there are methods which allow these types of models to learn to use special purpose tools of this kind[1]. [1] https://arxiv.org/abs/2302.04761 Toolformer: Language Models Can Teach Themselves to Use Tools

That’s sort of what happens in the “toolformer” paper[1] which sets out the basic architecture by which things like chatGPT plugins work. Language models can learn to use “tools” provided by plugins. Those tools might themselves bee the specialized models you are describing (although for some plugins they could be a non-AI tool such as web search or whatever). [1] https://arxiv.org/pdf/2302.04761.pdf