Tree of Thoughts: Deliberate Problem Solving with Large Language Models

I default to immediately asking GPT4 to review its solution and fix any mistakes it finds. There’s also an interesting paper about providing it guidance that it can make a “tree of thoughts” which allows it to move forward and backwards as it comes up with solutions then present its best solution to you. The paper suggests you can squeeze a lot more performance out of LLM’s (even smaller ones) this way. I’ve been wanting to experiment with my prompting in this fashion: https://arxiv.org/pdf/2305.10601.pdf

I didn't compare with the harness (focused on distillation) but the original ToT paper has a section on it: https://arxiv.org/abs/2305.10601

I have. The failure is with you and your extracting only the mechanical functionality of LLMs as all an LLM is. The high level macro structure of what ends up being trained is something nobody understands. Regardless of WHAT I say, the general consensus among academia and professionals is extremely different from what you characterize. I can cite dozens of research papers contrary to your point in a simple google search: https://www.sciencedirect.com/science/article/pii/S016028962... https://www.topbots.com/llm-reasoning-research-papers/ https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2205.10625 https://arxiv.org/abs/2205.15241 The fact that these research papers exist is testament against your delusional claim that you completely understand how LLMs work just by reading Attention is all you need.

This is not even remotely close and very silly. A ChainOfThought in a loop. TreeOfThoughts is a more sophisticated method, see - https://arxiv.org/pdf/2305.10601 The clue we all had with OpenAI for a long time that this was a search through a tree, they hired Noam Brown, and his past work all hinted towards that. Q, is obviously a search on a tree like A. So take something like CoT, build out a tree, search for the best solution across it. The search is the "system-2 reasoning"

On backtracking, I thought tree-of-thought enabled that? "considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices" https://arxiv.org/abs/2305.10601 Generally with you though, this thing is not leading to real smarts and that's accepted by many. Yes, it'll fill in a few gaps with exponentially more compute but it's more likely that an algo change is required once we've maxed out LLM's.

I'm really excited about Gemini. It's supposed to have some sort of planning and problem solving ability[2][3]. If it is like Tree of Thoughts [1], then I'm so hyped! Also take a peek at which institution most of the ToT authors are at. 1: https://arxiv.org/pdf/2305.10601.pdf 2: https://blog.google/technology/ai/google-io-2023-keynote-sun... 3: https://www.wired.com/story/google-deepmind-demis-hassabis-c...

Chain of thought: https://arxiv.org/abs/2201.11903 ReAct: https://arxiv.org/abs/2210.03629 Reflexion: https://arxiv.org/abs/2303.11366 Tree of thoughts: https://arxiv.org/abs/2305.10601 Good video on "Tree of thoughts" which also reviews / puts it in the context of other methods: https://www.youtube.com/watch?v=ut5kp56wW_4 Completion vs conversational interface is something you can read about in the OpenAI API documentation. For the remaining things I don't have single specific pointer at hand.

I don't know how far "pure" next-token prediction can go with planning, although I wouldn't count them out until their performance starts noticeably plateauing. But the tree of thought architecture is a very similar concept to what you're discussing, you should definitely give it a read: https://arxiv.org/abs/2305.10601 It's not everything involved in traditional planning, but it may be a framework to use more traditional planning algorithms on LLM output.

> The goal of stack exchange is to create a repository of question answer pairs of knowledge. The problem here is that generational AI isn't able to create these new pairs. I'm not so sure about that. I think we are quite close to an AI being able to put two and two together to create something marginally new. Maybe not an LLM by itself, but with some high-level iterative/recursive process like this: https://arxiv.org/pdf/2305.10601.pdf. What I'm saying is that no-AI policy maybe made sense in late 2022, but will not necessarily hold in late 2023.

Note that the repo author != the paper author. The research itself [1] seems legit. The paper author also wrote a paper called ReAct [2], which is one of the core components of the langchain framework. * [1] https://arxiv.org/abs/2305.10601 * [2] https://arxiv.org/abs/2210.03629