Guanzhi Wang, Yuqi Xie, Yunfan Jiang
Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar
2023 · arXiv
We introduce Voyager, the first LLM-powered embodied lifelong learning agent in Minecraft that continuously explores the world, acquires diverse skills, and makes novel discoveries without human intervention. Voyager consists of three key components: 1) an automatic curriculum that maximizes exploration, 2) an ever-growing skill library of executable code for storing and retrieving complex behaviors, and 3) a new iterative prompting mechanism that incorporates environment feedback, execution errors, and self-verification for program improvement. Voyager interacts with GPT-4 via blackbox queries, which bypasses the need for model parameter fine-tuning. The skills developed by Voyager are temporally extended, interpretable, and compositional, which compounds the agent's abilities rapidly and alleviates catastrophic forgetting. Empirically, Voyager shows strong in-context lifelong learning capability and exhibits exceptional proficiency in playing Minecraft. It obtains 3.3x more unique items, travels 2.3x longer distances, and unlocks key tech tree milestones up to 15.3x faster than prior SOTA. Voyager is able to utilize the learned skill library in a new Minecraft world to solve novel tasks from scratch, while other techniques struggle to generalize. We open-source our full codebase and prompts at https://voyager.minedojo.org/.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
6 selected excerpts · 6 total mentions
sounds like it could be many things. there was a well-known paper called Voyager by NASA in which an agent was able to write its own skills in the form of code and improve them over time. funnily enough this agent played minecraft, and its skills were to collect materials or craft things. https://arxiv.org/abs/2305.16291
I think LLM as a toolsmith like demonstrated in the Voyager paper (1) is another interesting approach to creating a system that can learn to do a task better over time. (1) https://arxiv.org/abs/2305.16291
This is very similar to what Voyager did https://arxiv.org/abs/2305.16291 Their implementation uses actual code, JS scripts in their case, as the stored trajectories, which has the neat feature of parameterization built in so trajectories are more reusable. I experimented with this for a bit for Muscle Mem, but a trajectory being performed by just-in-time generated scripts felt too magical and wild west. An explicit goal of Muscle Mem is to be a deterministic system, more like a DB, on which you as a user can layer as much nondeterminism as you feel comfortable with.
> but this is never learned permanently. Are you sure? I think "Open"AI uses the chat transcripts to help the next training run? > they are doomed to only learn what they were pre-trained with Fine-tuning. > The training data for this does not exist What does "this" refer to? Have you read the Voyager paper? (https://arxiv.org/abs/2305.16291) Any lesson learnt in the library could be used for fine-tuning or the next training run for a base model. > what would be needed would be the inner thoughts (predictions) of developers as they tackled a variety of tasks and were presented with various outcomes (feedback) continuously throughout the software development cycle Co-pilot…
gets to watch people figure stuff out - there's no reason that couldn't be used for the next version. Not only does it not need to read minds, but people go out of their way to write comments or chat messages to tell it what they think is going on and how to improve its code. > Days of human experience would take years to acquire And once learnt, that skill will never age, never get bored, never take annual leave, never go to the kids' football games, never die. It can be replicated as many millions of time as necessary. > they could train it to write crud apps To be fair, a lot of computer code is crud apps. But instead of learning it in one language, now it can do it in every language that existed on stackoverflow the day before its training run.
Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get the best results but you could potentially do what they did in the Voyager paper https://arxiv.org/abs/2305.16291 and have a pre-trained LLM just write the raw programs to control the robot and then save and re-use them with function calling. https://www.microsoft.com/en-us/research/group/autonomous-sy... https://www.microsoft.com/en-us/research/group/applied-robot... https://blogs.nvidia.com/blog/eureka-robotics-research/ https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to... https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-...
Just dropping this here since it looks like a lot of people haven't heard of it. Voyager: An Open-Ended Embodied Agent with Large Language Models https://arxiv.org/abs/2305.16291
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics.