Skip to content
The methodOpen data
Back to the collection

A Monte Carlo AIXI Approximation

Joel Veness, Kee Siong Ng, Marcus Hutter, William Uther, David Silver

2009 · arXiv

Statistics & probabilityAlgorithmsMachine learning
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. Our approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcement learning agents. Previously, it has been unclear whether the theory of AIXI could motivate the design of practical algorithms. We answer this hitherto open question in the affirmative, by providing the first computationally feasible approximation to the AIXI agent. To develop our approximation, we introduce a new Monte-Carlo Tree Search algorithm along with an agent-specific extension to the Context Tree Weighting algorithm. Empirically, we present a set of encouraging results on a variety of stochastic and partially observable domains. We conclude by proposing a number of directions for future research.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

tlb
Neutral mention · rule
The human mind isn't exactly Solomonoff, because we often hold theories that are more complicated than necessary. Working towards simpler scientific or mathematical theories takes decades. So it seems like we don't just have SI in the inner loop. People have made plausible attempts to approximate SI with practical amounts of computing, like AIXI (https://arxiv.org/abs/0909.0801).
Original comment The impossibility of intelligence explosion
orangecat
Neutral mention · rule
If you apply Ockham's razor in the form "prefer hypotheses with lower Kolmogorov complexity" together with Bayes' theorem, you get something that in a certain (rather artificial) sense is as effective a problem solver as anything mechanized can be. Unfortunately it's unimplementable in practice, because ... Coming across AIXI was what got me interested in algorithmic information theory. It is indeed useless in it's "pure" form, but there's a variation using Monte Carlo search that apparently does pretty well: http://arxiv.org/abs/0909.0801 (This is controversial, but it's my opinion and that of at least some other contributors here.) When assessing (e.g.) scientific…
Read the full comment
hypotheses for "simplicity" or "parsimony", something like Kolmogorov complexity is the Right Way to understand (un)simplicity; failure to appreciate this is, e.g., what makes some people dislike the "many worlds" interpretation of quantum mechanics on the grounds of extravagance, or think that "God did it" is a useful explanation for otherwise-surprising features of the universe. Yes. In fact after reading the Less Wrong quantum physics sequence (http://lesswrong.com/lw/r5/the_quantum_physics_sequence/), I find that "the wavefunction collapsed" and "God did it" are remarkably similar statements.
Original comment The Single Sign On

Keep following the thread.

More in Statistics & probability

More from the same topics.