Skip to content
The methodOpen data
Back to the collection

Relational inductive biases, deep learning, and graph networks

Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst

Show all 27 authors

Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicolas Heess, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu

2018 · arXiv

PsychologyMachine learningDatabases
Read the paper
HN score
0
Mentions
3
Readers
3
Months
3
What’s in a score?

From the abstract

Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. The following is part position paper, part review, and part unification. We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. Just as biology uses nature and nurture cooperatively, we reject the false choice between "hand-engineering" and "end-to-end" learning, and instead advocate for an approach which benefits from their complementary strengths. We explore how using relational inductive biases within deep learning architectures can facilitate learning about entities, relations, and rules for composing them. We present a new building block for the AI toolkit with a strong relational inductive bias--the graph network--which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. We discuss how graph networks can support relational reasoning and combinatorial generalization, laying the foundation for more sophisticated, interpretable, and flexible patterns of reasoning. As a companion to this paper, we have released an open-source software library for building graph networks, with demonstrations of how to use them in practice.

Metadata: arXiv. Abstracts retain their original authors’ rights.

In the conversation

Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels

3 selected excerpts · 3 total mentions

taliesinb
Neutral mention · rule
Hasselt and co, who authored the original Rainbow DQN model that was bested by SimPLe (this paper), responded recently with "When to use parametric models in reinforcement learning?" [1], which is an interesting read. Punchline: "We explain why their [SimPLe's] results may perhaps be considered surprising, and show that in a like-for-like comparison Rainbow DQN outperformed the scores of the model-based agent, with less experience and computation.". In other words, in an apples-to-apples comparison, model-based RL have not been shown to be more sample efficient than suitably optimized DQNs. Hasselt argues this is because if you need accurate updates to your Q values, you…
Read the full comment
need to trust the learned model you are using for simulated rollouts on the states on which you are sampling. But if your simulation model is trustworthy on these states, it is because it saw a lot of real transitions from these states from the actual environment. But then you might as well just have stored those transitions in a big enough replay buffer and use ordinary Q-learning with experience replay. And this indeed seems to be the case: when you give Rainbow DQN a nice big replay buffer, it is more sample efficient (both real and imagined samples) than SimPLe. Hasselt leaves some wiggle room for learned models to help with action selection and credit assignment, though. My counterargument to this (supplied with zero evidence of course!) would be that with the right inductive biases, a learned model can generalize quite accurately and with very few seen transitions, and hence be so sample efficient that it would outperform the replay memory approach. I'd imagine that the kinds of inductive biases that are appropriate for a varied meta-environment like Atari are quite general things like 'visually localized objects typically only interact when they approach or touch each other', and 'the arrow keys likely control one localized object'. There are approaches for how to encode such priors; [2] is a good survey paper, and [3] employs some of these ideas for RL. Moreover, these are the kinds of priors that one imagines are encoded or biased towards by evolution in actual animal brains. [1] https://arxiv.org/abs/1906.05243 [2] https://arxiv.org/abs/1806.01261 [3] https://arxiv.org/abs/1806.01830
Original comment Model-Based Reinforcement Learning for Atari
mlthrowaway1953
Neutral mention · rule
Is the Fields Medal going to still be relevant in 10 years, when most of the the major mathematical discoveries are made by deep learning and deep reinforcement learning systems? Already systems are learning to reason about concepts [1] and of course there is classical work on proof checkers [2]. It's very likely that the 2028 Fields medal will be awarded to a programmer, not some mathematical super-genius (assuming that the committee is fair, and not biased against machines). [1] https://arxiv.org/abs/1806.01261 [2] https://en.wikipedia.org/wiki/Four_color_theorem
Original comment 2018 Fields Medal and Nevanlinna Prize Winners
blueyes
Neutral mention · rule
Chatbots proposed to solve a problem that current technology is bad at: understanding natural language. Very few experts claimed that algorithms could master the combinatorics of natural language beyond a few very narrow domains. On top of the technical risk, the voice and chat UX does not give a lot of visibility to users, as this post points out. What happened was: algorithmic advances in other areas lead to analogous reasoning by non-experts whose self-promotion aligned with the media's need for something new, big and understandable. And then it failed. The irony, of course, is that research is making strides in NLU, it's just too late for the last wave of chatbots. Here are two recent papers from DeepMind: Learning to Follow Language Instructions with Adversarial Reward Induction https://arxiv.org/abs/1806.01946 Relational inductive biases, deep learning, and graph networks https://arxiv.org/abs/1806.01261
Original comment Chatbots were the next big thing: what happened?

Keep following the thread.

More in Psychology

More from the same topics.

2017arXiv

The Case for Learned Index Structures

Tim Kraska, Alex Beutel, Ed H. Chi +2

Indexes are models: a B-Tree-Index can be seen as a model to map a key to the position of a record within a sorted array, a Hash-Index as a model to map a key to a position of a record within an unsorted array, and a BitMap-Index as a model to indicate if a data record exists or not.

Databases
19 mentions14 months
5HN score