Neutral mention · rule
There were some interesting talks on neural differentiation applied to physics at ICLR. You probably saw: https://arxiv.org/abs/1906.01563 Very fun!
Back to the collection
Hamiltonian Neural Networks
Sam Greydanus, Misko Dzamba, Jason Yosinski
2019 · arXiv
- HN score
- 0
- Mentions
- 3
- Readers
- 3
- Months
- 3
From the abstract
Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we draw inspiration from Hamiltonian mechanics to train models that learn and respect exact conservation laws in an unsupervised manner. We evaluate our models on problems where conservation of energy is important, including the two-body problem and pixel observations of a pendulum. Our model trains faster and generalizes better than a regular neural network. An interesting side effect is that our model is perfectly reversible in time.
Metadata: arXiv. Abstracts retain their original authors’ rights.
In the conversation
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
Neutral mention · rule
Hamiltonian NNs are not a new thing. There was a NIPS 2019 paper [0] that attempted to do that same for some toy problems. In general the idea of including model or context-based information into neural networks goes along the line of Kahneman's System I and System II of the human mind. System I is the "emotional" brain that is fast and makes decisions quickly while System II is the "rational" brain that is slow and expensive and takes time to compute a response. Researchers have been trying to develop ML models that utilize this dichotomy by building corresponding dual modules but the major challenge remains in efficiently embedding the assumptions of the world dynamics into the models. [0] https://arxiv.org/abs/1906.01563 [1] https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
Neutral mention · rule
It's actually extraordinarily difficult to get a DNN model to accurately predict planetary motion over arbitrary time horizons. RNNs have a tendency to either decay or blow-up. The only way I know to make neural nets work well for this problem is to build in lots of physics into the model architecture, e.g., conservation of energy: https://arxiv.org/abs/1906.01563
Keep following the thread.
More in Machine learningMore from the same topics.
2023arXiv
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
60 mentions14 months
3.5HN score
2020arXiv
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
83 mentions35 months
3HN score
2016arXiv
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
16 mentions14 months
2.5HN score
2018arXiv
Talent vs Luck: the role of randomness in success and failure
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.
16 mentions14 months
2.5HN score