Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein
Tianyu Wang, Darren T. Schachter, Zoey Hu, Peter L. McMahon
2022 · Nature
Abstract Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability 1 . Deep-learning accelerators 2–9 aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far 10–22 have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ–in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics 23–26 , materials 27–29 and smart sensors 30–32 .
Metadata: Crossref. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
3 selected excerpts · 3 total mentions
Very interesting to see how work in this space is progressing. Reminds me of a Nature paper a few years back about using physical systems as neural networks: Deep physical neural networks trained with backpropagation (2022) https://www.nature.com/articles/s41586-021-04223-6
Premise 1: Any arbitrary 4D volume in spacetime (i.e., one connected component wrt physical interactions) can be considered a system. Premise 2: https://xkcd.com/505/ Conclusion: Every system, open or closed, is computational in a certain rigorous sense of implementing dynamics along the space dimensions (versus inputs, via transformations, into outputs) and demonstrating dynamics along the time dimension (flux of the energy+information carried through its constituent parts). If we can create a "computer" using rocks in a binary-ish encoding, surely we can implement "computers" in more efficient encodings that utilize certain properties of physical reality. We as humans…
began to approach this with mechanical computers using complex linkages, but only got so far before we decided on digital (binary) encoding and then enshrined that in silicon. To be sure, binary is a very effective way of organizing computation given our physical implementation(s), but most likely there are way more efficient ways of doing things. One fascinating direction we are beginning to explore to achieve more efficient computation is marrying machine learning with materials engineering![1] In fact, by this conclusion, we can only believe that the universe is merely the most efficient way to simulate itself, but this by no means precludes the possibility of simulating it in any other medium (encoding). [1] https://www.nature.com/articles/s41586-021-04223-6
https://www.nature.com/articles/s41586-021-04223-6 Abstract Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability1. Deep-learning accelerators2,3,4,5,6,7,8,9 aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far10,11,12,13,14,15,16,17,18,19,20,21,22 have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural…
networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ–in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics23,24,25,26, materials27,28,29 and smart sensors30,31,32.
More from the same topics.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
Haoran Xu, Fredrik Kjolstad
Fast compilation is important when compilation occurs at runtime, such as query compilers in modern database systems and WebAssembly virtual machines in modern browsers.
Scott Aaronson
In honor of Alan Turing's hundredth birthday, I unwisely set out some thoughts about one of Turing's obsessions throughout his life, the question of physics and free will.