Benjamin Recht
2018 · arXiv
This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best-studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and control might be combined to approach these challenges.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
4 selected excerpts · 4 total mentions
Ben Recht [0] has worked on control theory and reinforcement learning. See his blog post "What we've Learned to control" [1] and the survey paper "A Tour of Reinforcement Learning: The View from Continuous Control" [2] may be of interest. [0]: https://people.eecs.berkeley.edu/~brecht/ [1]: https://www.argmin.net/2020/06/29/tour-revisited/ [2]: https://arxiv.org/abs/1806.09460
If you have a good model, and can use model-based optimal control which has been understood for decades, then that is good but there's also not really a research problem? You can just do the simple, robust thing and it will work great. (i.e. "to publish more papers" is actually a legitimate reason if your job is explicitly to publish papers) You may enjoy the article, "A Tour of Reinforcement Learning: The View from Continuous Control". At least that researcher would agree that people doing RL don't pay enough attention to "classical" control. https://arxiv.org/abs/1806.09460
for more on that subject - check out this recent RL and OCT survey by Ben Recht, also from UC Berkeley: https://arxiv.org/abs/1806.09460
Ben's blog series culminated in a nice article[1] touring reinforcement learning. He also held a tutorial on the topic at ICML[2]. They might address some of your concerns. [1]: https://arxiv.org/abs/1806.09460 [2]: https://www.facebook.com/icml.imls/videos/428757614305426
More from the same topics.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.
Chiyuan Zhang, Samy Bengio, Moritz Hardt +2
Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small difference between training and test performance.
A. Pluchino. A. E. Biondo, A. Rapisarda
The largely dominant meritocratic paradigm of highly competitive Western cultures is rooted on the belief that success is due mainly, if not exclusively, to personal qualities such as talent, intelligence, skills, efforts or risk taking.