Continuous control with deep reinforcement learning

Stagnation is a known problem in reinforcement learning and similar methods. It's very easy to get stuck at a local maximum. My favorite fun example is https://gym.openai.com/envs/BipedalWalkerHardcore-v2/ where a standard DDPG(https://arxiv.org/abs/1509.02971) will get stuck at pits in the environment. Although it could get a higher score if it learned to jump, there is a penalty with falling in that makes it stabilize on standing still and running out the timer. Video: https://www.youtube.com/watch?v=DEGwhjEUFoI I suspect there is something similar going on with video/music recommendations. When a bad novel suggestion is made the penalty is likely too high to overcome (User immediately clicks off) with traditional reinforcement methods.

...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy: http://arxiv.org/abs/1509.02971 The fact that the (reinforcement) learning problem is hard or not is not directly related to whether the observation and action spaces are discrete or continuous.

It's handy knowing they're fiddling with ROS, and the research papers show use of DNN&RL for robotics via simulated tasks with actions keying off of raw pixel inputs (paper: http://arxiv.org/pdf/1509.02971v1.pdf), and grasp paper shows ambitions on dexterous manipulation. I think valuable to have some idea of the research areas. I'm biased because I wrote the article, though! Aiming to produce a more granular article as soon as I can report it out.