Layered direct policy search for learning hierarchical skills

End, F., Akrour, R., Peters, J. and Neumann, G. (2017) Layered direct policy search for learning hierarchical skills. In: International Conference on Robotics and Automation (ICRA), 29 May - 3 June 2017, Marina Bay Sands in Singapore.

Full content URL:

icra17_felix.pdf - Whole Document

Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive


Solutions to real world robotic tasks often require
complex behaviors in high dimensional continuous state and
action spaces. Reinforcement Learning (RL) is aimed at learning
such behaviors but often fails for lack of scalability. To
address this issue, Hierarchical RL (HRL) algorithms leverage
hierarchical policies to exploit the structure of a task. However,
many HRL algorithms rely on task specific knowledge such
as a set of predefined sub-policies or sub-goals. In this paper
we propose a new HRL algorithm based on information
theoretic principles to autonomously uncover a diverse set
of sub-policies and their activation policies. Moreover, the
learning process mirrors the policys structure and is thus also
hierarchical, consisting of a set of independent optimization
problems. The hierarchical structure of the learning process
allows us to control the learning rate of the sub-policies and
the gating individually and add specific information theoretic
constraints to each layer to ensure the diversification of the subpolicies.
We evaluate our algorithm on two high dimensional
continuous tasks and experimentally demonstrate its ability to
autonomously discover a rich set of sub-policies.

Keywords:Robotics, Motor Skill Learning, Hierarchical Policy Search
Subjects:H Engineering > H671 Robotics
G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
Related URLs:
ID Code:26737
Deposited On:17 Mar 2017 15:12

Repository Staff Only: item control page