Policy search for path integral control

Gomez, Vincenc and Kappen, Hilbert J. and Peters, Jan and Neumann, Gerhard (2014) Policy search for path integral control. In: Machine Learning and Knowledge Discovery in Databases - European Conference, ECML/PKDD 2014, 15 - 19 September 2014, Nancy, France.

Documents
pireps_ECML (2).pdf
[img]
[Download]
[img]
Preview
PDF
pireps_ECML (2).pdf - Whole Document

691kB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

Path integral (PI) control defines a general class of control problems for which the optimal control computation is equivalent to an inference problem that can be solved by evaluation of a path integral over state trajectories. However, this potential is mostly unused in real-world problems because of two main limitations: first, current approaches can typically only be applied to learn open-loop controllers and second, current sampling procedures are inefficient and not scalable to high dimensional systems. We introduce the efficient Path Integral Relative-Entropy Policy Search (PI-REPS) algorithm for learning feedback policies with PI control. Our algorithm is inspired by information theoretic policy updates that are often used in policy search. We use these updates to approximate the state trajectory distribution that is known to be optimal from the PI control theory. Our approach allows for a principled treatment of different sampling distributions and can be used to estimate many types of parametric or non-parametric feedback controllers. We show that PI-REPS significantly outperforms current methods and is able to solve tasks that are out of reach for current methods.

Keywords:Policy Search, Path Integrals, Optimal Control
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
Related URLs:
ID Code:25770
Deposited On:06 Apr 2017 13:55

Repository Staff Only: item control page