Robot Policy Learning from Demonstration Using Advantage Weighting and Early Termination

Mohtasib, Abdalkarim, Neumann, Gerhard and Cuayahuitl, Heriberto (2022) Robot Policy Learning from Demonstration Using Advantage Weighting and Early Termination. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

Documents
Robot Policy Learning from Demonstration Using Advantage Weighting and Early Termination
Author's accepted manuscript
[img] PDF
am_IROS_2022.pdf - Whole Document
Restricted to Repository staff only

1MB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

Learning robotic tasks in the real world is still highly challenging and effective practical solutions remain to be found. Traditional methods used in this area are imitation learning and reinforcement learning, but they both have limitations when applied to real robots. Combining reinforcement learning with pre-collected demonstrations is a promising approach that can help in learning control policies to solve robotic tasks. In this paper, we propose an algorithm that uses novel techniques to leverage offline expert data using offline and online training to obtain faster convergence and improved performance. The proposed algorithm (AWET) weights the critic losses with a novel agent advantage weight to improve over the expert data. In addition, AWET makes use of an automatic early termination technique to stop and discard policy rollouts that are not similar to expert trajectories---to prevent drifting far from the expert data. In an ablation study, AWET showed improved and promising performance when compared to state-of-the-art baselines on four standard robotic tasks.

Keywords:robot learning, deep reinforcement learning, robot manipulation
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G760 Machine Learning
H Engineering > H671 Robotics
Divisions:College of Science > School of Computer Science
ID Code:50442
Deposited On:23 Aug 2022 10:30

Repository Staff Only: item control page