A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning

Mohtasib, Abdalkarim, Neumann, Gerhard and Cuayahuitl, Heriberto (2021) A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning. In: Towards Autonomous Robotic Systems Conference (TAROS).

Full content URL: https://lcas.lincoln.ac.uk/wp/wp-content/uploads/2...

Documents
A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learning
[img]
[Download]
[img]
Preview
PDF
TAROS_2021colour.pdf - Whole Document

2MB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

Deep Reinforcement Learning (DRL) is a promising approach for teaching robots new behaviour. However, one of its main limitations is the need for carefully hand-coded reward signals by an expert. We argue that it is crucial to automate the reward learning process so that new skills can be taught to robots by their users. To address such automation, we consider task success classifiers using visual observations to estimate the rewards in terms of task success. In this work, we study the performance of multiple state-of-the-art deep reinforcement learning algorithms under different types of reward: Dense, Sparse, Visual Dense, and Visual Sparse rewards. Our experiments in various simulation tasks (Pendulum, Reacher, Pusher, and Fetch Reach) show that while DRL agents can learn successful behaviours using visual rewards when the goal targets are distinguishable, their performance may decrease if the task goal is not clearly visible. Our results also show that visual dense rewards are more successful than visual sparse rewards and that there is no single best algorithm for all tasks.

Keywords:Deep Reinforcement Learning, Reward Learning, Robot Learning
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G760 Machine Learning
H Engineering > H671 Robotics
Divisions:College of Science > School of Computer Science
ID Code:45983
Deposited On:23 Sep 2021 10:29

Repository Staff Only: item control page