Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization

Polvara, Riccardo, Patacchiola, Massimiliano, Hanheide, Marc and Neumann, Gerhard (2020) Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization. Robotics, 9 (1). ISSN 2218-6581

Full content URL: https://doi.org/10.3390/robotics9010008

Documents
Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization
Published Open Access manuscript
[img]
[Download]
[img]
Preview
PDF
Sim-to-Real Quadrotor Landing via Sequential Deep Q-Networks and Domain Randomization.pdf - Whole Document
Available under License Creative Commons Attribution 4.0 International.

5MB
Item Type:Article
Item Status:Live Archive

Abstract

The autonomous landing of an Unmanned Aerial Vehicle (UAV) on a marker is one of the most challenging problems in robotics. Many solutions have been proposed, with the best results achieved via customized geometric features and external sensors. This paper discusses for the first time the use of deep reinforcement learning as an end-to-end learning paradigm to find a policy for UAVs autonomous landing. Our method is based on a divide-and-conquer paradigm that splits a task into sequential sub-tasks, each one assigned to a Deep Q-Network (DQN), hence the name Sequential Deep Q-Network (SDQN). Each DQN in an SDQN is activated by an internal trigger, and it represents a component of a high-level control policy, which can navigate the UAV towards the marker. Different technical solutions have been implemented, for example combining vanilla and double DQNs, and the introduction of a partitioned buffer replay to address the problem of sample efficiency. One of the main contributions of this work consists in showing how an SDQN trained in a simulator via domain randomization, can effectively generalize to real-world scenarios of increasing complexity. The performance of SDQNs is comparable with a state-of-the-art algorithm and human pilots while being quantitatively better in noisy conditions.

Keywords:Deep Reinforcement Learning, Landing of Unmanned Aerial Vehicle, Unmanned Aerial Vehicles, Aerial Vehicles, Sim-to-Real
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
H Engineering > H671 Robotics
Divisions:College of Science > School of Computer Science
ID Code:40216
Deposited On:18 Mar 2020 09:45

Repository Staff Only: item control page