Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions

Del Duchetto, Francesco and Hanheide, Marc (2022) Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions. IEEE Robotics and Automation Letters, 7 (3). pp. 6934-6941. ISSN 2377-3766

Full content URL: https://doi.org/10.1109/LRA.2022.3178807

Learning on the Job: Long-Term Behavioural Adaptation in Human-Robot Interactions
Author's accepted manuscript
IROS22_Lindsey_RL(2).pdf - Whole Document
Available under License Creative Commons Attribution 4.0 International.

Item Type:Article
Item Status:Live Archive


In this work, we propose a framework for allowing autonomous robots deployed for extended periods of time in public spaces to adapt their own behaviour online from user interactions. The robot behaviour planning is embedded in a Reinforcement Learning (RL) framework, where the objective is maximising the level of overall user engagement during the interactions. We use the Upper-Confidence-Bound Value-Iteration (UCBVI) algorithm, which gives a helpful way of managing the exploration-exploitation trade-off for real-time interactions. An engagement model trained end-to-end generates the reward function in real-time during policy execution. We test this approach in a public museum in Lincoln (U.K.), where the robot is deployed as a tour guide for the visitors. Results show that after a couple of months of exploration, the robot policy learned to maintain the engagement of users for longer, with an increase of 22.8% over the initial static policy in the number of items visited during the tour and a 30% increase in the probability of completing the tour. This work is a promising step toward behavioural adaptation in long-term scenarios for robotics applications in social settings.

Keywords:HRI, reinforcement learning, art andentertainment robotics
Subjects:G Mathematical and Computer Sciences > G440 Human-computer Interaction
H Engineering > H671 Robotics
G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
ID Code:49961
Deposited On:05 Jul 2022 12:48

Repository Staff Only: item control page