Nonstrict hierarchical reinforcement learning for interactive systems and robots

Cuayahuitl, Heriberto, Kruijff-Korbayová, Ivana and Dethlefs, Nina (2014) Nonstrict hierarchical reinforcement learning for interactive systems and robots. ACM Transactions on Interactive Intelligent Systems (TiiS), 4 (3). p. 15. ISSN 2160-6455

Full content URL:

hc-acm-trans-tiis2014.pdf - Whole Document

Item Type:Article
Item Status:Live Archive


Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.

Keywords:Machine Learning, Industrial engineering, Interactive robots, Evaluation, NotOAChecked
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
Related URLs:
ID Code:22211
Deposited On:13 Feb 2016 19:08

Repository Staff Only: item control page