Cuayahuitl, Heriberto, Kruijff-Korbayová, Ivana and Dethlefs, Nina (2014) Nonstrict hierarchical reinforcement learning for interactive systems and robots. ACM Transactions on Interactive Intelligent Systems (TiiS), 4 (3). p. 15. ISSN 2160-6455
Full content URL: https://doi.org/10.1145/2659003
Documents |
|
|
PDF
hc-acm-trans-tiis2014.pdf - Whole Document 1MB |
Item Type: | Article |
---|---|
Item Status: | Live Archive |
Abstract
Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.
Keywords: | Machine Learning, Industrial engineering, Interactive robots, Evaluation, NotOAChecked |
---|---|
Subjects: | G Mathematical and Computer Sciences > G700 Artificial Intelligence G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing G Mathematical and Computer Sciences > G760 Machine Learning |
Divisions: | College of Science > School of Computer Science |
Related URLs: | |
ID Code: | 22211 |
Deposited On: | 13 Feb 2016 19:08 |
Repository Staff Only: item control page