Spatially-aware dialogue control using hierarchical reinforcement learning

Cuayahuitl, Heriberto (2011) Spatially-aware dialogue control using hierarchical reinforcement learning. ACM Transactions on Speech and Language Processing (TSLP), 7 (3). ISSN 1550-4875

Full content URL:


Request a copy
[img] PDF
hc-acm-trans-tslp2011.pdf - Whole Document
Restricted to Repository staff only

Item Type:Article
Item Status:Live Archive


This article addresses the problem of scalable optimization for spatially-aware dialogue systems. These kinds of systems must perceive, reason, and act about the spatial environment where they are embedded. We formulate the problem in terms of Semi-Markov Decision Processes and propose a hierarchical reinforcement learning approach to optimize subbehaviors rather than full behaviors. Because of the vast number of policies that are required to control the interaction in a dynamic environment (e.g., a dialogue system assisting a user to navigate in a building from one location to another), our learning approach is based on two stages: (a) the first stage learns low-level behavior, in advance; and (b) the second stage learns high-level behavior, in real time. For such a purpose we extend an existing algorithm in the literature of reinforcement learning in order to support reusable policies and therefore to perform fast learning. We argue that our learning approach makes the problem feasible, and we report on a novel reinforcement learning dialogue system that performs a joint optimization between dialogue and spatial behaviors. Our experiments, using simulated and real environments, are based on a text-based dialogue system for indoor navigation. Experimental results in a realistic environment reported an overall user satisfaction result of 89%, which suggests that our proposed approach is attractive for its application in real interactions as it combines fast learning with adaptive and reasonable behavior.

Keywords:Reinforcement learning, Dialogue Management, Indoor wayfinding, Evaluation
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G440 Human-computer Interaction
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
ID Code:22209
Deposited On:12 Feb 2016 08:52

Repository Staff Only: item control page