Cuayahuitl, Heriberto, Renals, Steve, Lemon, Oliver and Shimodaira, Hiroshi (2010) Evaluation of a hierarchical reinforcement learning spoken dialogue system. Computer Speech & Language, 24 (2). pp. 395-429. ISSN 0885-2308
Documents |
|
![]() |
PDF
hc-elsevier-csl2010.pdf - Whole Document Restricted to Repository staff only 2MB |
Item Type: | Article |
---|---|
Item Status: | Live Archive |
Abstract
We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ‘Precision-Recall’. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems.
Keywords: | Reinforcement learning, Dialogue systems, Spoken interaction, End-to-end evaluation |
---|---|
Subjects: | G Mathematical and Computer Sciences > G760 Machine Learning G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing G Mathematical and Computer Sciences > G790 Artificial Intelligence not elsewhere classified |
Divisions: | College of Science > School of Computer Science |
Related URLs: | |
ID Code: | 22208 |
Deposited On: | 14 Feb 2016 13:05 |
Repository Staff Only: item control page