Hierarchical reinforcement learning for situated language generation

Dethlefs, Nina and Cuayahuitl, Heriberto (2014) Hierarchical reinforcement learning for situated language generation. Natural Language Engineering, 21 (3). pp. 391-435. ISSN 1351-3249


Request a copy
[img] PDF
hc-nle2014.pdf - Whole Document
Restricted to Repository staff only

Item Type:Article
Item Status:Live Archive


Natural Language Generation systems in interactive settings often face a multitude of choices, given that the communicative effect of each utterance they generate depends crucially on the interplay between its physical circumstances, addressee and interaction history. This is particularly true in interactive and situated settings. In this paper we present a novel approach for situated Natural Language Generation in dialogue that is based on hierarchical reinforcement learning and learns the best utterance for a context by optimisation through trial and error. The model is trained from human–human corpus data and learns particularly to balance the trade-off between efficiency and detail in giving instructions: the user needs to be given sufficient information to execute their task, but without exceeding their cognitive load. We present results from simulation and a task-based human evaluation study comparing two different versions of hierarchical reinforcement learning: One operates using a hierarchy of policies with a large state space and local knowledge, and the other additionally shares knowledge across generation subtasks to enhance performance. Results show that sharing knowledge across subtasks achieves better performance than learning in isolation, leading to smoother and more successful interactions that are better perceived by human users.

Keywords:Reinforcement learning, Language generation, Situated interaction, Evaluation, NotOAChecked
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G760 Machine Learning
G Mathematical and Computer Sciences > G440 Human-computer Interaction
Divisions:College of Science > School of Computer Science
Related URLs:
ID Code:22213
Deposited On:12 Feb 2016 09:22

Repository Staff Only: item control page