A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents

Cuayahuitl, Heriberto and Ryu, Seonghan and Lee, Donghyeon and Kim, Jihie (2018) A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents. In: NeurIPS Workshop on Conversational AI, 7 December 2018, Montreal, Canada.

Documents
A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents [Conference or Workshop contribution]
[img]
[Download]
[img] PDF
dialogue-history-agent-final-final.pdf - Whole Document

756kB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

The amount of dialogue history to include in a conversational agent is often underestimated and/or set in an empirical and thus possibly naive way. This suggests that principled investigations into optimal context windows are urgently needed given that the amount of dialogue history and corresponding representations can play an important role in the overall performance of a conversational system. This paper studies the amount of history required by conversational agents for reliably predicting dialogue rewards. The task of dialogue reward prediction is chosen for investigating the effects of varying amounts of dialogue history and their impact on system performance. Experimental results using a dataset of 18K human-human dialogues report that lengthy dialogue histories of at least 10 sentences are preferred (25 sentences being the best in our experiments) over short ones, and that lengthy histories are useful for training dialogue reward predictors with strong positive correlations between target dialogue rewards and predicted ones.

Keywords:Chatbots, Reward Function Learning, Recurrent Neural Networks, Regression
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
G Mathematical and Computer Sciences > G730 Neural Computing
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G700 Artificial Intelligence
Divisions:College of Science > School of Computer Science
ID Code:34433
Deposited On:11 Dec 2018 16:05

Repository Staff Only: item control page