Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards

Cuayahuitl, Heriberto and Lee, Donghyeon and Ryu, Seonghan and Choi, Sungja and Hwang, Inchul and Kim, Jihie (2019) Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards. In: International Joint Conference on Neural Networks (IJCNN).

Documents
Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards
Accepted Manuscript

Request a copy
[img] PDF
PID5854907.pdf - Whole Document
Restricted to Repository staff only

1MB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

Training chatbots using the reinforcement learning paradigm is challenging due to high-dimensional states, infinite action spaces and the difficulty in specifying the reward function. We address such problems using clustered actions instead of infinite actions, and a simple but promising reward function based on human-likeness scores derived from human-human dialogue data. We train Deep Reinforcement Learning (DRL) agents using chitchat data in raw text—without any manual annotations. Experimental results using different splits of training data report the following. First, that our agents learn reasonable policies in the environments they get familiarised with, but their performance drops substantially when they are exposed to a test set of unseen dialogues. Second, that the choice of sentence embedding size between 100 and 300 dimensions is not significantly different on test data. Third, that our proposed human-likeness rewards are reasonable for training chatbots as long as they use lengthy dialogue histories of ≥10 sentences.

Keywords:neural networks, reinforcement learning, unsupervised learning, supervised learning, sentence embeddings, chatbots
Subjects:G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G760 Machine Learning
G Mathematical and Computer Sciences > G700 Artificial Intelligence
Divisions:College of Science > School of Computer Science
ID Code:35954
Deposited On:15 May 2019 09:07

Repository Staff Only: item control page