Ensemble-Based Deep Reinforcement Learning for Chatbots

Cuayahuitl, Heriberto and Lee, Donghyeon and Ryu, Seonghan and Cho, Yongjin and Choi, Sungja and Indurthi, Satish and Yu, Seunghak and Choi, Hyungtak and Hwang, Inchul and Kim, Jihie (2019) Ensemble-Based Deep Reinforcement Learning for Chatbots. Neurocomputing . ISSN 0925-2312

Full content URL: https://doi.org/10.1016/j.neucom.2019.08.007

Documents
Ensemble-Based Deep Reinforcement Learning for Chatbots
Accepted Manuscript

Request a copy
[img] PDF
chatbot-neurocomputing2019.pdf - Whole Document
Restricted to Repository staff only until 7 August 2020.

2MB
Item Type:Article
Item Status:Live Archive

Abstract

Trainable chatbots that exhibit fluent and human-like conversations remain a big challenge in artificial intelligence. Deep Reinforcement Learning (DRL) is promising for addressing this challenge, but its successful application remains an open question. This article describes a novel ensemble-based approach applied to value-based DRL chatbots, which use finite action sets as a form of meaning representation. In our approach, while dialogue actions are derived from sentence clustering, the training datasets in our ensemble are derived from dialogue clustering. The latter aim to induce specialised agents that learn to interact in a particular style. In order to facilitate neural chatbot training using our proposed approach, we assume dialogue data in raw text only – without any manually-labelled data. Experimental results using chitchat data reveal that (1) near human-like dialogue policies can be induced, (2) generalisation to unseen data is a difficult problem, and (3) training an ensemble of chatbot agents is essential for improved performance over using a single agent. In addition to evaluations using held-out data, our results are further supported by a human evaluation that rated dialogues in terms of fluency, engagingness and consistency – which revealed that our proposed dialogue rewards strongly correlate with human judgements.

Keywords:Deep Supervised Learning, Deep Unsupervised Learning, Deep Reinforcement Learning, Neural Chatbots
Subjects:G Mathematical and Computer Sciences > G730 Neural Computing
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
G Mathematical and Computer Sciences > G760 Machine Learning
G Mathematical and Computer Sciences > G700 Artificial Intelligence
Divisions:College of Science > School of Computer Science
ID Code:36668
Deposited On:22 Aug 2019 08:34

Repository Staff Only: item control page