A Divide-and-Conquer Approach to Neural Natural Language Generation from Structured Data

Dethlefs, Nina, Schoene, Annika and Cuayahuitl, Heriberto (2021) A Divide-and-Conquer Approach to Neural Natural Language Generation from Structured Data. Neurocomputing, 433 . pp. 300-309. ISSN 0925-2312

Full content URL: https://doi.org/10.1016/j.neucom.2020.12.083

Documents
A Divide-and-Conquer Approach to Neural Natural Language Generation from Structured Data
Authors' Accepted Manuscript
[img] PDF
nc-neurocomputing-2021.pdf - Whole Document
Restricted to Repository staff only until 5 January 2022.

911kB
Item Type:Article
Item Status:Live Archive

Abstract

Current approaches that generate text from linked data for complex real-world domains can face problems including rich and sparse vocabularies as well as learning from examples of long varied sequences. In this article, we propose a novel divide-and-conquer approach that automatically induces a hierarchy of “generation spaces” from a dataset of semantic concepts and texts. Generation spaces are based on a notion of similarity of partial knowledge graphs that represent the domain and feed into a hierarchy of sequence-to-sequence or memory-to-sequence learners for concept-to-text generation. An advantage of our approach is that learning models are exposed to the most relevant examples during training which can avoid bias towards majority samples. We evaluate our approach on two common benchmark datasets and compare our hierarchical approach against a flat learning setup. We also conduct a comparison between sequence-to-sequence and memory-to-sequence learning models. Experiments show that our hierarchical approach overcomes issues of data sparsity and learns robust lexico-syntactic patterns, consistently outperforming flat baselines and previous work by up to 30%. We also find that while memory-to-sequence models can outperform sequence-to-sequence models in some cases, the latter are generally more stable in their performance and represent a safer overall choice.

Keywords:Neural Networks, Artificial Intelligence, Natural Language Processing
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G760 Machine Learning
G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing
Divisions:College of Science > School of Computer Science
Related URLs:
ID Code:43748
Deposited On:26 Jan 2021 16:38

Repository Staff Only: item control page