Dethlefs, Nina, Schoene, Annika and Cuayahuitl, Heriberto (2021) A Divide-and-Conquer Approach to Neural Natural Language Generation from Structured Data. Neurocomputing, 433 . pp. 300-309. ISSN 0925-2312
Full content URL: https://doi.org/10.1016/j.neucom.2020.12.083
Documents |
|
|
PDF
nc-neurocomputing-2021.pdf - Whole Document 911kB |
Item Type: | Article |
---|---|
Item Status: | Live Archive |
Abstract
Current approaches that generate text from linked data for complex real-world domains can face problems including rich and sparse vocabularies as well as learning from examples of long varied sequences. In this article, we propose a novel divide-and-conquer approach that automatically induces a hierarchy of “generation spaces” from a dataset of semantic concepts and texts. Generation spaces are based on a notion of similarity of partial knowledge graphs that represent the domain and feed into a hierarchy of sequence-to-sequence or memory-to-sequence learners for concept-to-text generation. An advantage of our approach is that learning models are exposed to the most relevant examples during training which can avoid bias towards majority samples. We evaluate our approach on two common benchmark datasets and compare our hierarchical approach against a flat learning setup. We also conduct a comparison between sequence-to-sequence and memory-to-sequence learning models. Experiments show that our hierarchical approach overcomes issues of data sparsity and learns robust lexico-syntactic patterns, consistently outperforming flat baselines and previous work by up to 30%. We also find that while memory-to-sequence models can outperform sequence-to-sequence models in some cases, the latter are generally more stable in their performance and represent a safer overall choice.
Keywords: | Neural Networks, Artificial Intelligence, Natural Language Processing |
---|---|
Subjects: | G Mathematical and Computer Sciences > G700 Artificial Intelligence G Mathematical and Computer Sciences > G760 Machine Learning G Mathematical and Computer Sciences > G710 Speech and Natural Language Processing |
Divisions: | College of Science > School of Computer Science |
Related URLs: | |
ID Code: | 43748 |
Deposited On: | 26 Jan 2021 16:38 |
Repository Staff Only: item control page