Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces

Becker, Philipp and Pandya, Harit and Gebhardt, Gregor and Zhao, Cheng and Taylor, C. James and Neumann, Gerhard (2019) Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces. In: Proceedings of the 36th International Conference on Machine Learning, 9th - 15th June 2019, California, USA.

Full content URL: http://proceedings.mlr.press/v97/becker19a.html

Documents
Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces
Published PDF
[img]
[Download]
[img] PDF
becker19a.pdf - Whole Document

534kB
Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive

Abstract

In order to integrate uncertainty estimates into deep time-series modelling, Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models, however, such approaches typically rely on approximate inference tech- niques such as variational inference which makes learning more complex and often less scalable due to approximation errors. We propose a new deep approach to Kalman filtering which can be learned directly in an end-to-end manner using backpropagation without additional approximations. Our approach uses a high-dimensional factorized latent state representation for which the Kalman updates simplify to scalar operations and thus avoids hard to backpropagate, computationally heavy and potentially unstable matrix inversions. Moreover, we use locally linear dynamic models to efficiently propagate the latent state to the next time step. The resulting network architecture, which we call Recurrent Kalman Network (RKN), can be used for any time-series data, similar to a LSTM (Hochreiter & Schmidhuber, 1997) but uses an explicit representation of uncertainty. As shown by our experiments, the RKN obtains much more accurate uncertainty estimates than an LSTM or Gated Recurrent Units (GRUs) (Cho et al., 2014) while also showing a slightly improved prediction performance and outperforms various recent generative models on an image imputation task.

Keywords:Deep Learning, Kalman Filtering
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
ID Code:36286
Deposited On:24 Jun 2019 09:13

Repository Staff Only: item control page