Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering

Maleki, Sepehr, Maleki, Sasan and Jennings, Nicholas R. (2021) Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering. Applied Soft Computing, 108 . p. 107443. ISSN 1568-4946

Full content URL: https://doi.org/10.1016/j.asoc.2021.107443

This is the latest version of this item.

Documents
Unsupervised anomaly detection with LSTM autoencoders using statistical data-filtering
Authors' Accepted Manuscript
[img]
[Download]
[img]
Preview
PDF
Manuscript R3.pdf - Whole Document
Available under License Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International.

659kB
Item Type:Article
Item Status:Live Archive

Abstract

To address one of the most challenging industry problems, we develop an enhanced training algorithm for anomaly detection in unlabelled sequential data such as time-series. We propose the outputs of a well-designed system are drawn from an unknown probability distribution, U, in normal conditions. We introduce a probability criterion based on the classical central limit theorem that allows evaluation of the likelihood that a data-point is drawn from U. This enables the labelling of the data on the fly. Non-anomalous data is passed to train a deep Long Short-Term Memory (LSTM) autoencoder that distinguishes anomalies when the reconstruction error exceeds a threshold. To illustrate our algorithm’s efficacy, we consider two real industrial case studies where gradually-developing and abrupt anomalies occur. Moreover, we compare our algorithm’s performance with four of the recent and widely used algorithms in the domain. We show that our algorithm achieves considerably better results in that it timely detects anomalies while others either miss or lag in doing so.

Keywords:Machine learning, LSTM, Deep Learning, Anomaly Detection
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science
ID Code:44910
Deposited On:13 May 2021 10:46

Available Versions of this Item

Repository Staff Only: item control page