Deep Bayesian Self-Training

De Sousa Ribeiro, Fabio, Caliva, Francesco, Swainson, Mark , Gudmundsson, Kjartan, Leontidis, Georgios and Kollias, Stefanos (2020) Deep Bayesian Self-Training. Neural Computing and Applications, 32 (9). pp. 4275-4291. ISSN 0941-0643

Full content URL:

Deep Bayesian Self-Training
Accepted Manuscript

Request a copy
Deep Bayesian Self-Training
Published PDF
[img] PDF
NCA_Leontidis.pdf - Whole Document
Restricted to Repository staff only
Available under License Creative Commons Attribution-NoDerivatives 4.0 International.

[img] PDF
DeepBayesianSelf-Training_NCA.pdf - Whole Document
Available under License Creative Commons Attribution 4.0 International.

Item Type:Article
Item Status:Live Archive


Supervised Deep Learning has been highly successful in recent years, achieving state-of-the-art results in most tasks. However, with the ongoing uptake of such methods in industrial applications, the requirement for large amounts of annotated data is often a challenge. In most real world problems, manual annotation is practically intractable due to time/labour constraints, thus the development of automated and adaptive data annotation systems is highly sought after. In this paper, we propose both a (i) Deep Bayesian Self-Training methodology for automatic data annotation, by leveraging predictive uncertainty estimates using variational inference and modern Neural Network architectures, as well as (ii) a practical adaptation procedure for handling high label variability between different dataset distributions through clustering of Neural Network latent variable representations. An experimental study on both public and private datasets is presented illustrating the superior performance of the proposed approach over standard Self-Training baselines, highlighting the importance of predictive uncertainty estimates in safety-critical domains.

Keywords:Deep Learning, variational Bayesian inference, Optical Character Verification, Deep Neural Networks, Self Training, Machine Learning
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G400 Computer Science
G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Computer Science
ID Code:36321
Deposited On:05 Jul 2019 09:23

Repository Staff Only: item control page