A study of early stopping, ensembling, and patchworking for cascade correlation neural networks

Riley, Mike J. W. and Jenkins, Karl W. and Thompson, Chris P. (2010) A study of early stopping, ensembling, and patchworking for cascade correlation neural networks. International Journal of Applied Mathematics, 40 (4). pp. 307-316. ISSN 1992-9978

Documents
IJAM_40_4_12.pdf
[img]
[Download]
[img]
Preview
PDF
IJAM_40_4_12.pdf - Whole Document

1MB
Item Type:Article
Item Status:Live Archive

Abstract

The constructive topology of the cascade correlation algorithm makes it a popular choice for many researchers wishing to utilize neural networks. However, for multimodal problems, the mean squared error of the approximation increases significantly as the number of modes increases. The components of this error will comprise both bias and variance and we provide formulae for estimating these values from mean squared errors alone. We achieve a near threefold reduction in the overall error by using early stopping and ensembling. Also described is a new subdivision technique that we call patchworking. Patchworking, when used in combination with early stopping and ensembling, can achieve an order of magnitude improvement in the error. Also presented is an approach for validating the quality of a neural network’s training, without the explicit use of a testing dataset.

Keywords:Bias & Variance, Cascade Correlation, Early Stopping, Ensembling, Subdivision method
Subjects:G Mathematical and Computer Sciences > G760 Machine Learning
Divisions:College of Science > School of Engineering
ID Code:12080
Deposited On:07 Oct 2013 08:23

Repository Staff Only: item control page