Abstract

The self-organizing mixture network (SOMN) is a learning algorithm for mixture densities, derived from minimizing the Kullback-Leibler information by means of stochastic approximation methods. It has been shown the SOMN converges faster than the EM-based algorithms and generalizes better as it is based on the expected likelihood rather than the sample likelihood. The derived algorithm has similar updating forms to the self-organizing map (SOM), thus reveals the mixture interpreter role of the neighbourhood function used in the SOM. When the sample set is small, overfitting problems often occur in most algorithms. Further improvement can be achieved by averaging ensembles of the SOMNs. The algorithms have been applied to both experimental data and real-world problems. The results show that smoothed mixtures with improved accuracy have been obtained. Estimation variance has been reduced.