Theory of Monte Carlo Sampling-Based Alopex Algorithms for Neural Networks

We propose two novel Monte Carlo sampling-based Alopex algorithms for training neural networks. The proposed algorithms naturally combine the sequential Monte Carlo estimation and Alopex-like procedure for gradient-free optimization, and the learning proceeds within the recursive Bayesian estimation framework. Experimental results on various problems show encouraging convergence results.