Regularized Least Squares: A useful (Forgotten) tool for supervised and semi-supervised learning

This paper discusses the undervalued importance of Regularized Least Squares, and its continued usefulness in solving supervised and semisupervised learning problems. The common two moon classification problem was used to study and compare the effectiveness of three methods: the Support Vector Machine (Radial Basis Functions and 6 ^th Order Polynomials), Laplacian Regularized Least Squares, and K-Means with Regularized Least Squares (KM-RLS). The latter approach, which appears to be novel, is shown to be a very strong candidate for designing the hidden layer part of the classifier. As shown in the detailed results section, the KM-RLS method yields excellent results, and is computationally faster and less complex, when compared with the commonly used Support Vector Machine.