Derivation of momentum LMS algorithms by minimizing objective functions

The momentum least-mean-squares (m-LMS) algorithm is extensively used in neural network and signal processing applications, and is an arbitrary extension to the LMS algorithm. It is shown that several different versions of the m-LMS algorithm can be obtained by minimizing different objective functions. It appears that the minimization of weighted average square error function and the weighted accumulated square error function leads to two widely used m-LMS algorithms. The minimization of the weighted average square error function also provides two new versions of the m-LMS algorithm. These old and new versions of the m-LMS algorithm are applied to a parameter estimation problem. From the results, it is found that the new versions of the m-LMS algorithm provide smaller variance of the parameter estimates.<>