A dynamic regularized radial basis function network for nonlinear, nonstationary time series prediction

In this paper, constructive approximation theorems are given which show that under certain conditions, the standard Nadaraya-Watson (1964) regression estimate (NWRE) can be considered a specially regularized form of radial basis function networks (RBFNs). From this and another related result, we deduce that regularized RBFNs are m.s., consistent, like the NWRE for the one-step-ahead prediction of Markovian nonstationary, nonlinear autoregressive time series generated by an i.i.d. noise processes. Additionally, choosing the regularization parameter to be asymptotically optimal gives regularized RBFNs the advantage of asymptotically realizing minimum m.s. prediction error. Two update algorithms (one with augmented networks/infinite memory and the other with fixed-size networks/finite memory) are then proposed to deal with nonstationarity induced by time-varying regression functions. For the latter algorithm, tests on several phonetically balanced male and female speech samples show an average 2.2-dB improvement in the predicted signal/noise (error) ratio over corresponding adaptive linear predictors using the exponentially-weighted RLS algorithm. Further RLS filtering of the predictions from an ensemble of three such RBFNs combined with the usual autoregressive inputs increases the improvement to 4.2 dB, on average, over the linear predictors.