Neural and Adaptive Systems: Fundamentals Through Simulations

Statistical signal processing plays a critical role in a variety of disciplines: communications, control, and biomedical engineering, just to name a few. The traditional approach to statistical signal processing in the literature, published in the second half of the twentieth century was largely based on some simplifying assumptions: Wide-sense stationarity, Linearity, Gaussianity. These assumptions were made for mathematical tractability. Yet, many of the physical and biological phenomena encountered in real-life environments violate one or more of these assumptions with the net result that there is a “mismatch” between the underlying assumptions made in the design of a signal processor and the actual statistical characterization of the input data, which, in turn, may result in the loss of valuable information. The classical approach to the study of statistical signal processing may well have been justified in the twentieth century due to lack of computing power, which permeated much of that period. However, nowadays, with powerful computers at our disposal and at a reasonable cost, and with the signal-processing power of computers growing at an ever-increasing rate, the time has come for us to address some important issues in the undergraduate education of electrical engineering students—I say “electrical,” because that is the discipline where I am competent to make meaningful statements. In particular, we owe it to new generations of undergraduate students in electrical engineering to develop a practical appreciation of the characteristics of nonstationarity, nonlinearity, and non-Gaussianity, which are hallmarks of real-life data. These issues were discussed by this reviewer at some length in two articles published in the IEEE Signal Processing Magazine [1], [2]. With this background, I may now proceed with a review of the new book on “Neural and Adaptive Systems” by Principe et al. The authors of this book are to be congratulated for addressing the critical issues discussed above. Most important, they do so in a highly insightful manner and by utilizing mathematical tools to which undergraduate students in science and engineering will have been exposed by the time they reach their senior year. Insight into the understanding of neural networks and adaptive systems of various kinds is developed through the clever use of a simulation-driven approach in every chapter of the book. The simulations are based on NeuroSolutions, a versatile neural network simulator developed by Neuro-Dimension, and which is included in the CD-ROM accompanying the book. The book covers a total of 11 chapters, and three appendixes. According to this reviewer, the chapters may be grouped as follows: Chapters 1 and 2 are introductory, covering basic concepts dealing with the use of linear models for data fitting and the pattern-recognition problem. Chapters 3 through 7 cover different aspects of supervised feedforward neural networks and unsupervised neural networks. Specifically, the multilayer perceptron and the backpropagation algorithm used for its supervised training, function approximation using multilayer perceptrons, radial-basis function networks and support vector machines, Hebbian learning and its application to principal components analysis, competitive neural networks and self-organizing maps, are discussed in that order. Chapters 8 and 9 discuss elements of digital signal processing and linear adaptive filters. Chapters 10 and 11 conclude the book with a treatment of temporal processing through the use of recurrent neural networks, and the training of both supervised and self-organized forms of recurrent neural networks. The three appendixes present review material on linear algebra and pattern recognition, a tutorial on NeuroSolutions, and a data directory describing the data available on the CD-ROM. What is really impressive about this new book is that the authors do not shy off in discussing difficult concepts that are important to the understanding of adaptive and neural systems. Rather, they expose the reader to such concepts and try, through explanatory notes and the use of computer simulations, to make the reader develop an appreciation for them. While they do so, they make sure to keep the underlying mathematical concepts at a level well within the reach of senior students in science and engineering. To conclude, this book is strongly recommended for an undergraduate course on Adaptive and Neural Systems in those universities that already offer such a course.