Mutual information maximization: models of cortical self-organization

Unsupervised learning procedures based on Hebbian principles have been successful at modelling low-level feature extraction, but are insufficient for learning to recognize higher- order features and complex objects. In this paper we explore a class of unsupervised learning algorithms called Imax (Becker and Hinton 1992 Nature 355 161-3) that are derived from information-theoretic principles. The Imax algorithms are based on the idea of …