Intelligent estimation: A review of theory, applications, and recent advances

Recent developments in the field of deep learning have led to the widespread integration of artificial neural networks in various domains of application. Prominent contemporary artificial neural network training techniques are based on first-order gradient computation. The emphasis on algorithmic performance has driven the emergence of variant artificial neural network training methodologies. Estimation theory, traditionally considered a sub-field of statistics and signal processing, has been explored by various researchers for the development of non-gradient based training methods. Articles published with the aim of utilizing estimation-based artificial neural network training techniques have shown promising results. We identify the integration of estimation theory within the artificial neural network training procedure as intelligent estimation. In this paper, the field of intelligent estimation is analyzed in greater depth with emphasis on the algorithmic performance of novel implementations. Intelligent estimation with applications in the professional domain is also considered, and will help lay the foundation for future research in the literature.