Optimization Using Broyden-Update Self-Adjoint Sensitivities

Recently, a self-adjoint method was proposed for the computation of network parameter sensitivities. Here, we propose a modification to this method, which practically eliminates its overhead by using the Broyden update for the computation of the system matrix derivatives. We investigate the feasibility and the efficiency of the Broyden-update self-adjoint sensitivities in gradient based optimization. We compare in terms of CPU time the optimization processes with several well known optimization algorithms. We investigate their performance in two cases: 1) when the response and its sensitivities are provided by our method, and 2) when responses only are provided.