Count of eigenvalues in the generalized eigenvalue problem

We address the count of isolated and embedded eigenvalues in a generalized eigenvalue problem defined by two self-adjoint operators with a positive essential spectrum and a finite number of isolated eigenvalues. The generalized eigenvalue problem determines spectral stability of nonlinear waves in a Hamiltonian dynamical system. The theory is based on the Pontryagin's Invariant Subspace theorem in an indefinite inner product space but it extends beyond the scope of earlier papers of Pontryagin, Krein, Grillakis, and others. Our main results are (i) the number of unstable and potentially unstable eigenvalues {\em equals} the number of negative eigenvalues of the self-adjoint operators, (ii) the total number of isolated eigenvalues of the generalized eigenvalue problem is {\em bounded from above} by the total number of isolated eigenvalues of the self-adjoint operators, and (iii) the quadratic form defined by the indefinite inner product is strictly positive on the subspace related to the absolutely continuous part of the spectrum of the generalized eigenvalue problem. Applications to solitons and vortices of the nonlinear Schrödinger equations and solitons of the Korteweg--De Vries equations are developed from the general theory.