Count of eigenvalues in the generalized eigenvalue problem

We study isolated and embedded eigenvalues in the generalized eigenvalue problem defined by two self-adjoint operators with a positive essential spectrum and a finite number of isolated eigenvalues. The generalized eigenvalue problem determines the spectral stability of nonlinear waves in infinite-dimensional Hamiltonian systems. The theory is based on Pontryagin’s invariant subspace theorem and extends beyond the scope of earlier papers of Pontryagin, Krein, Grillakis, and others. Our main results are (i) the number of unstable and potentially unstable eigenvalues equals the number of negative eigenvalues of the self-adjoint operators, (ii) the total number of isolated eigenvalues of the generalized eigenvalue problem is bounded from above by the total number of isolated eigenvalues of the self-adjoint operators, and (iii) the quadratic forms defined by the two self-adjoint operators are strictly positive on the subspace related to the continuous spectrum of the generalized eigenvalue problem. Applications to the localized solutions of the nonlinear Schrödinger equations are developed from the general theory.