Families of ϕ‐congruence subgroups of the modular group

Abstract We introduce and study families of finite index subgroups of the modular group that generalize the congruence subgroups. Such groups, termed ϕ‐congruence subgroups, are obtained by reducing homomorphisms ϕ from the modular group into a linear algebraic group modulo integers. In particular, we examine two families of examples, arising on the one hand from a map into a quasi‐unipotent group, and on the other hand from maps into symplectic groups of degree four. In the quasi‐unipotent case, we also provide a detailed discussion of the corresponding modular forms, using the fact that the tower of curves in this case contains the tower of isogenies over the elliptic curve defined by the commutator subgroup of the modular group.