Do ReLU Networks Have An Edge When Approximating Compactly-Supported Functions?

We study the problem of approximating compactly-supported integrable functions while implementing their support set using feedforward neural networks. Our first main result transcribes this “structured” approximation problem into a universality problem. We do this by constructing a refinement of the usual topology on the space L¹loc^(Rd,R^D) of locally-integrable functions in which compactly-supported functions can only be approximated in L¹-norm …