Previous studies of verbal short-term memory (STM) indicate that STM for serial order may be linked to language development and developmental language disorder (DLD). To clarify whether a domain-general mechanism is impaired in DLD, we studied the relations between age, non-verbal serial STM, and language competence (expressive language, receptive language, and language reasoning). We hypothesized that non-verbal serial STM differences between groups of children with DLD and typically developing (TD) children are linked to their language acquisition differences. Fifty-one children with DLD and sixty-six TD children participated as part of the HelSLI project in this cross-sectional study. The children were 4–6-year-old monolingual native Finnish speakers. They completed several tests of language and cognitive functioning, as well as new game-like tests of visual and auditory non-verbal serial STM. We used regression analyses to examine how serial STM moderates the effect of age on language. A non-verbal composite measure of serial visual and auditory STM moderated cross-sectional development of receptive language in the children with DLD. This moderation was not observed in the TD children. However, we found more rapid cross-sectional development of non-verbal serial STM in the TD children than in the children with DLD. The results suggest that children with DLD may be more likely to have compromised general serial STM processing and that superior non-verbal serial STM may be associated with better language acquisition in children with DLD.