Infants and children are able to track statistical regularities in perceptual input, which allows them to acquire structural aspects of language and music, such as syntax. However, much more is known about the development of linguistic compared to musical syntax. In the present study, we examined 3.5-year-olds’ implicit knowledge of Western musical pitch structure using electroencephalography (EEG). Event-related potentials (ERPs) were measured while children listened to chord sequences that either 1) followed Western harmony rules, 2) ended on a chord that went outside the key, or 3) ended on an in-key but less expected chord harmonically. Whereas adults tend to show an early right anterior negativity (ERAN) in response to unexpected chords (Koelsch, 2009), 3.5-year-olds in our study showed an immature response that was positive rather than negative in polarity. Our results suggest that very young children exhibit implicit knowledge of the pitch structure of Western music years before they have been shown to demonstrate that knowledge in behavioral tasks.