Top-down modulation is an essential cognitive component in human perception. Despite mounting evidence of top-down perceptual modulation in adults, it is largely unknown whether infants can engage in this cognitive function. Here, we examined top-down modulation of motion perception in six- to eight-month-old infants via their smooth pursuit eye movements. With four experiments, we demonstrated that infants’ perception of motion direction can be flexibly shaped by briefly learned predictive cues when no coherent motion was available. The current findings present a novel view of infant perception and its development: Infant perceptual systems respond to predictive signals engendered from higher-level learning systems, leading to a flexible and context-dependent modulation of perception. This work also suggested that the infant brain is sophisticated, interconnected, and active when placed in a context where it can learn and predict.