A critical step toward object recognition is the segmentation of a scene into relevant regions. One of the most important cues for segmentation is that of common fate: Elements that move together are grouped together. Here we describe a new instantiation of common fate, in which elements move together not through physical space, but through luminance space. Experiment 1 shows that when elements of a scene become brighter together or darker together, observers group those elements together. Experiments 2 and 3 show that this effect is not due to the availability of fixed luminance differences between target and background regions, but requires common changes within each region in the direction of luminance over time. The effect is differentiated from the recently discovered grouping cue of temporal synchrony, and is considered instead to be an extension of Wertheimer's original grouping factor of common fate. Common fate for luminance, or generalized common fate, is an extremely strong cue for the segmentation of a scene, yielding a tremendous advantage over grouping by fixed luminance cues.