Performance on tactile temporal order judgments (TOJs) is impaired when the hands are crossed over the midline. The cause of this effect appears to be tied to the use of an external reference frame, most likely based on visual information. We measured the effect of degrading the external reference frame on the crossed-hand deficit through restriction of visual information across three experiments. Experiments 1 and 2 examined three visual conditions (eyes open–lights on, eyes open–lights off, and eyes closed–lights off) while manipulating response demands; no effect of visual condition was seen. In Experiment 3, response demands were altered to be maximally connected to the internal reference frame and only two visual conditions were tested: eyes open–lights on, eyes closed–lights off. Blindfolded participants had a reduced crossed-hands deficit. Results are discussed in terms of the time needed to recode stimuli from an internal to an external reference frame and the role of conflict between these two reference frames in causing this effect.