Accurate localization of touch requires the integration of two reference frames—an internal (e.g., anatomical) and an external (e.g., spatial). Using a tactile temporal order judgement task with the hands crossed over the midline, we investigated the integration of these two reference frames. We manipulated the reliability of the visual and vestibular information, both of which contribute to the external reference frame. Visual information was manipulated between experiments (Experiment 1 was done with full vision and Experiment 2 was done while wearing a blindfold). Vestibular information was manipulated in both experiments by having the two groups of participants complete the task in both an upright posture and one where they were lying down on their side. Using a Bayesian hierarchical model, we estimated the perceptual weight applied to these reference frames. Lying participants on their side reduced the weight applied to the external reference frame and produced a smaller deficit; blindfolding resulted in similar reductions. These findings reinforce the importance of the visual system when weighting tactile reference frames, and highlight the importance of the vestibular system in this integration.