Sex differences in favor of males have been documented in measures of spatial perspective taking. In this research, we examined whether social factors (i.e., stereotype threat and the inclusion of human figures in tasks) account for these differences. In Experiment 1, we evaluated performance when perspective-taking tests were framed as measuring either spatial or social (empathetic) perspective-taking abilities. In the spatial condition, tasks were framed as measures of spatial ability on which males have an advantage. In the social condition, modified tasks contained human figures and were framed as measures of empathy on which females have an advantage. Results showed a sex difference in favor of males in the spatial condition but not the social condition. Experiments 2 and 3 indicated that both stereotype threat and including human figures contributed to these effects. Results suggest that females may underperform on spatial tests in part because of negative performance expectations and the character of the spatial tests rather than because of actual lack of abilities.