This study explored whether tolerance of sexual harassment of men (TSHM) is driven by a common set of sexist ideologies typically found to be related to tolerance of sexual harassment of women. University students (N = 433) completed a modified version of the Sexual Harassment Attitude Scale (SHAS) designed to measure TSHM. Predictor variables included sexual harassment myths about women, modern sexism, hostile, and benevolent sexism toward men and women and participant gender. A factor analysis of the measure revealed two reliable factors (sexual harassment as flirtation and minimization of victimization). With the exception of benevolence toward women, men scored higher on all measures of sexism and TSHM, yet correlational patterns showed a similar trend among men and women, suggesting a shared ideological belief that justifies TSHM. Despite differences in participant sex on most measures, participant sex was not a significant moderating variable in regression analyses when examining factors relating to TSHM. These results provide support that tolerance of sexual harassment is driven by a common set of sexist attitudes.