Studies show that Whites tend to show the lowest level of support for affirmative action (AA) policies. Opponents of AA often argue that this is because it violates principles of meritocracy. However, self‐interest (based on social identification with those adversely affected) could also explain their opposition. In three studies, we varied whether an Asian or White male is adversely affected by AA to test another explanation; namely, that Whites' fairness judgments are based on both the adversely affected person's race and the fairness evaluator's ideological beliefs. Although we found some support for the meritocratic explanation, this was not sufficient to explain why Whites view AA as (un)fair. Instead, we found strong support for our prediction that Whites who are opposed to equality perceive more unfairness when a White (vs. Asian) was harmed by AA, whereas Whites who endorse egalitarian ideologies perceive the opposite. This finding suggests that neither self‐interest nor meritocratic explanations can fully account for Whites' opposition to AA. Copyright © 2014 John Wiley & Sons, Ltd.