How are party coalitions shaped and reshaped? Elected officials choose coalitions to win elections, but they must work to maintain those coalitions. Non-elected political actors, advancing an ideology at odds with the party coalition, can undermine the party. This article explores this possibility in the case of partisan change on slavery in the Antebellum United States. Intellectuals in 1850 divided into two camps over slavery and the other major issues of the day at a time when slavery cross-cut the two parties in Congress. The ideological division matches one that develops in Congress a decade later, suggesting that the parties responded not just to electoral incentives, but also to this elite division.