Scholars have seemingly established that constituents hold "out of step" legislators electorally accountable. Empirically, however, such claims have not been based on measures placing districts and perceptions of legislators’ preferences in the same space. We remedy this using the 2006 and 2008 Cooperative Congressional Election Studies, and Aldrich and McKelvey’s scaling procedure, finding that electoral success is roughly consistent with Downsian logic but not with the blanket statement that out-of-step incumbents are penalized. Voters punish out-of-step incumbents conditional on having a sufficiently more "in step" challenger. Effects are substantial, but so are incumbent advantages.