Scholars widely agree that the public is pragmatic about criminal justice. The empirical basis for this conclusion is the failure in several previous studies to find a sizable negative relationship between dispositional and situational crime attributions, or between support for punitive and rehabilitative crime policies. We suggest, however, that public pragmatism may be an artifact of the use of unidirectional question batteries in prior research to measure attribution styles and policy support. When such questions are used, acquiescent responding can introduce systematic error that is positively correlated across items and scales. Drawing on data from an experiment with a national sample (N = 826) of Internet panelists, we examine how this methodological approach impacts the bivariate correlations and multivariate relationships between attribution styles and between support for punitive and rehabilitative crime policies. The findings reveal that using unidirectional sets of questions to measure these concepts likely results in 1) inflated alpha reliability coefficients, 2) an underestimation of the magnitude of the negative relationships between attribution styles and between punitiveness and support for rehabilitation, and 3) an underestimation of the extent to which punitiveness and support for rehabilitation are driven by the same factors, working in opposite directions.