Objective To examine the impact of pay for performance in Medicaid on the quality and utilization of care. Data Sources Medicaid claims and encounter data in three intervention states (Pennsylvania, Minnesota, and Alabama) and three comparison states. Study Design Difference‐in‐difference analysis with propensity score‐matched comparison group. Primary outcomes of interest were Healthcare Effectiveness Data and Information Set (HEDIS)‐like process measures of quality, utilization by service category, and ambulatory care–sensitive admissions and emergency department visits. Principal Findings In Pennsylvania, there was a statistically significant reduction of 88 ambulatory visits per 1,000 enrollee months compared with Florida. In Minnesota, there was a significant decrease of 7.2 hospital admissions per thousand enrollee months compared with Wisconsin. In Alabama, where incentives were not paid out until the end of a 2‐year waiver period, there was a decline of 1.6 hospital admissions per thousand member months, and an increase of 59 ambulatory visits per 1,000 enrollees compared with Georgia. No significant quality improvements in intervention relative to control states. Conclusions Our findings are mixed, with no measurable quality improvements across the three states, but reductions in hospital admissions in two programs. As states move to value‐based payment for patient‐centered medical homes and Accountable Care Organizations, lessons learned from these pioneering states should inform program design.