Research Summary We used a quasi‐experimental design to evaluate the effectiveness of Moving On, a gender‐responsive, cognitive‐behavioral program designed for female offenders. Between 2001 and 2013, there were two distinct periods in which Moving On was administered with, and without, fidelity among female Minnesota prisoners. To determine whether program integrity matters, we examined the performance of Moving On across these two periods. By using multiple comparison groups, we found that Moving On significantly reduced two of the four measures of recidivism when it was implemented with fidelity. The program did not have a significant impact on any of the four recidivism measures, however, when it operated without fidelity. Policy Implication The growth of the “what works” literature and the emphasis on evidence‐based practices have helped foster the notion that correctional systems can improve public safety by reducing recidivism. Given that Moving On's success hinged on whether it was delivered with integrity, our results show that correctional practitioners can take an effective intervention and make it ineffective. Providing offenders with evidence‐based interventions that lack therapeutic integrity not only promotes a false sense of effectiveness, but also it squanders the limited supply of programming resources available to correctional agencies. The findings suggest that ensuring program integrity is critical to the efficient use of successful interventions that deliver on the promise of reduced recidivism.