In 2012, the editors of CPP published an exchange about the Drug Market Intervention (DMI) in High Point, NC, concluding that it may be a promising approach to crime control but questioning whether it could be implemented across different settings. In this effectiveness study, we followed a cohort of seven sites that participated in a Bureau of Justice Assistance–sponsored DMI training to assess implementation and outcomes. Three sites were not able to implement, and implementation fidelity varied across the four sites that did implement. Of the four sites that held at least one call‐in, only one was successful at reducing overall and drug crime (by 28% and 56%, respectively). This works out to an implementation rate of 57% with an average overall crime reduction of 16% (treatment‐on‐the‐treated) or 4% (intent‐to‐treat). The results of this study demonstrate the importance of replication and the careful study of implementation fidelity prior to wide dissemination. Policy Implications When the findings of an evaluation reveal an effective crime reduction program, particularly when it garners significant public attention, it is not uncommon to rush to judgment that it should be widely implemented. DMI is a perfect illustration of this shortsighted approach to evidence‐based crime prevention—multiple trials across a variety of contexts are necessary to understand whether a program is ready for broad dissemination and scale‐up. The DMI program was challenging for sites to implement and resulted in significant reductions in crime in the site with the implementation fidelity that was highest and most similar to the original site. Our findings echo earlier concerns that the approach may be less effective across diverse settings and illustrate why effectiveness studies are vital in the development of evidence‐based policy.