This article addresses the issues involved in using compliance with probation supervision as an interim outcome measure in evaluation research. We address the complex nature of compliance and what it implies. Like much research on probation and criminal justice more generally, it was not possible to use random assignment to treatment and comparison groups in the case study we address, which evaluated the SEED training programme. We therefore compare two different data analysis methods to adjust for prior underlying differences between groups, namely regression adjustment of treatment covariates that are related to the outcome measure in the sample data and regression adjustment using propensity scores derived from a wide range of baseline variables. The propensity score method allows for control of a wider range of baseline variables, including those which do not differ significantly between the two groups.