Objectives

This study expands upon Weisburd’s work (1993) by reexamining the relationship between sample size and statistical power in criminological experiments. This inquiry, now known as the Weisburd paradox, postulates that increasing the sample size of experiments does not always lead to increases in statistical power. The current research also begins to explore the potential sources of the Weisburd paradox.

Methods

The effect sizes and statistical power are computed for the outcome measures (n = 402) of all experiments (n = 66) included in systematic reviews published by the Campbell Collaboration’s Crime and Justice Coordinating Group. The design sensitivity of these experiments is reviewed by sample size, as well as other factors that may explain the variation in effect sizes and statistical power across studies.

Results

Effect sizes decline as the sample size of the experiment increases, whereas statistical power is unrelated to sample size but strongly associated with effect size. Disclosure of fidelity issues and publication bias is unrelated to statistical power and treatment effects. Variability in the dependent variable and sample demographics are significantly related to statistical power, but not to effect size.

Conclusions

The study finds support for the Weisburd paradox, as the ability to manipulate statistical power by increasing sample size is not as strong as statistical theory would suggest, and experiments with larger sample sizes generally produce smaller effects. It is believed that a relationship was not observed between sample size and statistical power because the sensitivity gained from increasing sample size is offset by effect size simultaneously decreasing.