Justifying the use of non-experimental methods and disqualifying the use of randomized controlled trials: challenging folklore in evaluation research in crime and justice

The key limitation of non-experimental evaluation methods is that they require an assumption that all confounding factors related to treatment are identified in the statistical models developed. The key advantage of randomized experiments is that this assumption can be relaxed. In this paper, I describe and explain why this assumption is so critical for non-experiments and why it can be ignored in randomized controlled trials (RCTs). I also challenge what I describe as “folklores” that are used to justify the use of non-randomized studies despite this statistical limitation, and to justify the failure of evaluation researchers in crime and justice to use randomized experiments despite their unique ability to overcome this limitation. I conclude by reinforcing what Joan McCord had argued after a life time of review of evaluations: “(W)henever possible” evaluation studies “should employ random assignment.”