| Affiliation: | (1) Learning Innovations at WestEd, 200 Unicorn Park Drive, 4th floor, Woburn, MA 01801-3324, USA;(2) USC School of Social Work, 669 West 34th Street, Los Angeles, CA 90089-0411, USA |
| Abstract: | Using meta-analysis, we report on an investigation of the evaluator's influence in the treatment setting on criminal recidivism outcomes. Many evaluators and users of evaluation of social interventions worry that mixing of the roles of program developer and program evaluator may bias results reported in intervention studies in a positive direction. We first review the results of prior investigations of this issue across 50 prior meta-analyses, finding 12 that tested the impact of investigator influence in the treatment setting. Eleven of these reported that effect size increased positively, sometimes substantially so, when evaluators were influential or involved in the treatment setting. We followed this with a meta-analysis of 300 randomized field trials in individually focused crime reduction, also finding intervention studies in which evaluators who were greatly influential in the treatment setting report consistently and substantially larger effect sizes than other types of evaluators. We discuss two major views — the ‘cynical’ and ‘high fidelity’ theories — on why this is consistently the case, and conclude with a further agenda for research. |
