Improving the evaluation of anticrime programs: There’s work to be done

The summary version of the National Research Council’s report, Improving the Evaluation of Anticrime Programs, published in the Journal of Experimental Criminology, was accompanied by eight commentaries from distinguished criminology researchers. This paper responds to those thoughtful and provocative commentaries by further discussing the two broad questions that they raised: What anticrime programs should be evaluated and what methods should be used to evaluate them. The main themes of this response are that (a) evaluation is needed for the programs, practices, and policies in actual use, not only those developed by criminologists; (b) programs developed and tested by criminologists are more likely to be used if they build on existing programs and give as much attention to “implementability” as to theory; and (c) evaluation of practices, policies, and area-wide programs will be limited unless credible applications of observational and quasi-experimental methods can be developed.

“To develop and test:” The inventive difference between evaluation and experimentation

The National Research Council (NRC) Report on Improving Evaluation of Anticrime Programs raises a fundamental question about the mission of evaluation research. The implicit premise of the report is that the mission of evaluation is to answer questions about programs developed by others; in short, to test anti-crime programs. In contrast, the mission of experimental criminology has, historically, been to develop anti-crime programs as well as to test them. There are times when an arm’s-length relationship between program and evaluation may be appropriate. Yet, such a separation necessarily produces a courtroom-like adjudication role for evaluators, rather than the laboratory-like, participant–inventor role that has characterized the best of experimental criminology. The recent case of the Chicago police’s “evaluating” the use of sequential suspect identification methods developed by academic psychologists shows the many flaws of the “testing-only” model. This suggests that providing “effective guidance of criminal justice policy and practice,” as the NRC report defines its focus [Lipsey, M. ed (2005). http://newton.nap.edu/pdf/0309097061/pdf_image/R1.pdf] will not only require evaluation research (defined as arm’s-length testing) but the full toolbox of experimental criminology to develop and test anti-crime programs.

Randomized experiments as the bronze standard

In this paper, the strengths and weakneses of randomized field experiments are discussed. Although it seems to be common knowledge that random assignment balances experimental and control groups on all confounders, other features of randomized field experiments are somewhat less appreciated. These include the role of random assignment in statistical inference and representations of the mechanisms by which the treatment has its impact. Randomized experiments also have important limitations and are subject to the fidelity with which they are implemented. In the end, randomized field experiments are still the best way to estimate causal effects, but are a considerable distance from perfection.     

Remembering Attack Ads: An Experimental Investigation of Radio

Political Behavior - This article seeks to advance our understanding of the influence of attack advertising on the public in two ways. First, we examine whether the content of individuals'...     

A role for structured observation in ethics

Progress in the natural sciences has depended upon the collection and use of carefully controlled observational data. By contrast, ethicists have failed to agree upon a role for observational data in their enterprise. Although factors embedded in the human condition obscure the role of observational data in ethical theory, barriers to the use of such data in ethics may be superable. Observation may not provide definitive answers to most ethical or metaethical questions. However observation of carefully constructed experimental conditions may provide the basis for cumulative progress in some branches of ethics.     

What Do Voters Want From Their Local MP?

This article summarises the findings from a study of what constituents want from their local Member of Parliament. 1 We make use of a survey technique known as conjoint analysis, wherein we present a national sample of British voters with profiles of hypothetical MPs who vary randomly in their characteristics, activities and behaviour. We find that voters like MPs who are independent from the party line and who do not focus exclusively on national policy work. MPs' gender and experience matter far less to constituents. Overall, voters want a Parliament made up of strong‐minded MPs who see their role as that of a constituency representative. This has important implications for parliamentary democracy in Britain.     

乡村精英变迁视角下的村社治理

20世纪80年代以来,我国村社治理结构的转型反映了当代中国政治生活领域中"国家-社会"关系的调整和"民主-自治"的社会治理变迁.在这一过程中,处于乡村社区政治体系边界的乡村精英呈现出向多元化方向的发展.如何使乡村精英在数量和质量上得到保证并充分发挥其在农村中的"守门员"角色,以实现转型社会中所渗透于乡村社区的多种能量间的良性互动,将是乡村治理的一个关键,也是社会主义新农村建设的一个应有之义.     

提高工具痕迹实验课效果的思考

教师要根据工具痕迹学实验课的目的,注重加强学生动手操作能力,注重实验课内容的创新,在传统工具痕迹实验方法的基础上,提高教学效果。     

Improving evaluation of anti-crime programs: Summary of a National Research Council report★

This article summarizes a report of the National Research Council: Improving Evaluation of Anti-crime Programs. It is based on a workshop, held in September 2003, in which participants presented and discussed examples of evaluation-related studies that represent the methods and challenges associated with research at three levels: interventions directed toward individuals; interventions in neighborhoods, schools, prisons, or communities; and interventions at a broad policy level. The article, and the report on which it is based, is organized around five questions that require thoughtful analysis in the development of any evaluation plan: What questions should the evaluation address? When is it appropriate to conduct an impact evaluation? How should an impact evaluation be designed? How should the evaluation be implemented? What organizational infrastructure and procedures support high quality evaluation? The authors highlight major considerations in developing and implementing evaluation plans for criminal justice programs and make recommendations for improvement of government funded evaluation studies.     

Cross-cutting Messages and Voter Turnout: Evidence from a Same-Sex Marriage Amendment

Does disagreement stimulate political participation, or discourage it? Some researchers find that exposure to cross-cutting views demobilizes voters. Selection bias in the way individuals expose themselves to disagreement and other sources of endogeneity pose challenges to causal inference. I address these concerns by using an experimental design that exogenously assigns cross-cutting or reinforcing messages. A random sample of North Carolina Democrats and Republicans received postcards summarizing either liberal or conservative opinions on a statewide same-sex marriage amendment. I find that individuals exposed to disagreement demobilize by 1.0 to 1.6 percentage points, with the majority of the combined effect attributable to a 2.0-percentage point decrease in turnout among Republicans receiving a Democratic message. I observe a similar level of demobilization when defining disagreement on the basis of predicted issue position on same-sex marriage in place of partisan affiliation. The effects are strongest among moderate supporters of traditional marriage that receive a cross-cutting treatment. The experimental design thus enables causal evidence on the nuanced interactions between political or issue position and exposure to campaign information from the opposing side.