Testing What Matters (If You Must Test at All): A Context‐Driven Approach to Substantive and Statistical Significance

For over a half century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation and assessment of research results. A long list of criticisms has fueled the so‐called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (e.g., scientific, substantive, social, political) significance. I introduce a simple, intuitive approach that grounds testing in subject‐area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to test for real‐world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation.     

The “Europeanization” of the Basel process: Financial harmonization between globalization and parliamentarization

Public policy initiatives aimed at the prevention of future financial crises originate with global harmonization in the form of executive standards issued by the Basel Committee on Banking Supervision. This article explores the role of the European Parliament (EP) in the process of adapting the standards in European Union legislation passed in 2013 as the Capital Requirements Regulation/Capital Requirements Directive IV. Unlike accounts casting the EP as increasingly dependent upon outside sources in order to meet its enhanced legislative role, we find it increasingly dexterous in developing and using in‐house policy ideas, expertise, and not least a common sense of institutional purpose. Notable EP successes in final legislation include (but are not restricted to) a headline cap on bankers' bonuses in the face of entrenched business and national interests. The argument is developed by drawing upon a broad range of interviews together with other primary and secondary sources, tracing the contribution of the EP from the early stages of agenda‐setting through to the development of an “esprit de corps” among the committee lead team which survived intact throughout the “black box” of trilogue negotiations. Besides illuminating the notoriously opaque trilogue process, the analysis also contributes to contemporary debates about whether the EP's increased legislative powers are resulting in a shift away from its traditional allegiances with diffuse interests toward a greater engagement with producer sources in order to fulfill requirements for policy expertise.     

The “war on cops,” retaliatory violence,and the murder of George Floyd*

The police murder of George Floyd sparked nationwide protests in the summer of 2020 and revived claims that public outcry over such high-profile police killings perpetuated a violent “war on cops.” Using data collected by the Gun Violence Archive (GVA) on firearm assaults of U.S. police officers, we use Bayesian structural time series (BSTS) modeling to empirically assess if and how patterns of firearm assault on police officers in the United States were influenced by the police murder of George Floyd. Our analysis finds that the murder of George Floyd was associated with a 3-week spike in firearm assaults on police, after which the trend in firearms assaults dropped to levels only slightly above that which were predicted by pre-Floyd data. We discuss potential explanations for these findings and consider their relevance to the contemporary discussion of a “war on cops,” violence, and officer safety.     

The Scary World of Online News? Internet News Exposure and Public Attitudes Toward Crime and Justice

Objectives

A substantial body of literature indicates that certain forms of media consumption may increase anxiety about crime and support for social controls. However, few studies have examined whether Internet news consumption is positively associated with such attitudes. The void is significant given the public’s increasing use of online news sources. This study addresses this research gap.

Methods

We draw on data from four national surveys conducted between 2007 and 2013, which collectively include interviews with more than 13,000 Americans. Using OLS and logistic regression, we assess the relationships between exposure to traditional and online media and perceptions of victimization risk, support for punitive crime policies, and views about police powers.

Results

Consistent with prior work, we find positive relationships between exposure to traditional forms of media—television news and crime programming—and anxiety about victimization and support for harsh crime policies. In contrast, Internet news exposure is generally not associated with anxieties about crime or support for getting tough on criminals. However, there is evidence of an interactive relationship between political ideology and Internet news exposure.

Conclusions

The results provide little support for cultivation theory in the context of Internet news consumption. We discuss the import of our findings, and suggest new lines of research to explore the correlates and the effects of exposure to online news sources.

     

Reading for Life and Adolescent Re‐Arrest: Evaluating a Unique Juvenile Diversion Program

We present results of an evaluation of Reading for Life (RFL), a diversion program for nonviolent juvenile offenders in a medium‐sized Midwestern county. The unique program uses philosophical virtue theory, works of literature, and small mentoring groups to foster moral development in juvenile offenders. Participants were randomly assigned to RFL treatment or a comparison program of community service. The RFL program generated large and statistically significant drops in future arrests. The program was particularly successful at reducing the recidivism of more serious offenses and for those groups with the highest propensity for future offenses.     

College Guidance for All: A Randomized Experiment in Pre‐College Advising

Pre‐college advising programs exist in most disadvantaged high schools throughout the United States. These programs supplement traditional advising by high school guidance counselors and attempt to help underrepresented and disadvantaged students overcome the complexities of the postsecondary admission and financial aid processes. Existing evidence on these programs often uses within‐school randomization where spillovers and alternative supports may confound estimates. We provide the first evidence on a whole school intervention resulting from a school‐level randomized controlled trial in the United States. The college access program we study uses a near‐peer model where a recent college graduate works at the school assisting students in the application and enrollment process. Pooled results across the first three years of program implementation find no significant impacts on overall college enrollment. However, subgroup analyses reveal positive, significant effects among the groups most targeted by the intervention: Hispanic and low‐income students. Most of the impact comes through increasing two‐year college enrollment, but this appears to be new entrants rather than inducing students to move from four‐year to two‐year colleges. The observed positive effects for these subgroups attenuate over time. We attribute this drop in the estimated impact to departures in fidelity of the experiment. Even among the cohorts for which we find positive enrollment impacts, we find no significant impacts on college persistence.     

Misleading by Example: The Effects of a Manager’s Unfair Customer Treatment on Service Employee Performance and Perceived Managerial Trustworthiness

We explore service worker reactions to a supervisor’s fair treatment of customers (i.e., customer-directed fairness), utilizing the group-value model of fairness to formulate two distinct predictions: (1) a status cuing effect, in which employees internalize social cues from the supervisor’s behavior to determine the social value of customers, and adapting their own customer-oriented behaviors to reflect the supervisor’s cue, and (2) a character indictment effect, in which employees use customer-directed fairness to assess the trustworthiness of the supervisor’s character. Results from experimental and field data provide evidence for these dual effects and show how each ultimately affects the employee’s in-role and extra-role customer service behavior. Implications are discussed with regard to the group-value model of fairness, alternative theories of fairness, and practical applications.     

Backloading to extinction: Coping with values conflict in the administration of Australia's federal biodiversity offset policy

Policy-makers are frequently required to consider and manage conflicting public values. An example of this in the environmental domain is biodiversity offset policy, which governments worldwide have adopted as a mechanism to balance environmental protection with socio-economic development. However, little work has examined administrative practices underpinning biodiversity offset policy implementation, and how the adoption of coping strategies to manage value conflicts may influence resulting policy outcomes. This study fills this research gap using a case study of Australia's federal biodiversity offset policy under the Environment Protection and Biodiversity Conservation (EPBC) Act 1999. Using data from 13 interviews of federal policy administrators, I show that the introduction of a new policy in 2012 enabled a shift from the use of precedent to a technical approach for setting offset requirements under the EPBC Act. Yet, multiple sources of policy ambiguity remain, and administrators have adopted post-approval condition-setting, or ‘backloading’—a form of cycling, facilitated by structural separation—to defer detailed assessments of offset requirements until after biodiversity losses are approved. Backloading thus undermines the effectiveness of environmental policy and will persist as coping strategy unless policy ambiguity is reduced via legislative amendments and adequate resourcing of biodiversity conservation.

Points for practitioners

• Biodiversity offset policy requires administrators to manage conflicting environmental and socioeconomic values.
• Technical decision tools reduce reliance on case-by-case decision-making, but multiple ambiguities persist.
• Backloading (post-approval condition-setting) defers values conflict, but reduces transparency, accountability, and policy effectiveness.
• Policy ambiguity must be reduced at the political level to facilitate effective biodiversity conservation.