On the value of formal assessment of uncertainty in regulatory analysis

The US Office of Management and Budget introduced in 2003 a new requirement for the treatment of uncertainty in Regulatory Impact Analyses (RIAs) of proposed regulations, requiring agencies to carry out a formal quantitative uncertainty assessment regarding a regulation’s benefits and costs if either is expected to reach $1 billion annually. Despite previous use in other contexts, such formal assessments of uncertainty have rarely been employed in RIAs or other regulatory analyses. We describe how formal quantitative assessments of uncertainty – in particular, Monte Carlo analyses – can be conducted, we examine the challenges and limitations of such analyses in the context of RIAs, and we assess how the resulting information can affect the evaluation of regulations. For illustrative purposes, we compare Monte Carlo analysis with methods typically used in RIAs to evaluate uncertainty in the context of economic analyses carried out for the US Environmental Protection Agency’s Nonroad Diesel Rule, which became effective in 2004.     

The “Science of Science Policy”: reflections on the important questions and the challenges they present

Developing the “Science of Science Policy” will require data collection and analysis related to the processes of innovation and technological change, and the effects of government policy on those processes. There has been much work on these topics in the last three decades, but there remain difficult problems of finding proxies for subtle concepts, endogeneity, distinguishing private and social returns, untangling cumulative effects, measuring the impact of government programs in a true “but for” sense, and sorting out national and global effects. I offer observations on how to think about these issues. This paper was presented as the Keynote Address at the NSF Workshop on Advancing Measures of Innovation: Knowledge Flows, Business Metrics, and Measurement Strategies, Arlington, VA, June 2006.     

The impact of police charges in incidents of wife abuse

The present study addressed the impact of an urban police force laying criminal charges in cases of wife abuse. Follow-up data from battered women and from police officers were collected to identify the long-term consequences of this police policy that began in 1981, including any negative or unintended side effects. Results indicated that the policy resulted in a dramatic increase in police-laid charges (2500%), and a shift from the Family to the Criminal Division of Provincial Court. Based on interview data, victims did not decrease their requests for police service (a feared side effect) and reported a significantly higher level of satisfaction with police service in this area. A significant reduction in police calls and victim-reported violence was associated with police intervention, especially when charges were laid. Paradoxically, police officers' attitudes about the policy were more negative than the actual positive changes that they helped to produce. Considerations for future research and professional training are outlined.     

The importance of “spillovers” in the policy mission of the advanced technology program

Government policies like the Advanced Technology Program (“ATP”) are intended, at least in part, to remedy the “market failure” inherent in the fact that a significant portion of the social benefits of new knowledge and technology are not captured by a firm that invests in R&D. ATP’s project selection, and its evaluation of the impact of its program, can be made more effective by explicitly incorporating the analysis of such “spillovers.” For project selection, this means identifying technological, organizational and economic factors that tend to oint to a large “spillover gap,” or deviation between the social and private rates of return to a proposed project. For program evaluation and assessment, it means adapting existing study methods that measure social returns to innovation in ways that explicitly capture spillover effects. This paper is based on a study that I performed for the ATP, Economic Analysis of Research Spillovers: Implications for the Advanced Technology Program, NIST GCR 97-708. I have benefited from comments and useful discussions with Zvi Griliches, Jeanne Powell, Rosalie Ruegg, and Richard Spivack. Some of the ideas in this paper grew out of previous joint research with James Adams. The views expressed herein are my own, however, and should not be attributed to any of these individuals or to the ATP.     

Enhancing judicial skills in domestic violence cases: the development,implementation, and preliminary evaluation of a model US programme

ABSTRACT

The overwhelming number and complexity of domestic violence cases in criminal and family courts has resulted in the development of education programmes to assist judges. There is limited research on judicial education in this area. This paper reviews one such initiative entitled ‘Enhancing Judicial Skills in Domestic Violence Cases’ (EJS) that has been developed and implemented over the last 20 years by the National Judicial Institute on Domestic Violence, a partnership of the US Department of Justice Office on Violence Against Women, National Council of Juvenile and Family Court Judges and Futures Without Violence. We present findings of a preliminary evaluation of the programme based on the self-reports of 480 judges who had taken the four-day workshop between 2006 and 2010. Overall, judges reported the programme to be engaging and effective. At a six-month follow‐up, most of the judges identified specific benefits and behavior changes in the areas of access to justice, judicial leadership, victim safety, and abuser accountability as a result of participating in the programme. Critical issues in judicial education are highlighted based on the authors’ experiences in the development and implementation of this programme.