Toward More “Evidence‐Informed” Policy Making?

The quality of public decision making depends significantly on the quality of analysis and advice provided through public organizations. Champions of “evidence‐informed” policy making claim that rigorous evaluation practices can significantly improve attainment of cost‐effective outcomes. After decades of experience, performance information is more sophisticated, but evaluation practices and capabilities vary enormously. Public agencies gather and process vast amounts of information, but there has been little analysis of how this information is actually utilized for policy and program improvement. This article examines how government agencies use evidence about policy and program effectiveness, with attention to four themes: (1) the prospects for improving “evidence‐informed” policy making, (2) the diversity of practices concerning evidence utilization and evaluation across types of public agencies and policy arenas, (3) recent attempts to “institutionalize” evaluation as a core feature of policy development and budget approval, and (4) the relationships between public agencies and nongovernmental sources of expertise.     

Big Data for Policy Analysis: The Good,The Bad,and The Ugly

Big data holds tremendous potential for public policy analysis. At the same time, its use prompts a number of issues related to statistical bias, privacy, equity, and governance, among others. Accordingly, there is a need to formulate, evaluate, and implement policies that not only mitigate the risks, but also maximize the benefits of using big data for policy analysis. This poses a number of challenges, which are highlighted in this essay.     

Revisiting the governance of privacy: Contemporary policy instruments in global perspective

The repertoire of policy instruments within a particular policy sector varies by jurisdiction; some “tools of government” are associated with particular administrative and regulatory traditions and political cultures. It is less clear how the instruments associated with a particular policy sector may change over time, as economic, social, and technological conditions evolve. In the early 2000s, we surveyed and analyzed the global repertoire of policy instruments deployed to protect personal data. In this article, we explore how those instruments have changed as a result of 15 years of social, economic and technological transformations, during which the issue has assumed a far higher global profile, as one of the central policy questions associated with modern networked communications. We review the contemporary range of transnational, regulatory, self‐regulatory, and technical instruments according to the same framework, and conclude that the types of policy instrument have remained relatively stable, even though they are now deployed on a global scale. While the labels remain the same, however, the conceptual foundations for their legitimation and justification are shifting as greater emphases on accountability, risk, ethics, and the social/political value of privacy have gained purchase. Our analysis demonstrates both continuity and change within the governance of privacy, and displays how we would have tackled the same research project today. As a broader case study of regulation, it highlights the importance of going beyond technical and instrumental labels. Change or stability of policy instruments does not take place in isolation from the wider conceptualizations that shape their meaning, purpose, and effect.     

The Big (Data) Bang: Policy,Prospects, and Challenges

Big data is increasingly the cornerstone on which policy making is based. However, with potential benefits and applications come challenges and dilemmas. In this set of symposium articles, authors examine the promise and problems of big data, exploring associated prospects, risks, parameters, and payoffs from a variety of perspectives. The articles address myriad challenges in the handling of big data sets, such as collection, validation, integrity, and security; ontological issues attending data analytics and conceptual transformations; the foundations of big data collection for social science research; the gap between the acquisition of data and its use to advance discovery and innovation; the costs and benefits of using big data in decision making and analysis; and, finally, related problems of privacy, security, and ethics. Issues such as these will continue to arise with increased use of big data as fundamental to policy making and governance in today's growing information society.     

Transparency in transnational governance: The determinants of information disclosure of voluntary sustainability programs

The rise of “new” transnational governance has intensified debates about a lack of accountability in global politics. Reviewing the mechanisms through which transparency can foster accountability beyond the state, this article explores the determinants of information disclosure in the field of transnational sustainability governance. Examining the institutional design of 113 voluntary sustainability programs, we find a positive correlation between the involvement of public actors and information disclosure. In contrast, the role of civil society is more ambiguous. There is no statistical support for arguments linking non‐governmental organization participation to increased transparency. At the same time, our analysis reveals a robust correlation between civil society‐led metagovernance and information disclosure. Moreover, we find that crowding has a negative effect on transparency, whereas normative peer pressures have no influence. At a broader level, the analysis reveals a lack of “deep transparency” among transnational sustainability governors. This limits the scope for transparency‐induced accountability in this policy domain.     

Winners and losers: Conflict management through strategic policy engagement

In high-conflict policy debates, individuals often make strategic decisions about the ways in which they engage in efforts to influence the direction of the debate. Some individuals act to expand the scope of the conflict, whereas others would prefer to contain the scope of the conflict and maintain status quo. This study empirically examines the relationship between activities of political engagement and goals of conflict expansion or containment in a particular setting in which there are clear “winners” and “losers.” This research first explores the tactics an individual undertakes as either conflict expansion or containment. The patterns uncovered are then tested against sectoral affiliation, to draw conclusions about key factors that explain some variation in policy engagement. Findings confirm that there are predictable patterns to engagement decisions, paving the way for future testing in different policy arrangements.     

Bridging the Divide between Evidence and Policy in Public Sector Decision Making: A Practitioner's Perspective

While policy advocates can help bridge the divide between evidence and policy in decision making by focusing on ambiguity and uncertainty, policy makers must also play a role by promoting and preserving deliberative processes that value evidence as a core element in leveling raw constituent opinion, ultimately resulting in a better‐informed electorate. Building on existing research and analytic capability, state legislatures can increase the demand for and delivery of relevant information, giving the institution the capacity to keep abreast of research in critical public policy areas. By implementing data and time‐conscious evaluative frameworks that emphasize evidence‐based decision making and longitudinal cost–benefit analytics at critical policy‐making junctures, the institutional culture can become less unpredictable and the “rules of the game” can be more transparent. In 2015, Mississippi's legislative leaders created a system to review requests for new programs and funding using such an evidence screen.     

The Politics of “Entrepreneurial” Economic Development Policy of States in the U.S.

“Entrepreneurial” economic development strategies at the state level in the United States, which focus on nurturing home‐grown, high‐growth businesses, lack immediate payoffs for politically powerful constituencies, a condition that would seem likely to limit their appeal compared to the alternative “locational” strategy of attracting large investments from elsewhere. Nonetheless, many U.S. states have added programs with entrepreneurial attributes to their economic development portfolios in recent years. This paper explores how the political obstacles to such programs have been overcome. In a few cases, an institutional innovation in the policy‐making process drew in new participants who provided ideas for and support to programs with entrepreneurial attributes. More commonly, the preferences of executive branch officials, especially governors, appear to have been critical to the enactment and implementation of such programs. This finding suggests that economic development policy making may be more technocratic than is commonly believed.     

Improving Legitimacy in Nanotechnology Policy Development Through Stakeholder and Community Engagement: Forging New Pathways

This paper reports on a multistakeholder engagement process conducted in an Australian policy setting that led to a new community engagement framework: Science and Technology Engagement Pathways (STEP). I describe the process in the context of a lack of awareness, experience, and culture of deliberative public engagement in Australia, particularly in relation to decision making concerning science, technology, and innovation. Increasing cautiousness in government and industry approaches to nanotechnology development, in Australia and elsewhere, creates an imperative and an opening for improved stakeholder and community engagement to improve the legitimacy and sustainability of decisions. In this context, STEP may stimulate movement toward deliberative engagement by raising awareness and commitment from diverse stakeholders and providing a structure for developments in engagement and public dialog. STEP potentially provides “rules of engagement” and “intervention pathways” for ongoing public engagement with science and technology developments and for critical “science in society” perspectives to inform policy.     

“Tell It Like It Is”: Decision Making,Groupthink, and Decisiveness among U.S. Federal Subcabinet Executives

In addition to difficulties gathering and evaluating complete information, cognitive limitations and biases preclude individuals from making fully value‐maximizing choices when making decisions. It has been suggested that, done properly, involving advisors can compensate for individual‐level limitations. However, the “groupthink” tradition has highlighted ways group‐aided decision making can fail to live up to its potential. Out of this literature has emerged a paradigm Janis calls “vigilant problem‐solving.” For this article, we interviewed 20 heads of subcabinet‐level organizations in the U.S. federal government, asking questions about how they made important decisions. Ten were nominated by “good‐government” experts, 10 chosen at random. We wanted to see whether there were differences in how members of those two groups made decisions, specifically, to what extent executives in the two categories used a “vigilant” process. We found, however, that similarities between the two groups overwhelmed differences: As best as we were able to measure, decision making by U.S. subcabinet executives tracks vigilant decision making recommendations fairly closely. The similarity reflects a common style of senior‐level decision making, which we theorize grows out of government bureaucracy's methodical culture. We did, however, develop evidence for a difference between outstanding executives and others on another dimension of decision making style. Outstanding executives valued decision making decisiveness—“bias for action”—more than the comparison group. Perhaps, then, what distinguishes outstanding executives from others is not vigilance but decisiveness. Contrary to the implications of the groupthink literature, the danger in government may be “paralysis by analysis” as much or more than groupthink.     

Big Data,intelligence, and analyst privacy: investigating information dissemination at an NSA-funded research lab

The Laboratory for Analytic Sciences (LAS) at North Carolina State University, funded by the National Security Agency, is a collaborative, long-term research enterprise focused on improving intelligence analysis using Big Data. In its work, LAS has recently begun dealing with the trade-off between the collection, storage, and use of large unclassified data-sets and analyst privacy. We discuss particular privacy challenges at LAS, analyze privacy principles in the life cycle of LAS unclassified data-sets, what intelligence analysts themselves think about these privacy concerns, and recommend possible best practices potentially applicable to LAS, as well as future Big Data laboratories and research centers that collaborate with intelligence communities.     

From Big Data to Knowledge: An Ontological Approach to Big Data Analytics

The introduction of Big Data sets in the healthcare domain has presented opportunities to engage in analytics of very large sets containing both structured and unstructured data. With advances in information technology (IT), these data sets have become available from diverse sources at greatly increased rates. The availability of Big Data sets has introduced complexities that we must address, not only in terms of semantics and analytics but also in terms of data management, storage, and distribution. Currently, the capabilities to ingest, analyze, and manage multipetabyte data sets have underscored the limitations of our analytics capabilities supported by relational database management systems. This essay argues that an ontology‐based approach to data analytics provides a practical framework to address the semantic challenges presented by Big Data sets. No ontological framework can address the operational and management requirements introduced by the availability of Big Data sets, however. There are also a number of IT architectural factors that must be considered in implementing such a framework.     

Assessing Instrument Mixes through Program‐ and Agency‐Level Data: Methodological Issues in Contemporary Implementation Research

Theories of policy instrument choice have gone through several “generations” as theorists have moved from the analysis of individual instruments to comparative studies of instrument selection and the development of theories of instrument choice within implementation “mixes” or “governance strategies.” Current “next generation” theory on policy instruments centers on the question of the optimality of instrument choices. However, empirically assessing the nature of instrument mixes is quite a complex affair, involving considerable methodological difficulties and conceptual ambiguities related to the definition and measurement of policy sector and instruments and their interrelationships. Using materials generated by Canadian governments, this article examines the practical utility and drawbacks of three techniques used in the literature to inventory instruments and identify instrument ecologies and mixes: the conventional “policy domain” approach suggested by Burstein (1991 ); the “program” approach developed by Rose (1988a ); and the “legislative” approach used by Hosseus and Pal (1997 ). This article suggests that all three approaches must be used in order to develop even a modest inventory of policy instruments, but that additional problems exist with availability and accessibility of data, both in general and in terms of reconciling materials developed using these different approaches, which makes the analysis of instrument mixes a time‐consuming and expensive affair.     

Science and Politics in the Regulation of Genetically Modified Organisms in Brazil

This paper analyzes the regulation of genetically modified organisms (GMOs) in Brazil, focusing on the debates that preceded the enactment in 2005 of the current Biosafety Law and the factors (both domestic and international) that influenced the policy‐making process. More specifically, it identifies the social and governmental actors that influenced the congressional debates, showing how their interests and agenda were translated into two different institutional frameworks. Here, the policy outcome was what we call a “hybrid” and “conciliatory” regulatory model, which formally acknowledges the importance of both technical/scientific and social/political considerations in the evaluation of GMO biosafety and engages most of the stakeholders in the regulatory process. The new policy regime also establishes a middle ground between the conflicting demands of promoting agricultural development and protecting the country's biodiversity.     

When Is Science Used in Science Policy? Examining the Importance of Scientific and Technical Information in National Research Council Reports

A frequent lament among researchers is that public policy makers should pay more attention to scientific and technical information (STI). If there is any single area where one might expect STI to be used in public policy making and agenda setting it is in science and technology policy. Many of the policy makers in science and technology policy are themselves scientists or researchers and presumably would prove especially receptive to STI. However, STI is only one of many types of information used in policy making and policy actors often differ in the extent to which they view STI as credible, particularly compared to other types of potentially policy‐relevant information. Research on credibility (the believability of information, information types, and media) has shown variance and policy makers’ “credibility maps.” Thus, some policy makers have preference for formal information generally and STI specifically, but others privilege raw data, personal experience, authority, history and anecdote, analogical reasoning, or conformance to ideology, to name just a few of the information choices. Here, we build on the current researchers’ previous bibliometrics‐based work and use data from 41 semi‐structured elite interviews with National Research Council (NRC) executives and staff and NRC committee members concerning the use of STI in reports issued by the NRC. Findings show that the use of STI in NRC reports varies according to the nature of the inquiry and the sponsor. Information used in the reports is based on not only the assessed credibility of information but also its perceived direct relevance and the availability of STI as compared to other types of information. In general, the amount of STI in the NRC reports tends to have modest effects on the likelihood that reports will be used in policy making or by the mass media. More important factors include the timing of the report with respect to political agendas, the party requesting the report, and the enacted roles of NRC staff members and committee chairs.     

理解治理多样性：一种国家治理的新科学

现代社会及其公共事务的复杂性对治理构成了严峻的挑战。为了应对公共事务的复杂性、多规模性、嵌套性和动态性,很多研究者提出了不同的治理理论,并且都试图寻找“治理万能药”,以一种治理方式适用所有情景。然而人类要真正解决所面临的公共事务治理难题,就需要实现“认识论哲学”的转型,以“多样性”的思维范式取代“单一性”思维范式,按照“治理多样性”的新范式来重构治理体制“。治理多样性”分析范式是一种治理的新科学,其分析框架主要包括“行动者、机制、物品属性、结构和绩效”等要素,核心是公共事务与治理系统的有效匹配,实现基于“多行动者——多机制——多属性——多结构——多目标”的治理,多样性是其内在逻辑。治理本身是一个复杂演进和相互调适的过程“。治理多样性”要应用于治理实践,就需要发展出“诊断、设计和学习模型”。     

Politics,policy making,data, and the homeless

Abstract

It is often said that a major obstacle to crafting effective policies concerning home‐lessness is the large uncertainty associated with estimates of the extent of the problem. Such uncertainty is due largely to the difficulties of identifying a “hidden” population. But how true is it that effective policy making for the homeless depends upon counting their population accurately? This paper reviews some critical relationships among politics, policy making, and data; examines data requirements for policy making affecting the homeless; argues the case for relative rather than absolute measurement; and assesses the importance for public policy of data problems in this area.     

Realizing the Promise: Government Information Systems and the Fourth Generation of Information Technology

Interoperability is more than “digital plumbing”—making sure that computers talk so that bits of data flow properly. Fundamentally, interoperability is people talking and sharing information. Sharing information reduces the “paperwork burden”on the citizen, streamlines work processes, and enriches the formulation, implementation, and evaluation of policy. Building on prior theory and research, this research has developed empirically derived, practical findings and recommendations to support thedevelopment of appropriate interoperable systems.     

Subnational Responses to Fracking in Canada: Explaining Saskatchewan's “Wild West” Regulatory Approach

This article assesses the regulatory response to fracking by Saskatchewan, Canada's second largest oil‐producing province. Public officials and industry representatives claim fracking regulations are “comprehensive” and “robust”; however, there has been no comparative assessment of this claim. To address this gap, we outline the dominant regulatory pathways of U.S. states and Canadian provinces, ranging from applying existing regulations with minimal revisions, to enacting broader revisions or bans. We account for this variation using a framework from Davis ( 2012 ) emphasizing governments’ dependence on the oil sector, the level of support for fracking among elected officials and policy makers, and the influence of key “constituencies.” The article then traces the growth and impact of fracking in Saskatchewan and analyzes new trends in the province's regulation of fracking. Given the province's application of existing regulations with minimal revisions and active weakening of enforcement, we identify Saskatchewan as taking Rabe and Borick's ( 2013 ) “conventional” regulatory approach, typical of Davis's “energy dominant” states.     

The Challenges of Consulting the Public on Science Policy: Examining the Development of European Risk Assessment Policy for Genetically Modified Animals

With the growing importance of public engagement in science policy making and declining levels of public trust in food production, the European Food Safety Authority (EFSA) has attempted to embed “good governance” approaches to strengthen scientific independence and open up risk decision making, which include the use of public consultations. However, “opening up” of risk assessment policies reveals some tensions; namely, balancing the goals of scientific excellence and transparency, protecting science from interests, addressing value judgments, and limited opportunities to debate ethical and social issues. EFSA's development of risk assessment policy for genetically modified animals is used as a case study to analyze these tensions. This analysis suggests that in order to fulfill good governance commitments and maintain trust in risk governance, closer cooperation between EFSA and the European Commission is required to provide “space” for debating the broader risk management issues. This publically accessible space may be needed alongside rather than instead of EFSA's consultation.