Survey Research in Public Administration: Assessing Mainstream Journals with a Total Survey Error Framework

Survey research is a common tool for assessing public opinions, perceptions, attitudes, and behaviors for analyses in many social science disciplines. Yet there is little knowledge regarding how specific elements of survey research methodology are applied in practice in public administration. This article examines five mainstream public administration journals over an eight‐year period regarding current methodological practice, organized around the total survey error framework. The findings show that survey research in the field of public administration features mainly small‐scale studies, heavy reliance on a single data collection mode, questionable sample selection procedures, and suspect sample frame quality. Survey data largely are analyzed without careful consideration of assumptions or potential sources of error. An informed evaluation of the quality of survey data is made more difficult by the fact that many journal articles do not detail data collection procedures. This study concludes with suggestions for improving the quality and reporting of survey research in the field.     

Comparative nationalism: the Basques in Spain and France

Abstract. Little research on peripheral nationalism goes beyond the case study approach. Comparative studies provide greater understanding of national self-identification by comparing different ethnic groups, yet encounter difficulties in rigorously delineating specific non-group causal factors. This study adds to our insights into peripheral nationalism by comparing self-identification within the same group but in two different political systems. Study of the Basques in both Spain and France permits a comparison of the same group, thus imposing an analytical control on ethnicity, under two different sets of state policy directions. Utilizing survey data, this investigation found sharp differences in national self-identification in the Basques in Spain and France. The argument is advanced that, even when other explanatory factors are considered, this difference in Basque identification should be understood in light of the different policy stances taken over time by the central governments in Paris and Madrid toward their respective Basque minorities. While initially counter-intuitive, it is suggested that toleration and non-repressive policies toward such minorities may actually serve in the long term to decrease peripheral national identification.     

Policy analysis: A conceptual base for a theory of improvement

The purpose of reason or rationality in the analysis of resource allocation decisions is to provide decision makers with guidance on policy decisions which result in improvement, i.e. a net increment in social welfare. The difficulty with what might be called the microrationality of cost-benefit analysis and cost-effectiveness analysis as conventionally practised is that unless the context of such rationality is established in the larger framework of what might in turn be called the macrorationality which reflects an awareness of the properties of the whole system, the policy analyst cannot know whether his recommendation will upgrade (constitute an improvement in) or degrade (constitute a deterioration in) the system as a whole. The paper attempts to suggest a first conceptual approach, and an implied research agenda, addressed to developing a more effective general context for policy analysis. The suggested approach has four major components; first, the establishment of the basis for policy analysis in the explicit specification of a general weighted objective function for national policy and a corresponding set of functions for particular programs and projects; second, the development and incorporation in cost-benefit and cost-effectiveness analysis of a set of analytical parameters which establish the basis for policy analyses which are in the first instance comparable, and in the second instance consistent with the national objective function; third, the development of a general analytical framework for cost-benefit and cost-effectiveness analysis which recognizes the essential interdependency of policy decisions, including the financing implications of expenditure policies; and, finally, the explicit recognition of the need for a criterion of overall system improvement in the development of an analytical framework for the measurement of the trades-off or opportunity costs between alternative policy mixes.     

Leveraging Administrative Data to Better Serve Children and Families

The challenge of complex social issues drives the need for data systems that can connect information across multiple service delivery systems. Integrated data systems provide significant value for needs assessment, program planning, policy decision making, and collective impact evaluation across a range of social issues. This case study describes a mature county‐based integrated data system. The case study has three aims: (1) to highlight the value of integrated data systems to research, (2) to outline common challenges that such initiatives face, and (3) to present several ongoing considerations for the field that could inform policy recommendations to facilitate the use of integrated data. The authors present examples from the experience of developing this integrated data system, including involvement in the design and evaluation of the first county‐level social impact bond. The case study highlights the benefits and limitations of integrated data use in research and the potential to advance evidence‐based social decision making.     

Gauging the Rigor of Qualitative Case Studies in Comparative Lobbying Research. A Framework and Guideline for Research and Analysis

Engaging in comparative lobbying research is a challenging task, because public affairs is complex and highly context-dependent. Thus, qualitative case studies have been researchers' primary choice. However, the case method has been subject to much debate surrounding its rigor, in terms of reliability, internal validity, and generalizability, and particularly its potential for theory building. To propose a framework for researchers conducting lobbying case studies as well as for reviewers receiving such work, we apply a positivistic approach on case study rigor from management studies and expand the framework to tackle the specific challenges of comparative qualitative lobbying research. Thus, we expand the research framework by a set of variables specific to public affairs: We add enrichment for internal validity, contextuality, and comparability for external validity, interdisciplinarity for construct validity, and hypertext-searchable databases for reliability. Thereby, we aim to transfer the lessons learned from management studies in terms of rigorous qualitative case studies to public affairs to help build novel and explanatory theory in the field and to provide guidance to researchers how to design a rigorous case study.     

Implementing Change on the Front Lines: A Management Case Study of West Feliciana Parish Hospital

In the policy implementation field, theorists are challenged to develop stepwise frameworks that facilitate the implementation of public policy. This case studies the implementation of a Medicare charity care policy in a small rural hospital using the top-down implementation framework. It offers an overview of top-down implementation theory and a discussion of top-down research findings as they relate to the real implementation experiences of hospital administrators. The case presents six recommendations for the effective implementation of a new policy on the front lines.     

Comparative Local Economic Development: A Framework for Further Research

Recent research on local economic development does not provide an adequate framework for anticipating the consequences of cross national policy transfer and emulation. The research focuses on case specific analysis of one or another program within specific national or economic situations, but such analyses are not structured by a common understanding of the forces which shape policy formation or implementation. This article argues that we must rethink how we understand local policy by combining state-centered and class-centered theories of local policy outcomes. It suggests that such an approach would provide a better basis for both anticipating policy outcomes and predicting transferability.     

The application of venture analysis to public policy decisions: A case study of photovoltaics

Venture analysis is a group of analytical techniques normally used by the private sector to assist in major capital decisions. The approach can be applied to public policy decisions in those cases where changing private sector investment decisions are the central focus. A case study concerning a temporary subsidy to an emerging energy technology is reviewed. Major strengths of the approach are its use of numerous data- and opinion-gathering techniques and its explicit treatment of market and supply uncertainties. The major weakness is its lack of ability to handle those social costs and benefits for which no market exists. Implications for other policy studies are also presented.     

Exploring the temporal dimension in policy evaluation studies

This article explores the time dimension in policy evaluation studies. We argue that time has been given little attention in policy evaluation studies, despite it being very important for the occurrence and assessment of policy success or failure. We therefore propose to make time a central element of policy evaluation. First, we explore the theoretical foundations behind the concept of time. Second, we present a case study to investigate the presence of time in that specific case and the evaluation thereof. We conclude with recommendations for policy evaluation studies.     

Explaining the content of impact assessment in the United Kingdom: Learning across time,sectors, and departments

While several studies have documented how evidence‐based policy instruments affect public policy, less research has focused on what causes changes over time in the analyses mandated by the instruments, especially in Britain. Thus, we take the analytical content of a pivotal regulatory reform instrument (impact assessment) as a dependent variable, draw on learning as a conceptual framework, and explain the dynamics of learning processes across departments, policy sectors, and time. Empirically, our study draws on a sample of 517 impact assessments produced in Britain (2005–2011). Experience and capacity in different departments matter in learning processes. Guidelines also matter, but moderately so. Departments specialize in their core policy sectors when performing regulatory analysis, but some have greater analytical capacity overall. Peripheral departments invest more in impact assessment than core executive departments. The presence of a regulatory oversight body enhances the learning process. Elections have different effects, depending on the context in which they are contested. These findings contribute to the literature on regulation, policy learning, and policy instruments.     

保障性住房政策过程的中央与地方政府关系——政策网络理论的分析与应用

为分析保障性住房政策过程中的中央与地方政府关系,找出问题解决的有效框架,采用案例分析方法,分析了保障性住房政策制定过程中中央部门间的讨价还价、政策执行过程中地方政府及其职能部门以及开发商等营利组织间的利益结盟与变通策略、其他参与主体被排斥等现象。在此基础上,引入政策网络理论,分析网络利益主体与网络利益关系。研究发现:保障性住房与民生密切相关,却积累了太多的"历史赊账";中央与地方政府间的非合作博弈是影响保障性住房政策过程以及政策结果的重要因素;非合作博弈问题需置于政策网络结构环境下去思考,政策网络理论能够提供"解释性"和"建构性"框架。论文研究对未来的保障性住房政策过程以及政策结果有重大实践意义。本文将网络利益分析与网络治理结合在一起,实现了政策网络两个理论流派的应用整合,对该领域的理论研究有重要价值。依据研究结论,保障性住房的政策制定与政策执行必须实现从封闭到开放的转变,为未来"十二五计划"做好铺垫。     

Governance Arrangements for Transboundary Issues: Lessons from the Food Waste Programs of Italian Regions

Focusing on transboundary issues contributes to highlighting how new governance modes can emerge from increasing interdependence and complexity in public policies. This article analyzes food waste as a transboundary issue that cuts across different policy subsystems (PSs), intersects multiple levels of government, and calls into question consolidated policy making for new modes of governance. The analytical framework provides a multidimensional approach and a new typology of governance arrangements to be used in empirical research. Evidence from an Italian case study and an empirical investigation of 20 regional food waste programs shows that regions can adopt different driving ideas and can activate a variety of governance arrangements and differentiated, as well as viable, modes of integration between consolidated PSs. The article argues that different tool mixes combined with different levels of governance complexity may lead to a variety of governance arrangements and a differentiated likelihood of policy integration.     

A Comparative Assessment of Elite Policy Recruits in Canada

Recent case studies and large-N survey evidence has confirmed long-suspected shortages of public sector “policy capacity”. Studies have found that government policy workers in various jurisdictions differ considerably with respect to types of policy work they undertake, and have identified uneven capacity for policy workers to access and apply technical and scientific knowledge to public issues. This suggests considerable difficulties for government’s ability to meet contemporary policy and governance challenges. Despite growing attention to these matters, studies have not examined the “elite” policy workers many governments recruit to address these capacity shortages. Using an established survey instrument, this study of two Canadian recruitment programs provides the first comparative analysis of elite policy recruits, as policy workers. Three research questions anchor the study: (1) What is the profile of these actors? (2) What types of policy work do “elite” policy analysts actually engage in? (3) How does their policy work compare by recruitment program? The article provides fresh comparative data on the nature of elite policy work and policy analytical capacity, but, more importantly, a crucial baseline for future comparative study of how elite recruitment may facilitate “supply-side” capacity gains expected from recruitment programs.     

EVALUATION RESEARCH AND POLITICAL SCIENCE: AN ARGUMENT AGAINST THE DIVISION OF SCHOLARLY LABOR

Political science might have been the birthplace of evaluation research, but it has instead been unreceptive to it for a number of reasons. Hofferbert's suggestion that political scientists become involved by studying the impact of policy on democratic processes does not take into account the ferment that has been taking place in the field of evaluation. Political scientists should not conduct evaluation studies using the same methodology they normally use because these may not produce the kind of information needed for improving policies or programs. A disciplinary division of labor is impractical for a number of reasons. Instead, they should use a policy sciences framework, which contains normative elements and focuses on potentials for change, among other things.     

退耕还林政策对农户生计的影响研究——基于家庭结构视角的可持续生计分析

以往关于退耕还林政策的分析,较少考虑微观的家庭人口因素。本研究在可持续生计分析框架下引入农户的家庭结构,并基于此框架使用农户模型具体分析了退耕还林政策对农户生计的影响。利用陕西周至县的农户调查数据进行实证研究,结果表明:家庭结构能够影响生计资本水平和时间配置,退耕还林政策对农户生计的作用因家庭结构而不同。退耕还林政策既没有针对生计资本贫乏的农户,也不一定能够促使农业劳动力向非农转化,并进一步提升农户的收入。有小孩但没有老年人的家庭较少从事外地打工活动,收入相对偏低,他们更依赖退耕补助。相关政策的制定和实施应当考虑家庭结构等人口因素。     

Advocacy coalitions and the practice of policy analysis

Sabatier and Jenkins-Smith (above) have presented a framework for describing the processes by which public policies change over time, and Heintz and Weyant (above) have provided an array of case studies illustrating and elaborating those processes. This article will focus explicitly on the manner in which the advocacy coalition framework bears on the practice of policy analysis, both from the perspective of the manager of analytical resources, and from the perspective of the practicing policy analyst.     

Crucial Factors in Implementing Radical Policy Change: A Comparative Longitudinal Study of Nordic Central Agency Relocation Programs

Abstract

Why do public policies succeed or fail? The aim of this article is to contribute to answering this enduring research question in policy research through a comparative study of the variable efforts by Nordic governments to relocate their central agencies from the capital regions over a period of several decades. This was a radical redistributive policy program premised on a policy instrument – coercion – which was very alien to political systems characterized as consensual democracies. Hence, it is no surprise that only two out of seven relocation programs of any substance were successful. The really intriguing research question here is how any relocation program was achievable at all in a policy context where this was very unlikely. A broadly based multi-theoretical analytical framework linking interest groups, institutions, human agency in the form of policy entrepreneurship/design and situational factors is employed to solve this research puzzle. Findings from this study offer important contributions to the following research fields: comparative public policy, radical policy change and most specifically the so-called third generation of public policy implementation research.     

Design Experiments: Engaging Policy Makers in the Search for Evidence about What Works

This article presents an argument for the greater use of design experiments, which can assist policy making because they provide both robust and timely evidence. We discuss their origins in education research, set out the methodology and propose some adaptations to the techniques used in these education studies to foster their application to a range of policy fields and problem areas. Design experiments need to meet two challenges. Can they provide valid evidence? Can they provide evidence that will be used by policy makers? Our argument shows how design experiments are robust when set against the classical canons of scientific study. We further claim that the design experiment approach offers a more viable means to developing evidence-based policy making than other forms of evaluation because of the timeliness of the insights that it provides.     

从GDP锦标赛到二元竞争:中国地方政府行为变迁的逻辑——基于1998—2006年中国省级面板数据的实证研究

在中国经济持续30多年高速发展中,地方政府是发展的主要动力,但从上世纪90年代末以来,地方政府在发展经济中所产生的负面效果越来越明显,中央出台了一系列政策去调整和引导地方政府行为,促使地方政府增加社会保障开支以促进社会公平。采用1998—2006年中国省级面板数据进行定量分析以考察地方政府行为的影响因素和变化。研究显示,地方政府在社会保障的财政开支上主要受当地经济发展状况和中央政府的重视程度影响;地方政府在回应中央政府政策调整的过程中,其行为并不同于"晋升锦标赛"所归纳的以GDP增长为主要目标的一元竞争模式,而是演变为在社会公平和经济发展的双重目标下,地方政府分别参与不同目标竞争的二元竞争模式。在既往对中国地方政府行为考察的政治激励和财政激励基础上提出了影响地方政府行为的激励结构、中央与地方的互动关系和地方所处的社会经济情况的分析框架,通过对地方政府行为变化的分析提出二元竞争模式以弥补"GDP晋升锦标赛模式"对地方政府行为解释力的不足。     

Learning our way out of environmental policy problems: a review of the scholarship

In acknowledgement of the complexity of environmental challenges, research on learning in environmental policy has grown substantially over the past two decades across a range of disciplines. Despite this growth, there are few comprehensive assessments of the literature on learning in environmental policy. This article fills this gap by providing insights on the overall coherence and impact of this body of scholarship. To do so, we analyze a sample of 163 articles from 2004 to 2014 using a standardized coding framework. The results provide an in-depth assessment of the status of the literature on learning in the context of environmental policy, as well as the quality of the literature. We demonstrate that despite the diversity in research questions and goals, the literature is lacking with respect to diversity in cases and context, theoretical development, clear conceptualization and operationalization of learning, and advancements in empirical approaches to study learning. From these insights, we discuss the challenges and opportunities for scholars in studying learning and provide recommendations for building the theoretical and methodological rigor of the field.