Are there really two cultures? A pilot study on the application of qualitative and quantitative methods in political science

In their 2012 publication A Tale of Two Cultures, Gary Goertz and James Mahoney argue that empirical research in the social sciences aiming at causal inference can be differentiated into a qualitative and a quantitative methodological culture. The two cultures differ fundamentally in how researchers approach and implement empirical studies. The argument is well laid out and comprehensively illustrated, but the empirical validity of the two cultures hypothesis has not yet been evaluated systematically. This note introduces a research project that aims to test the two cultures hypothesis via an empirical analysis of how qualitative and quantitative methods are applied. To determine whether there is a qualitative and quantitative method culture, the researchers initially sampled 30 articles from three journals (Comparative Political Studies, European Journal of Political Research, World Politics) in the 2008–2012 period. Based on this dataset, no evidence was found for the existence of coherent systems of methods practices in political science.     

Do People Really Lack Knowledge About the Economy? A Reply to Facchini

Facchini uses a behavioural approach to analyse the political beliefs of French people, who he believes are ‘more or less incompetent’ in economics. In this article I focus on his premise that the public are incompetent and that therefore their views, such as being opposed to the market in the case of the French people, should be interpreted as ‘perception bias’. Other economists may echo Facchini, claiming that people who voted Leave in the UK and for Trump in the USA did so because their lack of economic knowledge contributed to an ‘anti‐foreign bias’. However, I argue here that the existing empirical research showing that people lack economic knowledge is flawed. Many economists adopt a questionable approach to the interpretation of public knowledge and the evaluation of what knowledge is important.     

The Decision Not to Furl the Confederate Flag in My Classroom

Over the last year and a half, a number of scholars, pundits, and journalists have criticized college campuses for coddling students by constructing environments that protect them from offensive opinions and evidence that disconfirms their prior attitudes. In this article, I suggest two pedagogical techniques that can help students encounter and digest evidence that may go against their preconceived worldviews and opinions: teaching rational versus irrational thought and quantitative research design and critique. Guiding undergraduates through empirical studies on sensitive topics and encouraging them to critique their findings in a rational, academic manner can help ease them into more reasoned and less reactionary discourse about race, gender, religion, politics, and any other controversial topic in today’s contentious climate. I use my own experience teaching students research design in the months following a race-related murder to illustrate the effectiveness of this method in an extreme circumstance.     

Contextualising the Teaching of Statistics in Political Science

The teaching of quantitative research methods is an integral part of most postgraduate programmes in politics, but less common at undergraduate level. This article explores the extent to which research methods in general, and quantitative research methods in particular, form part of the curricula of Politics departments in the UK. We then discuss an approach for motivating interest in a subject that tends to be unpopular with many students. We recommend an approach that utilises the links between the general quantification of politics with the quantitative study of political phenomena, as well as a combination of basic research methods for all and more advanced student-focused training for some.     

The paradox of policy analysis: If it is not used,why do we produce so much of it?

This article explores the apparent paradox that our society invests heavily in policy analysis when empirical studies, political science theory, and common wisdom all suggest that analysis is not used by policymakers to make better policy decisions. It offers a critique of the traditional view of policy analysis and presents an alternative view derived from contemporary literature on the policy process and decisionmaking. The alternative view suggests that there are legitimate uses for analysis other than the problem‐solving use originally envisioned but apparently rarely attained. The two views imply different patterns of use of analysis by legislative committees—a contrast that I subject to an empirical test. An examination of quantitative data on policy analysis use by congressional committees from 1985 to 1994 lends support for the alternative view. The research has two implications. First, despite its scientific origins, policy analysis may be a more effective instrument of the democratic process than of the problem‐solving process. Second, the profession of policy analysis may be in better shape than many who are calling for fundamental changes to its practice seem to believe. © 1999 by the Association for Public Policy Analysis and Management.     

Managing the impersonal in a personalized public service

What happens when an apparently personalized small public service enters the information highway? How does it integrate the new frame of mind implied in the impersonal, open approach to information? The practical implications of implementing information technology as a means to use information strategically in the Maltese public service, with its apparently paradoxical administrative style, prompted the pilot study in early 2000 upon which this article is based. Using a simplified empirical approach, the study tested the hypothesis that small scale could affect the way leaders deal with the impersonal, such as information resource management. Although limited in scope, the results of the study support a qualified conclusion that small scale, through its link with personalization and associated informal mechanisms, does affect the way top managers in the public service deal with the impersonal, at least in the short and medium term. To what extent it does so, or the space for change, are subjects for further research. The article concludes with suggestions for further investigation into this topic, both in its narrow and wider applications. Copyright © 2002 John Wiley & Sons, Ltd.     

How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches

Researchers in comparative research increasingly use multilevel models to test effects of country‐level factors on individual behavior and preferences. However, the asymptotic justification of widely employed estimation strategies presumes large samples and applications in comparative politics routinely involve only a small number of countries. Thus, researchers and reviewers often wonder if these models are applicable at all. In other words, how many countries do we need for multilevel modeling? I present results from a large‐scale Monte Carlo experiment comparing the performance of multilevel models when few countries are available. I find that maximum likelihood estimates and confidence intervals can be severely biased, especially in models including cross‐level interactions. In contrast, the Bayesian approach proves to be far more robust and yields considerably more conservative tests.     

Political Participation as an Engine of Social Solidarity: A Sceptical View

Political theorists sometimes advocate the intensification of citizens' participation in politics, on the grounds that this serves as an engine for the cultivation of social solidarity and civic virtue. I argue against such an initiative and set out the case for a more nuanced examination of the effect of particular modes of political participation on social solidarity, in light of recent empirical literature. Against the assertions of the theorists in question, the research reveals that political participation per se does not cultivate the said virtues, whereas entrusting citizens with deliberating and deciding specific policy issues (specifically in the form of citizens' juries or CJs) does. Finally, it is argued that there is a rather limited scope to the implementation of deliberative bodies of the kind that cultivates solidarity. Consequently, intensifying political participation is not a reliable means through which social solidarity can be cultivated.     

Phantom Counterfactuals

Researchers often seek to identify the effects of a treatment on a sequence of behaviors, such as whether citizens register to vote and whether they then cast ballots. I show that average treatment effects (ATEs) are only identified until the first behavior (registering to vote) that affects the set of possible subsequent actions (voting). When one action changes the set of possible subsequent actions, it creates ‘phantom counterfactuals,’ or undefined potential outcomes, which render ATEs unidentified. I show that applied theory allows researchers to diagnose phantom counterfactuals, which helps to recognize unidentified ATEs and focus instead on other estimands that are identified. I illustrate this approach using a stylized model of crime reporting, showing how different theories generate different sets of identified estimands while holding constant an experimental design. I thereby establish the necessity of applied theory for causal identification in empirical research with sequential behavioral outcomes.     

It's About Time: The Lifespan of Information Effects in a Multiweek Campaign

I advance a theoretical and empirical framework that puts time and thus the temporal dynamics of candidate evaluation front and center in order to advance our understanding of the lifespan of information effects while enhancing the external validity of our experimental approaches. With these temporal properties in mind, I designed a “panel experiment” with research conducted over 12 weeks. This represents the first experimental approach to combine control over information exposure with attention to information processing throughout the course of a multiweek campaign. Against the backdrop of partisanship, empirical tests assess the ability of transient exposure to issue and character information to produce effects that endure beyond the moment the information is encountered either via memory‐based or on‐line processes. Findings reveal a remarkably limited role for enduring information effects and suggest a “rapid displacement” model of information processing where new information quickly displaces the accumulated stockpile of old information.     

对省级地方政府规模影响因素的定量研究

政府规模是我国推进政府改革进程中值得关注的一个重大课题,对其影响因素的探索对行政体制改革具有重要意义,而目前国内学界对政府规模影响因素的定量研究尚不多见,基本是政府规模问题研究的空白点.从政府规模的概念和影响因素入手,以2003年全国30个省级行政单位(台湾、香港、澳门、西藏除外)为样本,运用SPSS软件对政府规模的影响因素进行定量回归分析,对学界普遍认同的经济发展水平、经济体制、人口因素、地域面积以及民族因素进行考量,发现经济发展水平与中国省级政府规模呈负相关,说明瓦格纳法则并不适用于现阶段的中国;市场化指数与中国省级政府规模呈高度负相关;人口因素与政府规模呈高度负相关.这个结论与我国许多学者的经验观察与主观感受是相反的.地域面积与政府规模呈正相关关系,但在省级行政单位这一层面上影响非常微弱,不像有的学者强调得那样大;少数民族比重与政府规模呈高度正相关,这是一个新的定量研究发现.由此提出:在建立合理、优化的政府规模时,应更多地运用科学的研究方法,探讨其深层次的结构性原因,充分关注不同区域的特殊情况,对症下药地提出相应的改革方案.在推进政治、行政改革时更应慎之又慎,以构建与市场经济大背景相适应的公共服务型政府为基本的价值取向,坚持温和稳健,防止过度激进,坚持理性考察,避免煽情之作,以促进经济发展和社会进步.     

Intelligence and knowledge development: what are the questions intelligence analysts ask?

Question-asking is central to intelligence. Despite recognition of the importance of questions, in the absence of empirical research, much of what we understand about question-asking within the intelligence process remains speculative or limited to personal experience and memory. To develop an evidence-based approach to improving intelligence, this paper argues that empirical research into the questions that intelligence analysts ask is required. Drawing on insights from educational research, an initial set of research questions is proposed to understand the questions that intelligence analysts actually ask.     

How Governments Shape the Risk of Civil Violence: India's Federal Reorganization, 1950–56

Governments are absent from empirical studies of civil violence, except as static sources of grievance. The influence that government policy accommodations and threats of repression have on internal violence is difficult to verify without a means to identify potential militancy that did not happen. I use a within‐country research design to address this problem. During India's reorganization as a linguistic federation, every language group could have sought a state. I show that representation in the ruling party conditioned the likelihood of a violent statehood movement. Prostatehood groups that were politically advantaged over the interests opposed to them were peacefully accommodated. Statehood movements similar in political importance to their opponents used violence. Very politically disadvantaged groups refrained from mobilization, anticipating repression. These results call into question the search for a monotonic relationship between grievances and violence and the omission of domestic politics from prominent theories of civil conflict.     

Value,Nature, and Labor: A Defense of Marx

In this theoretical intervention, I argue that Karl Marx’s theory of value remains a powerful way to understand nature–society relations under capitalism. I suggest environmentalist critiques often misunderstand Marx’s value theory as a theory that “values” workers over nature. His critical theory is better understood as an explanation of how capitalist value exploits both workers and the environment. My defense of Marxian value theory is articulated through five “theses.” I provide empirical illustration based on recent research into the nitrogen fertilizer industry. (1) Value theory does not refer to all values. (2) Marx’s contention that nature does not contribute to value helps us explain its degradation under capitalism. (3) Marx’s value theory rooted in production allows for a critique of environmental economic valuation schemes (e.g. payments for ecosystem services) which are based on neoclassical value theories rooted in consumption/exchange. (4) Value is abstract social labor, but that means it also abstracts from nature. (5) Capital does value certain parts of nature and that matters. I conclude by advocating a “value theory of nature” in the spirit of Diane Elson’s powerful articulation of Marx’s “value theory of labor.”     

Comparative Politics and the Synthetic Control Method

In recent years, a widespread consensus has emerged about the necessity of establishing bridges between quantitative and qualitative approaches to empirical research in political science. In this article, we discuss the use of the synthetic control method as a way to bridge the quantitative/qualitative divide in comparative politics. The synthetic control method provides a systematic way to choose comparison units in comparative case studies. This systematization opens the door to precise quantitative inference in small‐sample comparative studies, without precluding the application of qualitative approaches. Borrowing the expression from Sidney Tarrow, the synthetic control method allows researchers to put “qualitative flesh on quantitative bones.” We illustrate the main ideas behind the synthetic control method by estimating the economic impact of the 1990 German reunification on West Germany.     

Does political knowledge increase turnout? Evidence from the 1997 British general election

A number of recent formal models predict a positive effect of political knowledge on turnout. Both information acquisition and turnout, however, are likely to be determined by a similar set of variables, rendering hard the identification of a causal link in empirical investigations. Available empirical regularities should therefore be interpreted as mere correlations. I address this problem by using an intrumental variables approach, where the instruments are represented by various proxies of information supply on mass media. Using survey data from the 1997 British General Election Study, I show that political knowledge has a sizeable influence on the probability of voting and that mass media play an important role in influencing political participation.     

Is group memebership a prerequisite for group identification?

The purpose of this research is to examine the theoretical considerations and empirical evidence for the operationalization of an important concept. I attempt to determine if group membership is a necessary condition for group identification and discuss the role of group identifications for translating political interests into political preferences.     

On the Second Edition of Lukes' Third Face

In this article I argue for two propositions. The first is that Lukes has established the possibility of power's third face, but that the most interesting social science questions about it are empirical: How often does it operate and under what conditions? These are topics for empirical research, not armchair reflection. The second proposition I defend is that the most interesting normative questions about power are best thought of as questions of institutional design geared to preventing domination without interfering with the legitimate exercise of power. Examples in support of both propositions are supplied.     

The ideal and reality of epistemic proceduralism

The complex debate about proceduralism in deliberative democratic theory is important for understanding alternative models for bridging theory and practice. In this article, I contrast Jürgen Habermas’ model of epistemic proceduralism with that of David Estlund. I begin by locating the differences between them in terms of contrasting interpretations of Rousseau’s idea of the general will. On this basis, I set out two competing models of democratic proceduralism – an instrumental conception and a constitutive conception – and show how Estlund’s critique of Habermas’ procedural theory of ‘deep deliberative democracy’ mistakenly presupposes that Habermas is committed to an instrumental conception. After clarifying the role of Habermas’ ideal speech situation, I explicate and defend a Habermasian model of reflexive epistemic proceduralism. I conclude by considering the implications of this model for understanding the relationship between normative theory and empirical research.     

Income-based voting and polarization over redistribution under alternative electoral systems

Democracies that have proportional electoral systems spend substantively more on welfare policies than those that have majoritarian systems. Theoretical accounts of this empirical regularity are generally tested using macro-level data, leaving micro-level implications untested. In this paper, I take an alternative approach, leveraging the fact that the theories in question make predictions about the electoral coordination between parties and voters around broad-based redistribution under alternative institutional arrangements. To test the theories, I create a novel measure of income-based voting, which captures the sensitivity of vote choice to changes in income and forms the dependent variable in a second stage model. Overall, I find robust support for more proportionality leading to more income-based voting.