Measuring error for adjacent policy position estimates: Dealing with uncertainty using CMP data

Careful users of CMP party position data should take the uncertainty of position estimates into account. We compare and evaluate two current approaches that provide error estimates for party positions. Researchers of the CMP group identify measurement error in quantitative content analysis as the cause of uncertainty about position estimates, whereas a second approach by Benoit et al. (2009) attributes the uncertainty of position estimates to a stochastic generation of election programs. We illustrate the commonalities and differences of these approaches and provide two empirical applications, the identification of the left–right order of parties and of policy shifts by parties, using CMP data for 25 countries. Despite conceptual differences, results in these applications are surprisingly similar.     

Getting it (approximately) right (and center and left!): Reliability and uncertainty estimates for the comparative manifesto data

The extensive estimates of party policy preferences produced by the CMP (Comparative Manifesto Project, now MARPOR) have proved robust and valid over a wide variety of research applications. But all estimates carry some error. We demonstrate that one of the two existing assessments of non-systematic error in the CMP data strongly overstates their reliability while the other understates it, leaving much potential for mis-estimation. We develop a new method which extends classical test theory and directly estimates overall data reliability; reliabilities and standard errors of measurement for each party system; and standard errors of measurement for each data-point. These should facilitate use of the CMP policy scores which are usually the only ones available for extended party and policy research.     

Assessing the ‘Gold Standard’ of party policy placements: Is computerized replication possible?

The Comparative Manifestos Project (CMP) offers data on party policy positions based on a coding scheme of 56 categories. To what extent can we replicate the CMP coding results by using computerized topical coding of digitalised manifestos? Both human and computer coding have a number of strengths and weaknesses so that the combination of strengths could help to arrive at both valid and reliable party policy placements, in particular the measurement of policy movements over time. More than 1500 digitalised party manifestos in 20 democracies in the period 1960–2009 are re-coded with a computerized CMP-compatible coding scheme in order to assess the differences with human coding using exactly the same issue categories and the same left–right scale.The analysis shows that it is possible to use computer coding in order to locate the potential weakness of the human coding and the other way around. It also illustrates that the validity and reliability of policy placements is a function of the conceptualisation and operationalisation of issues, of the size of documents and of scale construction. Computerized cross-validation of the CMP-coding results offers a new and powerful tool to assess its reliability.     

Treating Words as Data with Error: Uncertainty in Text Statements of Policy Positions

Political text offers extraordinary potential as a source of information about the policy positions of political actors. Despite recent advances in computational text analysis, human interpretative coding of text remains an important source of text-based data, ultimately required to validate more automatic techniques. The profession's main source of cross-national, time-series data on party policy positions comes from the human interpretative coding of party manifestos by the Comparative Manifesto Project (CMP). Despite widespread use of these data, the uncertainty associated with each point estimate has never been available, undermining the value of the dataset as a scientific resource. We propose a remedy. First, we characterize processes by which CMP data are generated. These include inherently stochastic processes of text authorship, as well as of the parsing and coding of observed text by humans. Second, we simulate these error-generating processes by bootstrapping analyses of coded quasi-sentences. This allows us to estimate precise levels of nonsystematic error for every category and scale reported by the CMP for its entire set of 3,000-plus manifestos. Using our estimates of these errors, we show how to correct biased inferences, in recent prominently published work, derived from statistical analyses of error-contaminated CMP data.     

Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis

Political scientists have long been concerned about the validity of survey measurements. Although many have studied classical measurement error in linear regression models where the error is assumed to arise completely at random, in a number of situations the error may be correlated with the outcome. We analyze the impact of differential measurement error on causal estimation. The proposed nonparametric identification analysis avoids arbitrary modeling decisions and formally characterizes the roles of different assumptions. We show the serious consequences of differential misclassification and offer a new sensitivity analysis that allows researchers to evaluate the robustness of their conclusions. Our methods are motivated by a field experiment on democratic deliberations, in which one set of estimates potentially suffers from differential misclassification. We show that an analysis ignoring differential measurement error may considerably overestimate the causal effects. This finding contrasts with the case of classical measurement error, which always yields attenuation bias.     

A Method of Automated Nonparametric Content Analysis for Social Science

The increasing availability of digitized text presents enormous opportunities for social scientists. Yet hand coding many blogs, speeches, government records, newspapers, or other sources of unstructured text is infeasible. Although computer scientists have methods for automated content analysis, most are optimized to classify individual documents, whereas social scientists instead want generalizations about the population of documents, such as the proportion in a given category. Unfortunately, even a method with a high percent of individual documents correctly classified can be hugely biased when estimating category proportions. By directly optimizing for this social science goal, we develop a method that gives approximately unbiased estimates of category proportions even when the optimal classifier performs poorly. We illustrate with diverse data sets, including the daily expressed opinions of thousands of people about the U.S. presidency. We also make available software that implements our methods and large corpora of text for further analysis.     

Politics,policy making,data, and the homeless

Abstract

It is often said that a major obstacle to crafting effective policies concerning home‐lessness is the large uncertainty associated with estimates of the extent of the problem. Such uncertainty is due largely to the difficulties of identifying a “hidden” population. But how true is it that effective policy making for the homeless depends upon counting their population accurately? This paper reviews some critical relationships among politics, policy making, and data; examines data requirements for policy making affecting the homeless; argues the case for relative rather than absolute measurement; and assesses the importance for public policy of data problems in this area.     

Technological Distribution in Uganda: Information and Communications Technology and the State in an Eastern African Nation

Information and communications technologies (ICTs) include old technologies—such as the radio and the television—as well as newer technologies—such as the Internet and wireless telephony. This study considers the process that the government of Uganda has used to adopt and implement ICT policy. This study also considers the techniques which the government of Uganda has used to distribute ICTS in public locations such as government offices, schools, and hospitals. In particular, this study attempts to consider the political motivations for distribution. The Ugandan government's attempt to distribute this technology reflects strengths in the area of distribution of artefacts, particularly to rural areas. Information and communications technologies are an important part of the Ugandan economy. In addition, ICTs strengthen the ability of citizens to communicate with each other across regional and language borders through shared access points. Methodologically, this paper uses the case study method. Semi‐structured interviews were conducted with politicians, policy makers, civil society activists, citizens, academics, medical personnel, regional government officials, and business people. This paper argues that politicians use ICTs as a component of a basket of goods and services that they can distribute to witnessing publics. This paper argues that ICT should be viewed as a type of infrastructure, and that as a public good, it can be used as a “club” good or “pork.” Although several authors discuss the potential of ICTs as democratizing, this paper documents that the Ugandan government has employed ICTs in oppressive ways, including for the surveillance of opposition leaders, and for social control.     

Educating for active compliance: discursive constructions in citizenship education

This article examines the discursive construction of ‘active citizenship’ within recent civics curriculum documents across three provinces in Canada. New secondary school civics curricula have emerged across liberal democratic states since the year 2000, presumably in response to the perception of youth as disengaged from political involvement. Many of the new curricula subsequently emphasize ‘active’ engagement within the polity. The central task of this paper is to better understand what such ‘active citizenship’ actually means, via the methodological tool of discourse analysis. Engaging a theoretical frame that incorporates Foucauldian governmentality theory and cultural theories of the role of the state in creating subjectivities, the paper ultimately argues that the ‘active citizen’ of contemporary civics curricula is, in fact, a deeply neoliberal subject. The article then draws on feminist theories of citizenship in order to assess the forms of exclusion that the curriculum documents inadvertently create, arguing that they ultimately participate in a long tradition of devaluing such elements of citizenship as relationality and emotional ties. We conclude that one of the fundamental goals of citizenship education – to expand access to citizenship participation for all – has failed.     

Indonesia,the ASEAN Socio-Cultural Community,and the contingent profile of regional ‘great-power management’

Why does Indonesia, whose regional leadership would seem to flow so obviously from its material and normative footprint, and whose role as primus inter pares is readily, if sometimes grudgingly, acknowledged by its peers, struggle to translate that position into an unambiguous, consistent, and effective regional presence? While there are many possible answers to this question, this article focuses on societal interpretations, arguing that such a role is exercised not in a vacuum, susceptible to measurement against a set of fixed criteria, but in the context of its respective region's unique and constantly evolving profile. Drawing on the English School's concept of institutions, and focusing on three areas within the ASEAN Socio-Cultural Community (migration, disaster preparedness, and the environment), the article argues that the paradox of Indonesia's powerful-but-not-powerful position reflects the kind of ‘great-power management’ that is to be expected in the restrictive context of regional international society, and is profoundly influenced by the uneven interplay of institutions embedded at different levels in the regional experience.     

How to scale coded text units without bias: A response to Gemenis

Coding non-manifesto documents as if they were genuine policy platforms produced at election time clearly raises serious issues with error when these codings are used in the standard manner to estimate left-right policy positions. In addition to the long term solution of improving the document base of the Manifesto Project identified by Gemenis (2012), we argue that immediate gains in manifesto-based estimates of policy positions can be realised by using the confrontational logit scales from Lowe et al. (2011), which addresses the problems of scale content and scale construction that are exacerbated by but not unique to the problems found in proxy documents.     

The logic of anti-Zionism: Soviet elites in the aftermath of the Six-Day War

Despite the opening of Soviet archives, and the surge in scholarly interest in anti-Zionism, scholars have not used declassified archive documents to shed new light on Soviet anti-Zionism in the wake of the Six-Day War. Based on such documents, Gjerde’s article challenges a view of post-1967 Soviet anti-Zionism that has been prevalent since it emerged during the Cold War: that it represented a ‘disguised’ form of antisemitism that Soviet leaders used as a political tool. To the contrary, Gjerde argues, the archive documents suggest Soviet anti-Zionism was more than a propaganda invention. Within higher Soviet echelons, a particular logic existed that fostered a view of ‘Zionism’ as an immense, conspiratorial threat to the Soviet Union. In one sense, this logic grew out of a more general tendency to view nonconformity as conspiracy: the Soviets had established extremely narrow boundaries for what constituted acceptable Jewish identity; and, when some Soviet Jews began to voice nationalist sentiments after the Six-Day War, Soviet leaders saw this expression of nonconformity as essentially a hostile act, warranting severe counter-measures. This is not to say Soviet anti-Zionism was not antisemitic but rather that to explain it merely as a propaganda tool is to ignore much of the complexity of its emergence.     

Not by Twins Alone: Using the Extended Family Design to Investigate Genetic Influence on Political Beliefs

Variance components estimates of political and social attitudes suggest a substantial level of genetic influence, but the results have been challenged because they rely on data from twins only. In this analysis, we include responses from parents and nontwin full siblings of twins, account for measurement error by using a panel design, and estimate genetic and environmental variance by maximum‐likelihood structural equation modeling. By doing so, we address the central concerns of critics, including that the twin‐only design offers no verification of either the equal environments or random mating assumptions. Moving beyond the twin‐only design leads to the conclusion that for most political and social attitudes, genetic influences account for an even greater proportion of individual differences than reported by studies using more limited data and more elementary estimation techniques. These findings make it increasingly difficult to deny that—however indirectly—genetics plays a role in the formation of political and social attitudes.     

Rating the Rankings: Assessing International Rankings of Public Service Performance

ABSTRACT

This paper documents the growth of international rankings of governance and public services and seeks to contribute to a second-generation approach to the analysis of this phenomenon. It does so primarily by setting out a method for ranking international ratings, building on and extending earlier work by other scholars, and applies that method to 14 international rankings of governance and public services, to explore the scope and limits of the approach. The final section argues that the development of such a method could form the basis for benchmarking international rankings as they develop in the future.     

Toleration and permissiveness

Professor Horowitz argues that it is important for political sociology to try to understand and analyse genocide as a distinct social phenomenon. For this purpose it must be clearly distinguished from other forms of human destruction, such as natural disasters, random killing, warfare and symbolic or cultural assaults. The collective nature of genocide sets it apart from other social evils, as it contradicts traditional Western approaches to law and morals which emphasize individual responsibility for actions.     

Estimating parties' left-right positions: Determinants of voters' perceptions' proximity to party ideology

The article aims to explore whether the accuracy of voters' perceptions of party ideology are affected by party position shifts and by the media's turn to non-left-right issues, such as political leadership, during election campaigns. Using data from the Comparative Study of Electoral Systems (CSES) and a country-specific left-right index based on data by the Comparative Manifestos Project (CMP), multilevel analyses reveal that emphasizing leadership issues can lead voters to reflect left-right positions more accurately. A party's left-right position shift between elections does not lead to a significant difference in voters' perceptions, while a shift on the economic sub-dimension of left-right ideology can even lead voters to more position clarity. However, multiple parties' shifting their positions seems to overburden respondents' cognitive capacity.     

Positions and saliency of immigration in party manifestos: A novel dataset using crowd coding

Immigration is one of the most widely debated issues today. It has, therefore, also become an important issue in party competition, and radical right parties are trying to exploit the issue. This opens up many pressing questions for researchers. To answer these questions, data on the self‐ascribed and unified party positions on immigration and immigrant integration issues is needed. So far, researchers have relied on expert survey data, media analysis data and ‘proxy’ categories from the Manifesto Project Dataset. However, the former two only give the mediated party position, and the latter relies on proxies that do not specifically measure immigration. The new dataset presented in this article provides researchers with party positions and saliency estimates on two issue dimensions – immigration and immigrant integration – in 14 countries and 43 elections. Deriving the data from manifestos enables the provision of parties’ unified and unfiltered immigration positions for countries and time points not covered in expert surveys and media studies, making it possible to link immigration and immigrant integration positions and saliency scores to other issue areas covered in the Manifesto Project Dataset. Well‐established criteria are used to distinguish between statements on (1) immigration control and (2) immigrant integration. This allows for a more fine‐grained analysis along these two dimensions. Furthermore, the dataset has been generated using the new method of crowd coding, which allows a relatively fast manual coding of political texts. Some of the advantages of crowd coding are that it is easily replicated and expanded, and, as such, presents the research community with the opportunity to amend and expand upon this coding scheme.     

Alienation and Belonging - Women's Citizenship and Emancipation: Visions for Sudan's Post-Islamist Future

This essay examines Sudanese Islamist debates about the position of women within an Islamic framework, oppositional groups' stances on the nature of a post-Islamist Sudan and women's role in the nation. The author critiques oppositional groups for a lack of vision for a post-Islamist gender egalitarian Sudan and feminism for its lack of clarity about the concept of women's emancipation. The author argues that all groups in Sudan have not extended a visionary approach to women, but have been limited to expressions about "women's rights." Using the concepts of "emancipation," "gender egalitarianism," "citizenship," "alienation," "belonging," and "subject," the author deconstructs segments of crucial political documents such as Islamic decrees, the new Sudanese Constitution (1998), the "Asmara Declaration" of the National Democratic Alliance, and various statements by political parties in exile. Using excerpts from women's narratives, the author attempts to illuminate Sudanese women's self-identification, belonging, and citizenship.     

Back to the Archives? A Critique of the Danish Part of the Manifesto Dataset

The dataset of the Comparative Manifesto Project (CMP) is one the best known and often used datasets in comparative studies of Western Europe. However, the Danish part of the data is very problematic. On average, nearly a third of the content of the Danish manifestos is deemed impossible to code. A close examination of the Danish data reveals large variations in the number of coded sentences across time and parties. An analysis of the manifestos on which the coding is based sets all the warning lights flashing. First, not all manifestos are in fact manifestos; a number consist of other political texts (i.e. speeches, drafts) or even stem from different elections (e.g. municipal manifestos masquerading as national manifestos). Second, the large variation in the length of the manifestos across time suggests that the information derived from them is not readily comparable. This leads to a distorted picture of the positions of the Danish parties. The conclusion of this article is that in order for the Danish part of the CMP dataset to become more trustworthy, it is necessary to search the archives, find the actual manifestos and recode them.     

EU Democracy Assistance Discourse in Its New Response to a Changing Neighbourhood

Drawing on Critical Discourse Analysis, this article compares the structure of key policy documents on European Union (EU) democracy promotion in the Southern Neighborhood before and after the “Arab Uprisings.” With reference to the key document presenting the EU's revised conception of democracy and strategic vision in the Southern Neighborhood, this article argues that, despite assertions of a paradigmatic shift in the EU's approach to democracy, the conceptual structure of these documents maintains unaltered the substantively liberal model for both development and democratization. This is likely to leave the EU's pre-Uprisings reputational deficit concerning democracy promotion unaltered.