Can experts judge elections? Testing the validity of expert judgments for measuring election integrity

Expert surveys have been used to measure a wide variety of phenomena in political science, ranging from party positions, to corruption, to the quality of democracy and elections. However, expert judgments raise important validity concerns, both about the object being measured as well as the experts. It is argued in this article that the context of evaluation is also important to consider when assessing the validity of expert surveys. This is even more important for expert surveys with a comprehensive, worldwide scope, such as democracy or corruption indices. This article tests the validity of expert judgments about election integrity – a topic of increasing concern to both the international community and academics. Evaluating expert judgments of election integrity provides an important contribution to the literature evaluating the validity of expert surveys as instruments of measurement as: (1) the object under study is particularly complex to define and multifaceted; and (2) election integrity is measured in widely varying institutional contexts, ranging from electoral autocracies to liberal democracies. Three potential sources of bias are analysed (the object, the experts and the context), using a unique new dataset on election integrity entitled the ‘Perceptions of Electoral Integrity’ dataset. The data include over 800 experts in 66 parliamentary and presidential elections worldwide. It is found that validity of expert judgments about election integrity is increased if experts are asked to provide factual information (rather than evaluative judgments), and if they are asked to evaluate election day (rather than pre‐election) integrity. It is also found that ideologically polarised elections and elections of lower integrity increase expert disagreement about election integrity. The article concludes with suggestions for researchers using the expert survey data on election integrity on how to check the validity of their data and adjust their analyses accordingly, and outlines some remaining challenges for future data collection using expert surveys.     

Measuring party positions and issue salience from media coverage: Discussing and cross-validating new indicators

Recent studies have started to use media data to measure party positions and issue salience. The aim of this article is to compare and cross-validate this alternative approach with the more commonly used party manifestos, expert judgments and mass surveys. To this purpose, we present two methods to generate indicators of party positions and issue salience from media coverage: the core sentence approach and political claims analysis. Our cross-validation shows that with regard to party positions, indicators derived from the media converge with traditionally used measurements from party manifestos, mass surveys and expert judgments, but that salience indicators measure different underlying constructs. We conclude with a discussion of specific research questions for which media data offer potential advantages over more established methods.     

Ethnic outbidding and nested competition: Explaining the extremism of ethnonational minority parties in Europe

The classical outbidding model of ethnic politics argues that democratic competition involving ethnic parties inevitably leads to ethnic outbidding where parties adopt ever more extreme positions. However, recent small‐N studies show that ethnic outbidding is only one of a range of strategies available to ethnic parties. This article seeks to explain why some ethnic parties are extremist, whereas others adopt moderate positions. Drawing on the ethnic outbidding and the nested competition model of ethnic party competition, it is hypothesised that the ethnic segmentation of the electoral market, and the relative salience of an ethnically cross‐cutting economic dimension of party competition, account for the varying degrees of extremism. Hypotheses are tested drawing on a novel, expert‐survey‐based dataset that provides indicators for the positions of 83 ethnonational minority parties in 22 European democracies in 2011. Results of ordinary least squares and two‐level linear regressions show that as the economic dimension gains importance, parties become more moderate relative to the party system mean. The electorate's ethnic segmentation has a positive effect on extremism, but this effect is not significant in all models. Contrary to expectations, higher ethnic segmentation of the party system is associated with more moderate positions in the majority of the estimated models.     

The second edition of the EPAC expert survey on ethnonationalism in party competition – testing for validity and reliability

This research note presents EPAC 2017, a dataset resulting from the second round of an expert survey on ethnonationalism in party competition. EPAC provides cross-sectional data on the positions of (ethno-) national and mainstream parties on an ethnonational (also often referred to as ‘territorial’ or ‘centre-periphery’) dimension, as well as other important dimensions of political competition. The 2017 edition covers 222 political parties in 22 multinational European countries. The research note presents the main survey items and performs a series of validity and reliability tests on the data. Results show that EPAC 2017 provides valid and reliable measures of party positions on an ethnonational dimension. A short analysis of party system changes in Spain and Bosnia and Herzegovina illustrates the opportunities of combining the 2011 and 2017 editions. The combined dataset allows studying the mobilization of the centre-periphery cleavage in party competition across Eastern and Western Europe and over time.     

Measuring party orientations towards European integration: Results from an expert survey

Abstract Some observers have held that political parties have been minor players in the process of European integration due to the low salience of the issue and the prevalence of intra party disagreement over European questions. Although recent scholarship and the rising salience of European issues have brought increased attention to the role of political parties, the study of the relationship between party positions and both public opinion and policy outcomes has been hampered by an absence of comparable data on party positions. This research note presents the findings of an expert survey on party positions on the issue of European integration. In addition to estimates of the parties' positions on the issue itself, this survey provides information on the importance of the issue of European integration to each party, and the extent of internal dissent within parties. The data also indicate that parties have, on average, become increasingly pro–European over the period 1984–1996. Both the salience of the issue of integration and the extent of intra–party disagreement have increased during this period. However, deep intra–party divisions appear less prevalent than commonly believed.     

Positions and saliency of immigration in party manifestos: A novel dataset using crowd coding

Immigration is one of the most widely debated issues today. It has, therefore, also become an important issue in party competition, and radical right parties are trying to exploit the issue. This opens up many pressing questions for researchers. To answer these questions, data on the self‐ascribed and unified party positions on immigration and immigrant integration issues is needed. So far, researchers have relied on expert survey data, media analysis data and ‘proxy’ categories from the Manifesto Project Dataset. However, the former two only give the mediated party position, and the latter relies on proxies that do not specifically measure immigration. The new dataset presented in this article provides researchers with party positions and saliency estimates on two issue dimensions – immigration and immigrant integration – in 14 countries and 43 elections. Deriving the data from manifestos enables the provision of parties’ unified and unfiltered immigration positions for countries and time points not covered in expert surveys and media studies, making it possible to link immigration and immigrant integration positions and saliency scores to other issue areas covered in the Manifesto Project Dataset. Well‐established criteria are used to distinguish between statements on (1) immigration control and (2) immigrant integration. This allows for a more fine‐grained analysis along these two dimensions. Furthermore, the dataset has been generated using the new method of crowd coding, which allows a relatively fast manual coding of political texts. Some of the advantages of crowd coding are that it is easily replicated and expanded, and, as such, presents the research community with the opportunity to amend and expand upon this coding scheme.     

Measuring party orientations towards European integration: results from an expert survey

Some observers have held that political parties have been minor players in the process of European integration due to the low salience of the issue and the prevalence of intra party disagreement over European questions. Although recent scholarship and the rising salience of European issues have brought increased attention to the role of political parties, the study of the relationship between party positions and both public opinion and policy outcomes has been hampered by an absence of comparable data on party positions. This research note presents the findings of an expert survey on party positions on the issue of European integration. In addition to estimates of the parties' positions on the issue itself, this survey provides information on the importance of the issue of European integration to each party, and the extent of internal dissent within parties. The data also indicate that parties have, on average, become increasingly pro–European over the period 1984–1996. Both the salience of the issue of integration and the extent of intra–party disagreement have increased during this period. However, deep intra–party divisions appear less prevalent than commonly believed.     

Estimating policy positions using political texts: An evaluation of the Wordscores approach

This paper evaluates a recently developed method for extracting policy positions from political texts, known as Wordscores. This computerized content analysis technique is a potentially powerful tool for scholars interested in the study of political elites, since it promises an easy and efficient way of inferring policy position from texts and speeches. In this article, we provide a systematic evaluation of this promising method. Using Danish manifestos and government speeches from 1945 to 2005, we compare the policy positions extracted using Wordscores with measures of positions from the well-known Comparative Manifesto Project and cross-validate these with party expert surveys. Our analysis shows that the word scoring technique arrives at largely similar estimates to independently derived position measures and produces time series of government positions with high face validity.     

When do parties emphasise extreme positions? How strategic incentives for policy differentiation influence issue importance

Parties have an incentive to take up extreme positions in order to achieve policy differentiation and issue ownership, and it would make sense for a party to stress these positions as well. These incentives are not the same for all issues and all parties but may be modified by other strategic conditions: party size, party system size, positional distinctiveness and systemic salience. Using manifesto‐based measures of salience and expert assessments of party positions, the findings in this article are that parties emphasise extreme positions if, first, they are relatively small in terms of vote share; second, the extreme position is distinctive from those of other parties; and third, other parties fail to emphasise the issue. These findings have consequences for our understanding of party strategies, party competition and the radicalisation of political debates.     

A problem with empirical studies of party policy shifts: Alternative measures of party shifts are uncorrelated

Many recent cross-national studies analyse the causes and electoral consequences of party policy shifts, using party position measures derived from election manifestos, expert surveys or voter surveys. However few studies validate their findings by analysing multiple measures of party policy shifts. In this article, data on European parties’ position shifts on both European integration and left-right ideology is analysed, showing that this is problematic because, while alternative measures of party policy positions correlate strongly in cross-sectional analyses, alternative measures of parties’ policy shifts are essentially uncorrelated in longitudinal analyses. Suggestions are offered on how to address this problem.     

The Mobilizing Effect of Parties' Moral Rhetoric

How does parties' use of moral rhetoric affect voter behavior? Prior comparative party research has studied party positions without much attention to how parties explain and justify their positions. Drawing insights from political and moral psychology, I argue that moral rhetoric mobilizes copartisan voters by activating positive emotions about their partisan preference. I expect this to hold among copartisans who are exposed to party rhetoric. To test my argument, I measure moral rhetoric by text-analyzing party manifestos from six English-speaking democracies and measure mobilization using copartisan turnout in survey data. The results support my argument. Furthermore, I find evidence in support of the theoretical mechanism using survey experiments and panel survey data from Britain. The article shows that moral rhetoric is a party campaign frame that has important consequences for voter behavior.     

Immigration, Europe and the 'new' cultural dimension

Kriesi et al. announced the birth of a new cleavage in contemporary Western Europe, one dividing the winners and losers of globalisation. Their studies in 2006 and 2008 contain analyses of party positions in six countries, based on the contents of editorial sections of newspapers. This article challenges the main conclusion of Kriesi et al. by demonstrating − on the basis of two expert surveys − that party positions are mainly structured by one dimension. The structure detected by Kriesi et al. in their analysis of parties is not found, except concerning voter positions. A consequence of this article's findings is that large groups of citizens are not represented by any parties, in particular those who are left-wing on socio-economic issues and right-wing on cultural issues. The article in its conclusion discusses possible causes for the differences between these findings and those of Kriesi et al., and the implications of these findings for democratic representation.     

Does Political Information Matter? An Experimental Test Relating to Party Positions on Europe

This article focuses on whether the provision of 'objectively' correct information to voters about where parties stand on an issue affects their placement of the parties, and ultimately their own position, on that issue. Classic theories of how mass publics make voting decisions assume that voters are able relatively accurately to place themselves and the parties on various issue dimensions. While these assumptions have been challenged, it is generally assumed that the provision of new information makes voters' placements more informed. We explicitly test this idea using a survey experiment focusing on one political issue – European integration. In the experiment, all respondents were twice asked to place the three main British parties and themselves on a bipolar scale of European integration. This was done towards the beginning, and then at the end of the survey. Most respondents were also given information on the 'informed' positions of the parties, derived from expert survey placement. Our analyses indicate that individuals' placements did change, and the tendency was related to both political sophistication and the inherent difficulty of placing the party. Only less sophisticated voters updated their placements, and these changes are concentrated on the placement of the Labour party, where the elite stance on Europe has been more conflicted. For all respondents we do not detect any corresponding changes in self-placement that would be congruent with 'cueing' effects.     

Still blurry? Economic salience,position and voting for radical right parties in Western Europe

Do radical right parties present blurry economic stances, or have they clarified their positions while moving towards the economic left? This article questions the strategic behaviour of radical right parties in Western Europe. It shows that although radical right parties have increased their discussion of economic issues, and expert placements of this party family on the economic dimension have become more centrist over time, the uncertainty surrounding these placements continues to be higher for the radical right than any other party family in Europe. The article then moves on to examine to what extent voter-party congruence on redistribution, immigration and other issues of social lifestyle predict an individual's propensity to vote for the radical right compared to other parties. Although redistribution is the component of economic policy where the radical right seems to be centrist, the findings indicate that it remains party-voter congruence on immigration that drives support for radical right parties, while the congruence level for redistribution has an insignificant effect. The article concludes that while radical right parties seem to have included some clearly left-leaning economic proposals, which shifted the general expert views of these parties to the economic centre, their overall economic profiles remain as blurry as ever.     

Do Policy Horizons Structure the Formation of Parliamentary Governments?: The Evidence from an Expert Survey

This article tests the hypothesis that parties in West European parliamentary systems operate under the constraint of "policy horizons," that is, limits or bounds on the extent to which they can compromise on policy for the purpose of entering coalition governments. The test is based on a new expert survey covering thirteen West European parliamentary democracies in which respondents were asked not only to locate party positions on a number of policy dimensions, but also to estimate the parties' limits of acceptable compromise on each dimension. The survey data are first analyzed using a new computer program, Horizons 3D, to determine which parties have intersecting horizons in these systems—and hence the ability to form coalition governments under the hypothesis. These calculations are then employed to assess whether policy horizons structure the choice of governing coalition beyond any effect conveyed by the policy distances among parties. Although the potential for error in these data is considerable, the estimated horizons, with few exceptions, appear to play the role hypothesized for them .     

Party leadership and party institutionalisation: Three phases of development

The use of the party manifesto data (PMD) to identify parties' position in the political space provides a rather distorted picture of the Italian party system. Three possible explanations for this are explored, namely that the Italian party system is exceptional, that there are flaws in the data and there might be flaws in the methodology. The article argues that none of these explanations is fully satisfactory and advances the hypothesis that the PMD left–right scores do not indicate parties' positions but instead indicate parties' direction, that is how (and how much) parties move to adjust to changing political conditions and to remain competitive. Statistical analyses, performed to test the validity of the directional interpretation of the left–right scores, support this new interpretation.     

Parties are always right: the effects of party cues and policy information on attitudes towards EU issues

Abstract

This article analyses the influence that political parties exert upon citizens’ opinions about European Union issues. By measuring at the same time the content and source effects on political attitudes, the article considers the possibility that voters pay less attention to the arguments used in a political message than to its source. Results from an online survey experiment in Spain show that partisan voters use a heuristic model of processing when taking positions on an unfamiliar EU issue, even though the prevalence of the source effect is moderated by the respondent’s political sophistication and party attachment. The results also indicate that some respondents tend to pay less attention to a message’s content when the message comes from their preferred party. Such findings raise concerns about the possibility for EU issue voting to guarantee the accountability of political elites and party–voter linkages.     

Representation as a median mandate: Taking cross‐national differences seriously

The extent and ways in which popular preferences influence government policy are absolutely central to our understanding of modern democracy. Paul Warwick's discussion of these in the European Journal of Political Research in 2010 puts itself at the heart of the debate with its critique of the median mandate theory of McDonald and Budge, proposing an alternative ‘bilateralist’ concept of representation. This article questions whether this concept has much to add to our theoretical understanding of representational processes. However, Warwick's further conceptual points deserve serious consideration. These concern the time horizons within which representative processes work, and the status of the median position given multi‐motivated voting. At the evidential level, Warwick argues that survey‐based measures of voter and party left–right positions fail to produce the correspondence between median and government policy positions that median mandate theory would have us expect. However, survey‐based measures of median voter and party placements obscure important cross‐national variation. Using the Comparative Study of Electoral Systems ( CSES 2007 ), as Warwick does, this article shows that survey respondents norm their own and their country's party positions to their national context. The consequence is to make the political centre in all nations appear similar. Allowing for the relevant cross‐national differences brings the relationship between the median voter and government position back in line with expectations.     

Do voters learn? Evidence that voters respond accurately to changes in political parties’ policy positions

A premise of the mass–elite linkage at the heart of representative democracy is that voters notice changes in political parties’ policy positions and update their party perceptions accordingly. However, recent studies question the ability of voters accurately to perceive changes in parties’ positions. The study advances this literature with a two-wave panel survey design that measured voters’ perception of party positions before and after a major policy shift by parties in the government coalition in Denmark 2011–2013. Two key findings extend previous work. First, voters do indeed pay attention to parties when they visibly change policy position. Second, voters update their perceptions of the party positions much more accurately than would have been expected if they merely relied on a ‘coalition heuristic’ as a rule-of-thumb. These findings imply that under some conditions voters are better able to make meaningful political choices than previous work suggests.     

The Company You Keep: How Voters Infer Party Positions on European Integration from Governing Coalition Arrangements

Recent studies document that voters infer parties’ left‐right positions from governing coalition arrangements. We show that citizens extend this coalition‐based heuristic to the European integration dimension and, furthermore, that citizens’ coalition‐based inferences on this issue conflict with alternative measures of party positions derived from election manifestos and expert placements. We also show that citizens’ perceptions of party positions on Europe matter, in that they drive substantial partisan sorting in the electorate. Our findings have implications for parties’ election strategies and for mass‐elite policy linkages.