Network Analysis and the Law: Measuring the Legal Importance of Precedents at the U.S. Supreme Court

Timothy R. Johnson Department of Political Science, University of Minnesota, Twin Cities, 1414 Social Sciences Building, 267 19th Ave. South, Minneapolis, MN 55455 e-mail: trj{at}umn.edu James F. Spriggs, II Department of Political Science, Washington University in St. Louis, Campus Box 1063, One Brookings Drive, St Louis, MO 63130 e-mail: jspriggs{at}artsci.wustl.edu Sangick Jeon Department of Political Science, Stanford University, 616 Serra Street, Encina Hall West, Room 100, Stanford, CA 94305-6044 e-mail: sjeon{at}stanford.edu Paul J. Wahlbeck Department of Political Science, George Washington University, 1922 F Street, N.W. Suite 401, Washington, DC 20052 e-mail: wahlbeck{at}gwu.edu e-mail: jhfowler{at}ucsd.edu (corresponding author) We construct the complete network of 26,681 majority opinionswritten by the U.S. Supreme Court and the cases that cite themfrom 1791 to 2005. We describe a method for using the patternsin citations within and across cases to create importance scoresthat identify the most legally relevant precedents in the networkof Supreme Court law at any given point in time. Our measuresare superior to existing network-based alternatives and, forexample, offer information regarding case importance not evidentin simple citation counts. We also demonstrate the validityof our measures by showing that they are strongly correlatedwith the future citation behavior of state courts, the U.S.Courts of Appeals, and the U.S. Supreme Court. In so doing,we show that network analysis is a viable way of measuring howcentral a case is to law at the Court and suggest that it canbe used to measure other legal concepts. Authors' note: We appreciate the suggestions of Randy Calvert,Frank Cross, Pauline Kim, Andrew Martin, Richard Pacelle, JimRogers, Margo Schlanger, Amy Steigerwalt, and participants inthe Workshop on Empirical Research in the Law at WashingtonUniversity in St Louis School of Law. We presented former versionsof this article at the 2006 meeting of the Midwest PoliticalScience Association, Chicago, April 20–23; the 2006 meetingof the Southern Political Science Association, Atlanta, GA,January 5–7; and the 2006 Empirical Legal Studies Conference,Austin, TX, October 27–28.     

Shirking in the Contemporary Congress: A Reappraisal

Michael H. Crespin Department of Political Science, Michigan State University, 303 S. Kedzie Hall, East Lansing, MI 48824 e-mail: e-mail: crespinm{at}msu.edu Jeffery A. Jenkins Department of Political Science, Northwestern University, 601 University Place, Evanston, IL 60208 e-mail: e-mail: j-jenkins3{at}northwestern.edu Ryan J. Vander Wielen Department of Political Science, Washington University in St. Louis, Campus Box 1027, One Brookings Drive, St. Louis, MO 63130 e-mail: e-mail: rjvander{at}artsci.wustl.edu This paper replicates the findings that appeared in the article"Severing the Electoral Connection: Shirking in the ContemporaryCongress," published in the American Journal of Political Science(44:316–325), in which Lawrence Rothenberg and MitchellSanders incorporated a new research design and, contrary toall previous studies, found evidence of ideological shirkingin the U.S. House of Representatives. We investigate the robustnessof their results by reestimating their model with Congress-specificfixed effects and find that their results no longer hold.     

The Search for Comparability: Response to Binder

Lawrence S. Rothenberg Department of Political Science, University of Rochester, Rochester, NY 14627 e-mail: lrot{at}mail.rochester.edu (corresponding author) Binder (n.d., Taking the measure of Congress: Reply to Chiouand Rothenberg. Political Analysis. Forthcoming) highlightsareas of agreement and disagreement with our discussion of preferencemeasurement and legislative gridlock. We now both agree thatW-NOMINATE scores—employed in Binder (1999, The dynamicsof legislative gridlock. American Political Science Review 9:519–33)to measure key independent variables, including bicameral differences—shouldnot be used when examining multichamber legislatures over time.We continue to disagree over whether Common Space scores orBinder's conference vote measure is superior. In this response,we show that, although several of the theoretical and statisticalobjections that Binder (n.d.) raises to our Common Space measuredo not apply, they are all relevant for her conference voteanalog. Additionally, we detail how, despite protests to thecontrary, the conference vote measure is plagued by insufficientdata. Finally, we demonstrate how new efforts to show that Binder's (1999)results continue to hold are not robust.     

Comparing Legislators and Legislatures: The Dynamics of Legislative Gridlock Reconsidered

Lawrence S. Rothenberg Department of Political Science, University of Rochester, Rochester, NY 14627 e-mail: lrot{at}mail.rochester.edu (corresponding author) Although political methodologists are well aware of measurementissues and the problems that can be created, such concerns arenot always front and center when we are doing substantive research.Here, we show how choices in measuring legislative preferenceshave influenced our understanding of what determines legislativeoutputs. Specifically, we replicate and extend Binder's highlyinfluential analysis (Binder, Sarah A. 1999. The dynamics oflegislative gridlock, 1947–96. American Political ScienceReview 93:519–33; see also Binder, Sarah A. 2003. Stalemate:Causes and consequences of legislative gridlock. Washington,DC: Brookings Institution) of legislative gridlock, which emphasizeshow partisan, electoral, and institutional characteristics generatemajor legislative initiatives. Binder purports to show thatexamining the proportion, rather than the absolute number, ofkey policy proposals passed leads to the inference that thesefeatures, rather than divided government, are crucial for explaininggridlock. However, we demonstrate that this finding is underminedby flaws in preference measurement. Binder's results are a functionof using W-NOMINATE scores never designed for comparing Senateto House members or for analyzing multiple Congresses jointly.When preferences are more appropriately measured with commonspace scores (Poole, Keith T. 1998. Recovering a basic spacefrom a set of issue scales. American Journal of Political Science42:964–93), there is no evidence that the factors thatshe highlights matter. Authors' note: Thanks to Sarah Binder and Keith Poole for furnishingdata used in our analysis and to Chris Achen and Kevin Clarkefor advice. All errors remain our own. Online appendix is availableon the Political Analysis Web site.     

A Robust Transformation Procedure for Interpreting Political Text

Georg Vanberg Department of Political Science, University of North Carolina, Chapel Hill, NC 27599-3265 e-mail: gvanberg{at}unc.edu (corresponding author) In a recent article in the American Political Science Review,Laver, Benoit, and Garry (2003, "Extracting policy positionsfrom political texts using words as data," 97:311–331)propose a new method for conducting content analysis. TheirWordscores approach, by automating text-coding procedures, representsan advance in content analysis that will potentially have alarge long-term impact on research across the discipline. Toallow substantive interpretation, the scores produced by theWordscores procedure require transformation. In this note, weaddress several shortcomings in the transformation procedureintroduced in the original program. We demonstrate that theoriginal transformation distorts the metric on which contentscores are placed—hindering the ability of scholars tomake meaningful comparisons across texts—and that it isvery sensitive to the texts that are scored—opening upthe possibility that researchers may generate, inadvertentlyor not, results that depend on the texts they choose to includein their analyses. We propose a transformation procedure thatsolves these problems. Authors' note: We would like to thank Ken Benoit, Michael Laver,three anonymous referees, and the editor for comments on earlierversions of this article.     

Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls

Andrew Gelman Departments of Statistics and Political Science, Columbia University, New York, NY 10027 e-mail: gelman{at}stat.columbia.edu Joseph Bafumi Department of Political Science, Columbia University, New York, NY 10027 We fit a multilevel logistic regression model for the mean ofa binary response variable conditional on poststratificationcells. This approach combines the modeling approach often usedin small-area estimation with the population information usedin poststratification (see Gelman and Little 1997, Survey Methodology23:127–135). To validate the method, we apply it to U.S.preelection polls for 1988 and 1992, poststratified by state,region, and the usual demographic variables. We evaluate themodel by comparing it to state-level election outcomes. Themultilevel model outperforms more commonly used models in politicalscience. We envision the most important usage of this methodto be not forecasting elections but estimating public opinionon a variety of issues at the state level.     

The Constrained Instability of Majority Rule: Experiments on the Robustness of the Uncovered Set

Michael S. Lynch Department of Political Science, University of Kansas, 504 Blake Hall, Lawrence, KS 66044 e-mail: mlynch{at}ku.edu Gary J. Miller and Itai Sened Department of Political Science, Washington University in St. Louis, Campus Box 1063, One Brooking Drive, St. Louis, MO 63130 e-mail: gjmiller{at}wustl.edu e-mail: sened{at}wustl.edu (corresponding author) The uncovered set has frequently been proposed as a solutionconcept for majority rule settings. This paper tests this propositionusing a new technique for estimating uncovered sets and a seriesof experiments, including five-player computer-mediated experimentsand 35-player paper-format experiments. The results supportthe theoretic appeal of the uncovered set. Outcomes overwhelminglylie in or near the uncovered set. Furthermore, when preferencesshift, outcomes track the uncovered set. Although outcomes tendto occur within the uncovered set, they are not necessarilystable; majority dominance relationships still produce instability,albeit constrained by the uncovered set. Authors' note: We thank Matthew M. Schneider for research assistance.We thank James Holloway, Tse-Min Lin, Jim Granato, Randall L.Calvert, Rick K. Wilson, faculty and students of the Juan MarchInstitute, and reviewers of Political Analysis for their veryhelpful comments and suggestions.     

A Bayesian Multilevel Modeling Approach to Time-Series Cross-Sectional Data

Joseph Bafumi Department of Government, Dartmouth College,6108 Silsby HallHanover, NH 03755 e-mail: joseph.bafumi{at}dartmouth.edu Luke Keele Department of Political Science, Ohio State University,2137 Derby Hall, 154 N Oval Mall, Columbus, OH 43210 e-mail: keele.4{at}polisci.osu.edu David Park Department of Political Science, George Washington University,1922 F Street, N.W. 414C, Washington, DC 20052 e-mail: dkp{at}gwu.edu e-mail: bshor{at}uchicago.edu (corresponding author) The analysis of time-series cross-sectional (TSCS) data hasbecome increasingly popular in political science. Meanwhile,political scientists are also becoming more interested in theuse of multilevel models (MLM). However, little work existsto understand the benefits of multilevel modeling when appliedto TSCS data. We employ Monte Carlo simulations to benchmarkthe performance of a Bayesian multilevel model for TSCS data.We find that the MLM performs as well or better than other commonestimators for such data. Most importantly, the MLM is moregeneral and offers researchers additional advantages. Authors' note: A previous version of this article was presentedat the 2005 Midwest Political Science Meeting. We would liketo thank the following for comments and advice in writing thispaper: Andrew Gelman, Nathaniel Beck, Greg Wawro, Sam Cooke,John Londregan, David Brandt. Any errors are our own.     

Statistical Backwards Induction: A Simple Method for Estimating Recursive Strategic Models

Curtis S. Signorino 303 Harkness Hall, Department of Political Science, University of Rochester, Rochester, NY 14627 e-mail: curt.signorino{at}rochester.edu Robert W. Walker Department of Political Science, Center for Applied Statistics, Washington University in Saint Louis, Campus Box 1063, One Brookings Drive, St. Louis, MO 63130 e-mail: rww{at}wustl.edu e-mail: mbas{at}gov.harvard.edu (corresponding author) We present a simple method for estimating regressions basedon recursive extensive-form games. Our procedure, which canbe implemented in most standard statistical packages, involvessequentially estimating standard logits (or probits) in a manneranalogous to backwards induction. We demonstrate that the techniqueproduces consistent parameter estimates and show how to calculateconsistent standard errors. To illustrate the method, we replicateLeblang's (2003) study of speculative attacks by financial marketsand government responses to these attacks. Authors' note: Our thanks to Kevin Clarke, John Londregan, JeffRitter, Ahmer Tarar, and Kuzey Yilmaz for helpful discussionsconcerning this paper. A previous version was presented at the2002 Political Methodology Summer Meeting.     

Practical Issues in Implementing and Understanding Bayesian Ideal Point Estimation

Andrew Gelman Department of Statistics and Department of Political Science, Columbia University, New York, NY e-mail: gelman{at}stat.columbia.edu, www.stat.columbia.edu/~gelman/ David K. Park Department of Political Science, Washington University, St. Louis, MO e-mail: dpark{at}artsci.wustl.edu Noah Kaplan Department of Political Science, University of Houston, Houston, TX e-mail: nkaplan{at}uh.edu Logistic regression models have been used in political sciencefor estimating ideal points of legislators and Supreme Courtjustices. These models present estimation and identifiabilitychallenges, such as improper variance estimates, scale and translationinvariance, reflection invariance, and issues with outliers.We address these issues using Bayesian hierarchical modeling,linear transformations, informative regression predictors, andexplicit modeling for outliers. In addition, we explore newways to usefully display inferences and check model fit.     

Comparing Incomparable Survey Responses: Evaluating and Selecting Anchoring Vignettes

Jonathan Wand Department of Political Science, Encina Hall, Room 308 West, Stanford University, Stanford, CA 94305-6044 e-mail: wand{at}stanford.edu e-mail: king{at}harvard.edu (corresponding author) When respondents use the ordinal response categories of standardsurvey questions in different ways, the validity of analysesbased on the resulting data can be biased. Anchoring vignettesis a survey design technique, introduced by King et al. (2004,Enhancing the validity and cross-cultural comparability of measurementin survey research. American Political Science Review 94 [February]:191–205), intended to correct for some of these problems.We develop new methods both for evaluating and choosing anchoringvignettes and for analyzing the resulting data. With surveyson a diverse range of topics in a range of countries, we illustratehow our proposed methods can improve the ability of anchoringvignettes to extract information from survey data, as well assaving in survey administration costs.     

Much Ado About Very Little: Reply to Herron

Mitchell S. Sanders Department of Political Science, University of Notre Dame, Notre Dame, IN 46556 e-mail: msander1{at}nd.edu While Herron (2004, Political Analysis 12:182–190) iscorrect that sensitivity to changes in underlying scale andhow they affect estimates and inferences is generally important,our assumption in Rothenberg and Sanders (2000, American Journalof Political Science 44:310–319) that W-NOMINATE scalescan be directly compared from one Congress to another to studylegislative shirking is quite defensible because scale variabilityis not a substantial problem. Not only are the assumptions inour original analysis regarding variability very reasonable,because any variability is quite small, but effects on consistencyare marginal and, to the degree that they are relevant, indicatethat our test of the shirking hypothesis is conservative. Furthermore,even generous estimates of variability in W-NOMINATE betweenone immediate Congress and another have little impact on results.In addition, Herron's analysis includes an unaddressed censoringproblem that again, while unlikely to have much substantiverelevance, indicates that Rothenberg and Sanders have workedagainst themselves in trying to find shirking. In conclusion,the issues that Herron highlights are of marginal consequencefor the original analysis and, to the extent they matter, onlybuttress the findings generated and the inferences drawn.     

Empirical versus Theoretical Claims about Extreme Counterfactuals: A Response

Langche Zeng Department of Political Science, University of California, San Diego, La Jolla, CA 92093 e-mail: lazeng{at}ucsd.edu In response to the data-based measures of model dependence proposedin King and Zeng (2006), Sambanis and Michaelides (2008) proposealternative measures that rely upon assumptions untestable inobservational data. If these assumptions are correct, then theirmeasures are appropriate and ours, based solely on the empiricaldata, may be too conservative. If instead, and as is usuallythe case, the researcher is not certain of the precise functionalform of the data generating process, the distribution from whichthe data are drawn, and the applicability of these modelingassumptions to new counterfactuals, then the data-based measuresproposed in King and Zeng (2006) are much preferred. After all,the point of model dependence checks is to verify empirically,rather than to stipulate by assumption, the effects of modelingassumptions on counterfactual inferences. Author’s note: Easy-to-use software to implement the methodsdiscussed here, called "WhatIf: Software for Evaluating Counterfactuals,"is available at http://gking.harvard.edu/whatif. All informationnecessary to replicate the analyses herein can be found in King and Zeng (2008).Conflict of interest statement. None declared.     

Parties in Elections, Parties in Government, and Partisan Bias

Adam Meirowitz Department of Politics, Princeton University, Princeton, NJ 08544 e-mail: ameirowi{at}princeton.edu Thomas Romer Department of Politics and Woodrow Wilson School of Public and International Affairs, Princeton University, Princeton, NJ 08544 e-mail: romer{at}princeton.edu Political parties are active when citizens choose among candidatesin elections and when winning candidates choose among policyalternatives in government. But the inextricably linked institutions,incentives, and behavior that determine these multistage choicesare substantively complex and analytically unwieldy, particularlyif modeled explicitly and considered in total, from citizenpreferences through government outcomes. To strike a balancebetween complexity and tractability, we modify standard spatialmodels of electoral competition and governmental policy-makingto study how components of partisanship—such as candidateplatform separation in elections, party ID-based voting, nationalpartisan tides, and party-disciplined behavior in the legislature—arerelated to policy outcomes. We define partisan bias as the distancebetween the following two points in a conventional choice space:the ideal point of the median voter in the median legislativedistrict and the policy outcome selected by the elected legislature.The study reveals that none of the party-in-electorate conditionsis capable of producing partisan bias independently. Specifiedcombinations of conditions, however, can significantly increasethe bias and/or the variance of policy outcomes, sometimes insubtle ways.     

Issue Attitudes and Survey Continuity across Interview Mode in the 2000 NES

Nathan J. Kelly Department of Political Science, University at Buffalo, The State University of New York, Buffalo, NY 14260-4120 e-mail: nkelly{at}buffalo.edu H. Whitt Kilburn Department of Political Science, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-3265 e-mail: wkilburn{at}email.unc.edu Can researchers draw consistent inferences about the U.S. public'sissue attitudes when studying survey results from both the in-personand telephone interview modes of the 2000 National ElectionStudies (NES) survey? We address this question through an analysiscontrasting the distribution of issue attitudes across modesin the dual sample design of the 2000 NES. We find clear differencesacross mode even when applying a method devised by the NES toimprove comparability by recoding issue attitude scales fromthe in-person mode. We present an alternative method of recodingthese scales, which substantially improves comparability betweenmodes. Through an analysis of the covariance structure of theissues and simple models of vote choice, we discuss the implicationsof the results for the study of issue attitudes in the 2000NES.     

Event Dependence and Heterogeneity in Duration Models: The Conditional Frailty Model

Suzanna De Boef and Kyle A. Joyce Department of Political Science, 219 Pond Laboratory, The Pennsylvania State University, University Park, PA 16802 e-mail: sdeboef{at}psu.edu e-mail: kjoyce{at}psu.edu e-mail: jboxstef+{at}osu.edu (corresponding author) We introduce the conditional frailty model, an event historymodel that separates and accounts for both event dependenceand heterogeneity in repeated events processes. Event dependenceand heterogeneity create within-subject correlation in eventtimes thereby violating the assumptions of standard event historymodels. Simulations show the advantage of the conditional frailtymodel. Specifically they demonstrate the model's ability todisentangle the sources of within-subject correlation as wellas the gains in both efficiency and bias of the model when comparedto the widely used alternatives, which often produce conflictingconclusions. Two substantive political science problems illustratethe usefulness and interpretation of the model: state policyadoption and terrorist attacks. Authors' note: Three anonymous reviewers gave valuable advice.Replication materials and an online appendix are available onthe Political Analysis Web site. Any errors are our own responsibility.     

Random Coefficient Models for Time-Series-Cross-Section Data: Monte Carlo Experiments

Jonathan N. Katz Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA 91125 e-mail: jkatz{at}caltech.edu e-mail: nathaniel.beck{at}nyu.edu (corresponding author) This article considers random coefficient models (RCMs) fortime-series–cross-section data. These models allow forunit to unit variation in the model parameters. The heart ofthe article compares the finite sample properties of the fullypooled estimator, the unit by unit (unpooled) estimator, andthe (maximum likelihood) RCM estimator. The maximum likelihoodestimator RCM performs well, even where the data were generatedso that the RCM would be problematic. In an appendix, we showthat the most common feasible generalized least squares estimatorof the RCM models is always inferior to the maximum likelihoodestimator, and in smaller samples dramatically so. Authors' note: We gratefully acknowledge the financial supportof the National Science Foundation. Katz also acknowledges thesupport of the Center for Advanced Study in the Behavioral Sciences.We are thankful to Jake Bowers, Rob Franzese, Andy Gelman, SandyGordon, Bill Greene, and Luke Keele for comments; to Larry Bartelsfor always reminding us that our judgment may outperform thedata; as well as to the anonymous reviewers of Political Analysis.     

In Defense of Comparative Statics: Specifying Empirical Tests of Models of Strategic Interaction

Christopher Zorn Department of Political Science, University of South Carolina, Columbia, SC 29208 e-mail: zorn{at}sc.edu e-mail: ccarrub{at}emory.edu (corresponding author) Beginning in 1999, Curtis Signorino challenged the use of traditionallogits and probits analysis for testing discrete-choice, strategicmodels. Signorino argues that the complex parametric relationshipsgenerated by even the simplest strategic models can lead towildly inaccurate inferences if one applies these traditionalapproaches. In their stead, Signorino proposes generating stochasticformal models, from which one can directly derive a maximumlikelihood estimator. We propose a simpler, alternative methodologyfor theoretically and empirically accounting for strategic behavior.In particular, we propose carefully and correctly deriving one'scomparative statics from one's formal model, whether it is stochasticor deterministic does not particularly matter, and using standardlogit or probit estimation techniques to test the predictions.We demonstrate that this approach performs almost identicallyto Signorino's more complex suggestion. Authors' note: We would like to thank Randy Calvert, Mark Hallerberg,Andrew Martin, Eric Reinhardt, Chris Stanton, and Craig Voldenfor their valuable feedback on this project. All remaining errorsare our own. Replication materials are available at the PoliticalAnalysis Web site.     

The Introduction of Voter Registration and Its Effect on Turnout

David M. Konisky Department of Political Science, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Room E53-386, Cambridge, MA 02139 e-mail: sda{at}mit.edu e-mail: konisky{at}mit.edu (corresponding author) Studies of voter turnout across states find that those withmore facilitative registration laws have higher turnout rates.Eliminating registration barriers altogether is estimated toraise voter participation rates by up to 10%. This article presentspanel estimates of the effects of introducing registration thatexploits changes in registration laws and turnout within states.New York and Ohio imposed registration requirements on all oftheir counties in 1965 and 1977, respectively. We find thatthe introduction of registration to counties that did not previouslyrequire registration decreased participation over the long termby three to five percentage points. Though significant, thisis lower than estimates of the effects of registration fromcross-sectional studies and suggests that expectations aboutthe effects of registration reforms on turnout may be overstated.