An Empirical Validation Study of Popular Survey Methodologies for Sensitive Questions

When studying sensitive issues, including corruption, prejudice, and sexual behavior, researchers have increasingly relied upon indirect questioning techniques to mitigate such known problems of direct survey questions as underreporting and nonresponse. However, there have been surprisingly few empirical validation studies of these indirect techniques because the information required to verify the resulting estimates is often difficult to access. This article reports findings from the first comprehensive validation study of indirect methods. We estimate whether people voted for an anti‐abortion referendum held during the 2011 Mississippi General Election using direct questioning and three popular indirect methods: list experiment, endorsement experiment, and randomized response. We then validate these estimates against the official election outcome. While direct questioning leads to significant underestimation of sensitive votes against the referendum, indirect survey techniques yield estimates much closer to the actual vote count, with endorsement experiment and randomized response yielding the least bias.     

Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan

List and endorsement experiments are becoming increasingly popular among social scientists as indirect survey techniques for sensitive questions. When studying issues such as racial prejudice and support for militant groups, these survey methodologies may improve the validity of measurements by reducing nonresponse and social desirability biases. We develop a statistical test and multivariate regression models for comparing and combining the results from list and endorsement experiments. We demonstrate that when carefully designed and analyzed, the two survey experiments can produce substantively similar empirical findings. Such agreement is shown to be possible even when these experiments are applied to one of the most challenging research environments: contemporary Afghanistan. We find that both experiments uncover similar patterns of support for the International Security Assistance Force (ISAF) among Pashtun respondents. Our findings suggest that multiple measurement strategies can enhance the credibility of empirical conclusions. Open‐source software is available for implementing the proposed methods.     

When Should We Use Unit Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

Many researchers use unit fixed effects regression models as their default methods for causal inference with longitudinal data. We show that the ability of these models to adjust for unobserved time‐invariant confounders comes at the expense of dynamic causal relationships, which are permitted under an alternative selection‐on‐observables approach. Using the nonparametric directed acyclic graph, we highlight two key causal identification assumptions of unit fixed effects models: Past treatments do not directly influence current outcome, and past outcomes do not affect current treatment. Furthermore, we introduce a new nonparametric matching framework that elucidates how various unit fixed effects models implicitly compare treated and control observations to draw causal inference. By establishing the equivalence between matching and weighted unit fixed effects estimators, this framework enables a diverse set of identification strategies to adjust for unobservables in the absence of dynamic causal relationships between treatment and outcome variables. We illustrate the proposed methodology through its application to the estimation of GATT membership effects on dyadic trade volume.     

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Kosuke Imai Department of Politics, Princeton University, Princeton, NJ 08544 e-mail: kimai{at}princeton.edu Gary King Department of Government, Harvard University, 1737 Cambridge Street, Cambridge, MA 02138 Elizabeth A. Stuart Departments of Mental Health and Biostatistics, Johns Hopkins Bloomberg School of Public Health, 624 North Broadway, Room 804, Baltimore, MD 21205 e-mail: estuart{at}jhsph.edu e-mail: king{at}harvard.edu (corresponding author) Although published works rarely include causal estimates frommore than a few model specifications, authors usually choosethe presented estimates from numerous trial runs readers neversee. Given the often large variation in estimates across choicesof control variables, functional forms, and other modeling assumptions,how can researchers ensure that the few estimates presentedare accurate or representative? How do readers know that publicationsare not merely demonstrations that it is possible to find aspecification that fits the author's favorite hypothesis? Andhow do we evaluate or even define statistical properties likeunbiasedness or mean squared error when no unique model or estimatoreven exists? Matching methods, which offer the promise of causalinference with fewer assumptions, constitute one possible wayforward, but crucial results in this fast-growing methodologicalliterature are often grossly misinterpreted. We explain howto avoid these misinterpretations and propose a unified approachthat makes it possible for researchers to preprocess data withmatching (such as with the easy-to-use software we offer) andthen to apply the best parametric techniques they would haveused anyway. This procedure makes parametric models producemore accurate and considerably less model-dependent causal inferences. Authors' note: Our thanks to Dan Carpenter and Jeff Koch fordata; Alberto Abadie, Neal Beck, Sam Cook, Alexis Diamond, BenHansen, Guido Imbens, Olivia Lau, Gabe Lenz, Paul Rosenbaum,Don Rubin, and Jas Sekhon for many helpful comments; and theNational Institutes of Aging (P01 AG17625-01), the NationalInstitute of Mental Health (MH066247), the National ScienceFoundation (SES-0318275, IIS-9874747, SES-0550873), and thePrinceton University Committee on Research in the Humanitiesand Social Sciences for research support. Software to implementthe methods in this paper is available at http://GKing.Harvard.edu/matchitand a replication data file is available as Ho et al. (2006).     

Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis

Political scientists have long been concerned about the validity of survey measurements. Although many have studied classical measurement error in linear regression models where the error is assumed to arise completely at random, in a number of situations the error may be correlated with the outcome. We analyze the impact of differential measurement error on causal estimation. The proposed nonparametric identification analysis avoids arbitrary modeling decisions and formally characterizes the roles of different assumptions. We show the serious consequences of differential misclassification and offer a new sensitivity analysis that allows researchers to evaluate the robustness of their conclusions. Our methods are motivated by a field experiment on democratic deliberations, in which one set of estimates potentially suffers from differential misclassification. We show that an analysis ignoring differential measurement error may considerably overestimate the causal effects. This finding contrasts with the case of classical measurement error, which always yields attenuation bias.     

Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment

Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments .     

A Case of Sudden Infant Death Due to Incomplete Kawasaki Disease

Although Kawasaki disease (KD) is a self‐limiting disease, it may cause sudden cardiac death. Diagnosis of KD is principally based on clinical signs; however, some infant cases do not meet the criteria. Such cases are identified as incomplete KD. The sudden death risk in incomplete KD cases is similar to conventional KD. In our 5‐month‐old case, he had been admitted to a hospital for a fever and suppuration at the site of Bacille de Calmette et Guerin (BCG) vaccination. However, after discharge from the hospital, his C‐reactive protein (CRP) levels declined, he got indisposed and died suddenly. A medico‐legal autopsy revealed myocarditis, coronaritis, platelet‐aggregated emboli in coronary arteries, and myocardial degeneration, suggesting that the fatal myocardial infarction was due to thrombus emboli in the coronary arteries. Forensic pathologists therefore should pay attention to the cardiac pathology originated from incomplete KD as a potential cause in cases of sudden infant death.     

Japan's ikumen discourse: macro and micro perspectives on modern fatherhood

Ikumen is a buzzword that describes fathers who are actively involved in childrearing. This article focuses on the process in which the term ikumen and its meaning are diffused and investigates how soon-to-be fathers, themselves potential ikumen, view the ikumen discourse. Our endeavor is to grasp the transformation of father roles and the wider family in contemporary Japan through the public and individual engagement with the term ikumen. In this article, we combine macro and micro approaches to analyze ikumen as a discourse circulating in Japanese society and study the way in which the subjects of the discourse – soon-to-be fathers – view the ikumen discourse and interact with it. Utilizing an analysis of newspaper articles, we inquire into the process in which the concept ikumen became popular in Japan and the images that are affiliated with it. Through interviews, we then investigate how soon-to-be fathers as potential ikumen themselves perceive the term. Our results show the way in which an ikumen discourse emerged in Japan in opposition to the term kazoku sabisu, which since the 1970s describes the activities fathers engage in for the satisfaction of their families and fulfillment of social expectations. However, while the term is very popular in the media, the young families we interviewed see ikumen more skeptically. They both resist the popular discourse and adapt it into their construction of paternal identities through a differentiation between a strong versus a weak ikumen image.     

Swedish Diplomacy in the Asia-Pacific Region

The year 2018 marks the 150th anniversary of the establishment of diplomatic relations between Japan and Sweden. It is believed that the first contact between Sweden and Japan came during the Edo Period and was mediated by the Netherlands. The Treaty of Friendship, Commerce and Navigation between Japan and Sweden, signed in 1868, also came into being thanks to the good offices of the Dutch. This treaty, like many others into which Japan entered at that time, was an “unequal treaty.” As Japan increased its presence in East Asia, this unequal treaty with Sweden was revised in 1911. The world was changing. Sweden and Japan shared a common interest in opposing Russia’s ambitions. Russia in particular was a common threat to the diplomatic posture and security of both Japan and Sweden. These two nations, positioned like bookends on either side of Russia, could be important strategic partners for one another and allies in their relationships with Russia. Considering the current issue of North Korea, Sweden also has the potential to act as a mediator because it maintains diplomatic relations with North Korea. There is reason to believe that the diplomatic relationship between Japan and Sweden will develop further.