An Empirical Validation Study of Popular Survey Methodologies for Sensitive Questions

When studying sensitive issues, including corruption, prejudice, and sexual behavior, researchers have increasingly relied upon indirect questioning techniques to mitigate such known problems of direct survey questions as underreporting and nonresponse. However, there have been surprisingly few empirical validation studies of these indirect techniques because the information required to verify the resulting estimates is often difficult to access. This article reports findings from the first comprehensive validation study of indirect methods. We estimate whether people voted for an anti‐abortion referendum held during the 2011 Mississippi General Election using direct questioning and three popular indirect methods: list experiment, endorsement experiment, and randomized response. We then validate these estimates against the official election outcome. While direct questioning leads to significant underestimation of sensitive votes against the referendum, indirect survey techniques yield estimates much closer to the actual vote count, with endorsement experiment and randomized response yielding the least bias.     

Comparing and Combining List and Endorsement Experiments: Evidence from Afghanistan

List and endorsement experiments are becoming increasingly popular among social scientists as indirect survey techniques for sensitive questions. When studying issues such as racial prejudice and support for militant groups, these survey methodologies may improve the validity of measurements by reducing nonresponse and social desirability biases. We develop a statistical test and multivariate regression models for comparing and combining the results from list and endorsement experiments. We demonstrate that when carefully designed and analyzed, the two survey experiments can produce substantively similar empirical findings. Such agreement is shown to be possible even when these experiments are applied to one of the most challenging research environments: contemporary Afghanistan. We find that both experiments uncover similar patterns of support for the International Security Assistance Force (ISAF) among Pashtun respondents. Our findings suggest that multiple measurement strategies can enhance the credibility of empirical conclusions. Open‐source software is available for implementing the proposed methods.     

When Should We Use Unit Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

Many researchers use unit fixed effects regression models as their default methods for causal inference with longitudinal data. We show that the ability of these models to adjust for unobserved time‐invariant confounders comes at the expense of dynamic causal relationships, which are permitted under an alternative selection‐on‐observables approach. Using the nonparametric directed acyclic graph, we highlight two key causal identification assumptions of unit fixed effects models: Past treatments do not directly influence current outcome, and past outcomes do not affect current treatment. Furthermore, we introduce a new nonparametric matching framework that elucidates how various unit fixed effects models implicitly compare treated and control observations to draw causal inference. By establishing the equivalence between matching and weighted unit fixed effects estimators, this framework enables a diverse set of identification strategies to adjust for unobservables in the absence of dynamic causal relationships between treatment and outcome variables. We illustrate the proposed methodology through its application to the estimation of GATT membership effects on dyadic trade volume.     

Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference

Kosuke Imai Department of Politics, Princeton University, Princeton, NJ 08544 e-mail: kimai{at}princeton.edu Gary King Department of Government, Harvard University, 1737 Cambridge Street, Cambridge, MA 02138 Elizabeth A. Stuart Departments of Mental Health and Biostatistics, Johns Hopkins Bloomberg School of Public Health, 624 North Broadway, Room 804, Baltimore, MD 21205 e-mail: estuart{at}jhsph.edu e-mail: king{at}harvard.edu (corresponding author) Although published works rarely include causal estimates frommore than a few model specifications, authors usually choosethe presented estimates from numerous trial runs readers neversee. Given the often large variation in estimates across choicesof control variables, functional forms, and other modeling assumptions,how can researchers ensure that the few estimates presentedare accurate or representative? How do readers know that publicationsare not merely demonstrations that it is possible to find aspecification that fits the author's favorite hypothesis? Andhow do we evaluate or even define statistical properties likeunbiasedness or mean squared error when no unique model or estimatoreven exists? Matching methods, which offer the promise of causalinference with fewer assumptions, constitute one possible wayforward, but crucial results in this fast-growing methodologicalliterature are often grossly misinterpreted. We explain howto avoid these misinterpretations and propose a unified approachthat makes it possible for researchers to preprocess data withmatching (such as with the easy-to-use software we offer) andthen to apply the best parametric techniques they would haveused anyway. This procedure makes parametric models producemore accurate and considerably less model-dependent causal inferences. Authors' note: Our thanks to Dan Carpenter and Jeff Koch fordata; Alberto Abadie, Neal Beck, Sam Cook, Alexis Diamond, BenHansen, Guido Imbens, Olivia Lau, Gabe Lenz, Paul Rosenbaum,Don Rubin, and Jas Sekhon for many helpful comments; and theNational Institutes of Aging (P01 AG17625-01), the NationalInstitute of Mental Health (MH066247), the National ScienceFoundation (SES-0318275, IIS-9874747, SES-0550873), and thePrinceton University Committee on Research in the Humanitiesand Social Sciences for research support. Software to implementthe methods in this paper is available at http://GKing.Harvard.edu/matchitand a replication data file is available as Ho et al. (2006).     

Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis

Political scientists have long been concerned about the validity of survey measurements. Although many have studied classical measurement error in linear regression models where the error is assumed to arise completely at random, in a number of situations the error may be correlated with the outcome. We analyze the impact of differential measurement error on causal estimation. The proposed nonparametric identification analysis avoids arbitrary modeling decisions and formally characterizes the roles of different assumptions. We show the serious consequences of differential misclassification and offer a new sensitivity analysis that allows researchers to evaluate the robustness of their conclusions. Our methods are motivated by a field experiment on democratic deliberations, in which one set of estimates potentially suffers from differential misclassification. We show that an analysis ignoring differential measurement error may considerably overestimate the causal effects. This finding contrasts with the case of classical measurement error, which always yields attenuation bias.     

Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment

Randomized experiments are becoming increasingly common in political science. Despite their well-known advantages over observational studies, randomized experiments are not free from complications. In particular, researchers often cannot force subjects to comply with treatment assignment and to provide the requested information. Furthermore, simple randomization of treatments remains the most commonly used method in the discipline even though more efficient procedures are available. Building on the recent statistical literature, we address these methodological issues by offering general recommendations for designing and analyzing randomized experiments to improve the validity and efficiency of causal inference. We also develop a new statistical methodology to explore causal heterogeneity. The proposed methods are applied to a survey experiment conducted during Japan's 2004 Upper House election, where randomly selected voters were encouraged to obtain policy information from political parties' websites. An R package is publicly available for implementing various methods useful for designing and analyzing randomized experiments .     

Japan's ikumen discourse: macro and micro perspectives on modern fatherhood

Ikumen is a buzzword that describes fathers who are actively involved in childrearing. This article focuses on the process in which the term ikumen and its meaning are diffused and investigates how soon-to-be fathers, themselves potential ikumen, view the ikumen discourse. Our endeavor is to grasp the transformation of father roles and the wider family in contemporary Japan through the public and individual engagement with the term ikumen. In this article, we combine macro and micro approaches to analyze ikumen as a discourse circulating in Japanese society and study the way in which the subjects of the discourse – soon-to-be fathers – view the ikumen discourse and interact with it. Utilizing an analysis of newspaper articles, we inquire into the process in which the concept ikumen became popular in Japan and the images that are affiliated with it. Through interviews, we then investigate how soon-to-be fathers as potential ikumen themselves perceive the term. Our results show the way in which an ikumen discourse emerged in Japan in opposition to the term kazoku sabisu, which since the 1970s describes the activities fathers engage in for the satisfaction of their families and fulfillment of social expectations. However, while the term is very popular in the media, the young families we interviewed see ikumen more skeptically. They both resist the popular discourse and adapt it into their construction of paternal identities through a differentiation between a strong versus a weak ikumen image.     

Visualization of latent fingerprints using fluorescence lifetime imaging on paper emitting strong fluorescence

Latent fingerprints were successfully visualized using fluorescence lifetime imaging (FLIM) on paper which emits strong fluorescence with a lifetime close to that of fingerprints and thus from which it is difficult for time-resolved spectroscopy to visualize fingerprints. Latent fingerprint samples on paper were excited using a 450 nm or 532 nm nanosecond pulsed-laser, and time-resolved fluorescence images were obtained at a delay time of 6–16 ns in intervals of 1 ns, to the excitation pulse. The excitation beam was expanded using a lens, and the fluorescence from the fingerprints was captured using an intensified CCD camera. Because of the large fluorescence intensity of the background paper of approximately two to four orders of magnitude larger than that of the fingerprint, the fingerprint was not visualized on each fluorescence image by time-resolved spectroscopy. However, the fingerprint was visualized in a FLIM image constructed using a series of the fluorescence images for the case with the fluorescence intensity of the background paper being four orders of magnitude larger than that of the fingerprint. The difference in fluorescence lifetime in the FLIM image of the visualized fingerprint and background paper was in the order of 0.1 ns, which was an order of magnitude smaller than the inherent fluorescence lifetime of a few nanoseconds for the fingerprints and paper. It was demonstrated that, at a background fluorescence intensity with a certain order of magnitude larger than that of fingerprints, FLIM has the potential to visualize latent fingerprints which cannot be visualized by time-resolved spectroscopy.     

A Statistical Method for Empirical Testing of Competing Theories

Empirical testing of competing theories lies at the heart of social science research. We demonstrate that a well‐known class of statistical models, called finite mixture models, provides an effective way of rival theory testing. In the proposed framework, each observation is assumed to be generated either from a statistical model implied by one of the competing theories or more generally from a weighted combination of multiple statistical models under consideration. Researchers can then estimate the probability that a specific observation is consistent with each rival theory. By modeling this probability with covariates, one can also explore the conditions under which a particular theory applies.We discuss a principled way to identify a list of observations that are statistically significantly consistent with each theory and propose measures of the overall performance of each competing theory. We illustrate the relative advantages of our method over existing methods through empirical and simulation studies.     

Materializing the ‘non-Western’: two stories of Japanese philosophers on culture and politics in the inter-war period

This paper investigates the risk presumably involved in the narratives of non-Western international relations theory (IRT) by focusing on a similar historical case in Japan. It reveals the risk of uncritically accepted geographical division, and particularly focuses on the discourses of the Kyoto School's theory of world history as an example of non-Western narratives in the past, which was to ‘overcome’ the Western civilization similar to the contemporary non-Western IRT. However, they are also infamous for providing justification for the wartime regime in Japan for their aggression in the Asian continent. What is the connection between their philosophy and support for the imperialist regime? If there is a connection between them, is there any possibility of the resurrection of the same results in the case of non-Western IRT? To answer these questions, the article introduces the philosophy of Tosaka Jun who was critical of the School but, unlike Kyoto School philosophers, stubbornly fought against the mainstream politics of the time.