EDA for HLM: Visualization when Probabilistic Inference Fails

e-mail: jwbowers{at}umich.edu (corresponding author) e-mail: kwdrake{at}umich.edu Nearly all hierarchical linear models presented to politicalscience audiences are estimated using maximum likelihood undera repeated sampling interpretation of the results of hypothesistests. Maximum likelihood estimators have excellent asymptoticproperties but less than ideal small sample properties. Multilevelmodels common in political science have relatively large samplesof units like individuals nested within relatively small samplesof units like countries. Often these level-2 samples will beso small as to make inference about level-2 effects uninterpretablein the likelihood framework from which they were estimated.When analysts do not have enough data to make a compelling argumentfor repeated sampling based probabilistic inference, we showhow visualization can be a useful way of allowing scientificprogress to continue despite lack of fit between research designand asymptotic properties of maximum likelihood estimators.