Using Preferred Applicant Random Assignment (PARA) to Reduce Randomization Bias in Randomized Trials of Discretionary Programs

Randomization bias occurs when the random assignment used to estimate program effects influences the types of individuals that participate in a program. This paper focuses on a form of randomization bias called “applicant inclusion bias,” which can occur in evaluations of discretionary programs that normally choose which of the eligible applicants to serve. If this nonrandom selection process is replaced by a process that randomly assigns eligible applicants to receive the intervention or not, the types of individuals served by the program—and thus its average impact on program participants—could be affected. To estimate the impact of discretionary programs for the individuals that they normally serve, we propose an experimental design called Preferred Applicant Random Assignment (PARA). Prior to random assignment, program staff would identify their “preferred applicants,” those that they would have chosen to serve. All eligible applicants are randomly assigned, but the probability of assignment to the program is set higher for preferred applicants than for the remaining applicants. This paper demonstrates the feasibility of the method, the cost in terms of increased sample size requirements, and the benefit in terms of improved generalizability to the population normally served by the program.     

Three Strategies to Prevent Unintended Pregnancy

This paper presents results from fiscal impact simulations of three national‐level policies designed to prevent unintended pregnancy: A media campaign encouraging condom use, a pregnancy prevention program for at‐risk youth, and an expansion in Medicaid family planning services. These simulations were performed using FamilyScape, a recently developed agent‐based simulation model of family formation. In some simulation specifications, policies’ benefits are monetized by accounting for projected reductions in government expenditures on medical care for pregnant women and infants. In a majority of these specifications, policies’ fiscal benefit‐cost ratios are less than 1. However, in specifications that account additionally for projected savings to programs that provide a broader range of benefits and services to young children, all three policies have benefit‐cost ratios that are comfortably greater than 1. The results from my preferred specifications suggest that the simulated policies would produce returns to taxpayers on each dollar spent of between $2 to $6. On the whole, the results of these simulations imply that all three policies are sound public investments.     

Does inconvenience explain low take‐up? Evidence from unemployment insurance

Application inconvenience is one popular explanation for why many individuals do not receive the social benefits for which they are eligible. Applications take time and some individuals may decide that the financial benefits do not outweigh these time costs. This paper investigates this explanation using cross‐state variation in administrative changes that made applying for unemployment insurance (UI) benefits substantially more convenient over the past decade. We find that the introduction of phone‐ and Internet‐based claiming did not have an appreciable impact on overall UI take‐up, nor did it lead to a shift toward recipients that are higher income or likely to be receiving the maximum benefit amount. These findings are inconsistent with a time‐ and transaction‐cost explanation for low take‐up, since remote UI claiming is less time intensive. This suggests that reducing application barriers alone may not be an effective tool for increasing program participation. © 2010 by the Association for Public Policy Analysis and Management.     

SSI eligibility and participation among the oldest old: evidence from the AHEAD

The proportion of elderly SSI recipients aged 70 or older has been growing in recent years, perhaps because of rising life expectancies overall and a higher incidence of poverty among the oldest old. In 1999, 84 percent of all elderly SSI recipients were 70 or older. This article examines Supplemental Security Income (SSI) eligibility and participation among the oldest old. The analysis was based on 1993 data from the Study of Assets and Health Dynamics Among the Oldest Old that were used to build a detailed SSI eligibility model to identify individuals who meet the federal criteria for SSI income and resource eligibility. The participation rate among those eligible for federal SSI benefits is 53.9 percent, which is generally consistent with the findings of other studies. Furthermore, eligible participants would receive a significantly higher federal SSI benefit than eligible nonparticipants. Correspondingly, eligible participants have significantly lower incomes and assets than eligible nonparticipants. An econometric model is used to estimate the influence of various demographic, financial, and health care use characteristics on the probability of SSI participation among eligible individuals and couples. The model corrects for measurement error in calculated benefits and for misclassifying someone as ineligible. The empirical results show that the effect of higher SSI benefits on the probability of participation is substantial--a $100 increase in benefits would increase the probability of participating for an average eligible unit by 15 percentage points. Many of the demographic, financial, and health care use variables also are important predictors of SSI participation among the oldest old. The eligibility and participation models are also used to simulate the effect of increasing the SSI unearned income disregard from $20 to $125. Those made eligible by this policy change would receive a very low federal SSI benefit on average, suggesting that they are on the margin of eligibility under the original program rules. The simulated participation rate is 48.8 percent--5 percentage points lower than under the original program rules--reflecting the low benefit that new eligibles would receive. Only 36 percent of those made eligible by the new program rules are predicted to participate. These SSI eligibility and participation models are potentially useful tools for policy analysis. It is fairly straightforward to use these models to change a feature of SSI eligibility, reestimate the group of eligible individuals and couples, and predict participation among those who are eligible under the simulated program rules. New eligibles can be compared with those eligible under original program rules. New participants can be compared with old participants. Although these models focus only on individuals aged 70 or older, this type of analysis can be helpful in estimating the potential distributional effects of proposed SSI policy changes.     

Does state AFDC generosity affect child SSI participation?

This article examines the extent of interactions or spillovers between the Supplemental Security Income (SSI) and Aid to Families with Dependent Children (AFDC) programs for children. In the early 1990s, the Social Security Administration substantially relaxed child eligibility criteria for SSI benefits. Since the changes, the number of U.S. children receiving cash and medical benefits through SSI tripled to nearly 1 million. The article describes a family's decision to participate in SSI and/or AFDC, and uses state‐level data for three years before, and three years after, the Zebley decision to estimate the effect of state program generosity on child program participation. The expansions in child SSI eligibility increased child SSI participation and contributed to increased total program participation by children in the early 1990s. Child SSI participation increased more in states with lower AFDC payments and higher state SSI supplementation payments. These results suggest that families use SSI and AFDC as substitutes. At least 32 percent of the Zebley increase in SSI is likely attributable to the SSI–AFDC benefit gap for the median AFDC benefit state. © 2000 by the Association for Public Policy and Management.     

Can participation be induced? Some evidence from developing countries 1

Influenced by Amartya Sen, over the last decade, The World Bank has allocated nearly US$80 billion to local participatory development projects targeting poverty, improved public service delivery, and strengthened social cohesion and government accountability. But the success of these programs is hindered by both endogenous local factors and flawed program design and implementation. Two especially important local obstacles are (1) entrenched interests of political agents, civil bureaucrats, and non-governmental organizations (NGOs) with either incentives to resist or capabilities to appropriate program resources, and (2) poverty and illiteracy, as the poor and illiterate participate less and benefit less from participatory projects than do the wealthier, more educated, and more connected. After reviewing hundreds of participatory projects, three lessons are clear for program planning. First, contextual factors like inequality, history, geography, and political systems (among others) are important. Second, communities do not necessarily have a ready stock of ‘social capital’ to mobilize. Third, induced participatory interventions work best when supported by a responsive state – donors cannot substitute for a non-functional state, and successful programs combine enlightened state action from above with social mobilization from below. Future participatory development projects would benefit substantially from revised planning and considerably more attention paid to evaluation and monitoring. Project managers have historically paid little attention to context, monitoring, or evaluation, in part because The World Bank’s operational policies did not provide incentives to do so. Donor agencies should also exercise greater patience and allow for flexible, long-term engagement to facilitate contextual and programmatic learning, including learning from failure.     

Who is "62 enough"? Identifying respondents eligible for Social Security early retirement benefits in the Health and Retirement Study

Workers are not instantly eligible for Social Security retirement benefits on their 62nd birthdays, nor can they receive benefits in the month they turn 62. This note discusses how well researchers can do using data from the Health and Retirement Study (HRS) to identify respondents old enough to receive and report early Social Security retirement benefits. It shows that only some workers aged 62 at the time of an HRS interview will be "62 enough" to have received a Social Security benefit and reported it in the survey. In general, workers become eligible for a retirement benefit the month after they turn 62, and they may receive their first payment the month after that. Until recently, payments were received very early in the month, but in mid-1997 and later, the Social Security Administration (SSA) staggered benefit payments over the course of a month. Therefore, many beneficiaries will not be able to report the receipt of their first benefit payment until the third month after their birthday in more recent HRS interviews. This note describes the best approach for approximating the pool of HRS respondents who are old enough to have reported the receipt of their first retirement benefit. It then applies the procedure to an analysis by Burkhauser, Couch, and Phillips, who used the 1994 HRS data to distinguish between those who took early retirement benefits upon turning 62 and those who postponed the receipt of benefits. Because these authors did not provide for respondents who were not "62 enough" to receive a benefit at the time of the interview, they understated the proportion of respondents who took retirement benefits at age 62.     

The Impact of Homeless Prevention on Residential Instability: Evidence From the Homelessness Prevention and Rapid Re-Housing Program

ABSTRACT

Millions of individuals and families in the United States do not have access to stable housing. Recent policies in the United States and the rest of the developed world emphasize programs intended to prevent homelessness through temporary financial assistance. This article explores the impact of the largest homelessness prevention program in U.S. history, the Homelessness Prevention and Rapid Re-housing Program (HPRP), on residential instability, using a national sample of families with children enrolled in school. The identification strategy exploits variations on the location of HPRP providers. Using data on the ratio of K–12 students experiencing homelessness in school districts, we find that HPRP is associated with reductions in the percentage of homeless students for districts closer to an HPRP provider. However, the impacts of HPRP fade out when program benefits end, bringing into question whether homeless prevention can help families achieve self-sufficiency in the long run.     

The devil may be in the details: how the characteristics of SCHIP programs affect take-up.

In this paper, we explore whether the specific design of a state's program has contributed to its success in meeting two objectives of the Children's Health Insurance Program (SCHIP): increasing the health insurance coverage of children in lower income families and doing so with a minimum reduction in their private health insurance coverage (crowd-out). In our analysis, we use two years of Current Population Survey data, 2000 and 2001, matched with detailed data on state programs. We focus on two populations: the eligible population of children, broadly defined--those living in families with incomes below 300 percent of the federal poverty line (FPL)--and a narrower group of children, those who we estimate are eligible for Medicaid or SCHIP. Unique state program characteristics in the analysis include whether the state plan covers families; whether the state uses presumptive eligibility; the number of months without private coverage that are required for eligibility; whether there is an asset test; whether a face-to-face interview is required; and specific outreach activities. Our results provide evidence that state program characteristics are significant determinants of program success.     

Transitions from AFDC to SSI before welfare reform

The Supplemental Security Income (SSI) and Aid to Families with Dependent Children (AFDC) programs serve populations with similar characteristics. SSI serves adults and children with disabilities who are in low-income families, and AFDC serves low-income families with children. Because of that overlap, policy changes in one program can affect the other. In 1996, Congress enacted the Personal Responsibility and Work Opportunity Reconciliation Act, which transformed AFDC into the Temporary Assistance for Needy Families (TANF) program. Many people have expected that implementing that welfare reform legislation would eventually increase SSI participation, for two reasons. First, TANF includes new work requirements and time limits that induce more AFDC/TANF recipients with disabilities to obtain SSI benefits. Second, the change in the funding mechanism--from open-ended funding on a matching basis for AFDC to cash assistance block grants for TANF--gives states a stronger incentive to shift welfare recipients to SSI. This article examines the interaction between the SSI and AFDC programs in the prereform period (1990 to 1996) and discusses the potential implications of welfare reform on that interaction. Using matched data from the Survey of Income and Program Participation and Social Security Administration (SSA) records, our analysis focuses on how the interaction of those programs affects young women (aged 18 to 40) and children (aged 0 to 17). We find a very strong link between AFDC and SSI for young women and children. Significant portions of young female and child SSI beneficiaries in the 1990-1993 period were in AFDC families or had received AFDC in the past. In addition, a substantial share of young women and children who received AFDC during that period eventually entered SSI. Because the SSI program is now serving a much larger population of families with young women and children than in the past, SSA might need to develop policies to better serve that group. The findings also suggest that the prereform period is a poor baseline against which to measure the impact of TANF, primarily because of the instability in programs and policies.     

Incentivizing self‐regulation: Federal vs. state‐level voluntary programs in US climate change policies

How does program sponsorship influence the design of voluntary programs? Why and how do voluntary programs on climate change sponsored by the state and federal governments in the United States vary in their institutional design? Scholars emphasize the signaling role of voluntary programs to outside stakeholders, and the excludable benefits that induce firms to take on non‐trivial costs of joining voluntary programs. Scholars have noted several types of benefits, particularly reputational benefits programs provide, but have not systematically studied why different programs emphasize different types of benefits. We suggest that excludable benefits are likely to take different forms depending on the institutional context in which program sponsors function. We hypothesize that federal programs are likely to emphasize less tangible reputational benefits while state programs are likely to emphasize more tangible benefits, such as access to technical knowledge and capital. Statistical analyses show the odds of a voluntary program emphasizing tangible benefits increases by several folds when the program is sponsored by the state as opposed to federal government.     

Women, marriage, and Social Security benefits

The majority of women Social Security beneficiaries receive at least part of their benefit based on their status as the wives or widows of entitled workers. This article discusses the impact of past and present marital status of women as a factor in establishing eligibility for monthly benefits and the amount of the benefit payment. The data are drawn from the 1980 and 1985 June Marital History Supplements to the Current Population Survey. Whether or not they are currently receiving auxiliary benefits, most older women are potentially eligible for them based on their current marital status or past marital duration.     

Impacts of abstinence education on teen sexual activity, risk of pregnancy, and risk of sexually transmitted diseases.

This paper examines the impacts of four abstinence-only education programs on adolescent sexual activity and risks of pregnancy and sexually transmitted diseases (STDs). Based on an experimental design, the impact analysis uses survey data collected in 2005 and early 2006 from more than 2,000 teens who had been randomly assigned to either a program group that was eligible to participate in one of the four programs or a control group that was not. The findings show no significant impact on teen sexual activity, no differences in rates of unprotected sex, and some impacts on knowledge of STDs and perceived effectiveness of condoms and birth control pills     

STATE-LOCAL RELATIONS AND THE AMERICAN JAIL CRISIS: AN ASSESSMENT OF STATE JAIL MANDATES

Years of neglect, combined with the recent development of a “hands on” judicial policy have resulted in a jail crisis in America. States, recognizing their contribution to this situation, have begun to develop policies designed to assist local jail officials in operating safe, humane, and constitutional facilities. These state efforts are a form o f mandating, and primarily concern minimum operating standards and inspections programs. In this research we investigate three questions related to state mandating of local jail operations: (1) what is the extent of state involvement, (2) what factors are associated with state involvement, and (3) what is the impact of state involvement? Our findings indicate (1) states have been active in developing and administering jail mandates, (2) state involvement is associated with state-level corrections policy, and (3) jail standards and inspections programs have a substantial impact on local jail performance. We conclude with a discussion of how state and local governments may proceed in the process of developing standards, inspections, and enforcement procedures.     

Social security: a program and policy history

Many of the federal and state programs that provide income security to U.S. families have their roots in the Social Security Act (the Act) of 1935. This Act provided for unemployment insurance, old-age insurance, and means-tested welfare programs. The Great Depression was clearly a catalyst for the Social Security Act of 1935, and some of its provisions--notably the means-tested programs--were intended to offer immediate relief to families. However, the old-age insurance program-the precursor to today's Old-Age, Survivors, and Disability Insurance, or Social Security, program-was not designed specifically to deal with the economic crisis of that era. Indeed, monthly benefit payments, under the original Act, were not scheduled to begin until 1942. In addition, from the beginning, the Social Security program has embodied social insurance principles that were widely discussed even before the onset of the Great Depression. The first four decades of the Social Security program were, in general, ones of expansion. In fact, the program was expanded even before it became truly operational. In 1939, amendments added child, spouse, and survivor benefits to the retirement benefits authorized by the 1935 Act. Those amendments also allowed for monthly benefits to begin in 1940. Although the program was not changed substantially during the war years and the initial postwar period, the 1950s were a transformational decade in the program's history: benefit amounts were increased substantially, coverage under the program became close to universal, and a new disability insurance benefit was offered. The 1960s witnessed additional growth in Social Security, but the most important development in social insurance occurred in health insurance, with the creation of the Medicare program in 1965. Legislative actions in the 1970s had profound effects on the Social Security program and, indeed, set the stage for many of today's reform debates. Large benefit increases, a new benefit formula that was erroneously generous, and other changes in the early 1970s created a situation in which annual program costs, as a share of gross domestic product, increased during a 12-year period from about 3 percent to 5 percent. In 1977, amendments to the Act corrected the flawed benefit formula and made other changes in the financing of the system to shore up the program. Thus, the 1970s represent a watershed in the program's history-program growth gave way to increasing concerns about the program's finances. Those concerns were reflected in the amendments to the Act in 1983, which were the last major changes to the program. These amendments, based largely on recommendations from a commission chaired by Alan Greenspan, adjusted benefits and taxes to address pressing near-term financing problems faced by the system. Although the Greenspan Commission focused to a large extent on short-range issues, the resulting reforms have generated large surpluses in the program and the buildup of a substantial trust fund. However, the looming retirement of the baby boomers and several other demographic factors will, according to projections, result in the exhaustion of the trust fund by 2042.     

Workers' compensation: 1980-84 benchmark revisions

The first workers' compensation program was introduced 80 years ago. Its purpose was to compensate occupationally injured workers and their families for lost wages and medical expenses from job-related injury, regardless of fault. Today, each of the State and Federal programs that provides coverage to more than 86 percent of the work force uses a combination of private insurance, State or Federal funds, and self-insurance to meet its benefit obligations. The workers' compensation program is of continuing interest to the Social Security Administration (SSA) for several reasons. Since 1965, Social Security Disability Insurance benefits have been subject to reduction if such benefits, when combined with those provided under workers' compensation laws, exceed 80 percent of the worker's earnings. Because the two programs have gaps in protection as well as duplication in coverage, a periodic review of the workers' compensation program is necessary. In addition, SSA administers Part B of the Black Lung program--established to provide income-maintenance protection to coal miners disabled by pneumoconiosis--to about 1 million beneficiaries whose claims were filed before July 1973. This article provides revised benchmark data on the workers' compensation programs and presents a review of program operations during the early 1980's.     

Why teens do not benefit from work experience programs: Evidence from brother comparisons

Social scientists' research on the consequences of teenage employment suggests that government programs providing teenagers with work experience should have worked better than they have. One explanation is that the government failed to deliver an otherwise effective treatment. This article considers three alternative explanations. The first is that the apparent benefits of working during the teenage years are illusory, reflecting only unmeasured differences between teens who do and do not work. A second explanation is that the groups targeted by government programs, the disadvantaged, benefit less from working as teenagers. A third explanation asserts that program participants and working teens experience different benefits because they have different employment experiences. Using information on brother pairs, this article examines these issues. It determines that conventional analyses of the returns to teenage employment greatly overstate the benefits poor minority teenagers receive from working. Our results suggest that the mixed success of previous programs is not primarily due to poor implementation or government involvement per se. Rather, these programs have had limited success because, for those teenagers targeted, work experience during the teenage years does not raise future earnings.     

The Effect of Safety Net Generosity on Maternal Mental Health and Risky Health Behaviors

Single mothers are more likely to experience mental health problems and stress-related negative health behaviors than their married counterparts, but a more generous safety net may improve these outcomes. We use a simulated safety net eligibility approach that accounts for interactions across safety net programs and relies on changing policies across states and time to identify causal effects of safety net generosity on psychological distress and risky behaviors of single mothers. Results suggest that a more generous safety net is protective of maternal mental health: a $1,000 increase to the simulated potential combined cash and food benefit package reduces severe psychological distress by 8.4 percent. Breaking out effects by individual programs while still controlling for potential benefits from other programs, we find protective effects of tax credits, cash benefits provided by Temporary Assistance for Needy Families, and food benefits provided by Supplemental Nutritional Assistance, but no effects of Medicaid eligibility. The effects are primarily driven by single mothers with the lowest levels of education. We find no significant effects of generosity on daily smoking, but we find evidence that benefits reduce the likelihood of heavy drinking. Results suggest that government investments in resources available to low-income families can be effective at improving well-being.     

Innovation in Welfare Policy: Evaluating State Efforts to Encourage Work among Low‐Income Families1

This article examines what states are doing to help Temporary Assistance to Needy Families (TANF) recipients move into the workforce and become self‐supporting. We first present documentation regarding many different state TANF policies aimed at encouraging work, including benefit structures, time limits, work requirements, sanctions, and work supports. We then discuss how effective these policies are at helping welfare recipients get jobs and increase their income, by reviewing research on each of these elements. We conclude that the TANF program has been effective in increasing employment and decreasing welfare caseloads and expenses, but that this has not resulted in overall income gains for welfare recipients in the long run. We also conclude that the most promising state programs have a strong employment focus combined with focused training and educational opportunities. In addition, we also find that sanctions appear to be harming those most in need of assistance, and therefore that such policies should be reevaluated and refocused.     

Eligibility for the Medicare buy-in programs, based on a survey of income and program participation simulation.

Medicare buy-in programs are designed to reduce out-of-pocket expenses of beneficiaries with modest income and assets. This article provides estimates of the size of the Medicare beneficiary population eligible for the Qualified Medicare Beneficiary (QMB) program, the Specified Low-Income Medicare Beneficiary (SLMB) program, and the Qualified Individual-1 (QI-1) program. The buy-in programs use the same resource limits (twice those used in the Supplemental Security Income (SSI) program) but different thresholds for determining income eligibility. The QMB program uses 100 percent of the poverty line as the cutoff, QI-1 covers persons above 120 percent but at or below 135 percent of the poverty line, and the SLMB program is in between. Making informed judgments about the rate of participation in the buy-in programs and the need for outreach requires an accurate estimate of the size of the eligible population. If that population is underestimated, policymakers might come to unduly optimistic conclusions about current buy-in participation. In contrast, an overestimate may make current participation seem too low. If policymakers react to an upwardly biased estimate of the eligible population by increasing outreach, they are bound to be disappointed by the results of that effort. Estimates of the eligible population from past studies of the QMB and SLMB programs range from 5.1 million to 9.1 million. In the absence of new information, it is difficult to judge the accuracy of those estimates because the methodologies had substantial shortcomings that might bias the results. The most common shortcomings include the lack of high-quality, monthly income data and the lack of information on assets from the same data file that was used to estimate participation and income eligibility for Medicare. The current study uses the most recently available (as of August 2000) Survey of Income and Program Participation (SIPP) file that is matched to the Social Security Administration's (SSA's) administrative records. The data file covers 1995 information. Estimates were also obtained using 1991 data to assess the sensitivity of eligibility estimates to the year chosen. The SIPP has several major advantages over other data sources because it contains relevant, high-quality information on both income and assets for establishing financial eligibility for the buy-in programs. First, the SIPP collects detailed and conceptually appropriate information on monthly, rather than annual, income and therefore has more complete information about income than do other surveys. As a result, SIPP-based estimates of poverty are substantially lower than estimates based on the Current Population Survey. Second, the SIPP also collects information on assets at the individual level. Thus, the survey provides enough detail to measure the major income and asset exclusions directly. Finally, the SIPP data are matched to SSA administrative records: Medicare eligibility can therefore be accurately measured, and self-reported data on Social Security and SSI benefits can be replaced with more accurate monthly information. Our 1995 simulation estimates that approximately 4.8 million persons in the U.S. noninstitutionalized population were eligible for the QMB program and an additional 1.6 million for the SLMB program. The total--roughly 6.5 million--is within the range of estimates from past studies but is closer to the lower end, suggesting that the eligible population is smaller than was previously believed. When the estimated QI-1 eligible population of 0.9 million is added, the total for the three buy-in programs is 7.4 million. Because the QI-1 program did not exist in 1995, only the estimated 6.5 million QMBs and SLMBs would actually have been eligible to receive benefits. The 7.4 million figure represents the 1995 Medicare beneficiaries who would be eligible for buy-in under program rules for 2000. Adjusting that number to account for increases in the Medicare population between 1995 and 1999 yields an estimated eligible population of 7.8 million in 1999. Compared with other elderly Medicare recipients, eligible elderly QMBs and SLMBs have poorer health, more functional limitations, and higher rates of health care use. Thus, not only are their income resources relatively limited, but their need for potentially expensive medical care is also greater. Similar differences were not found in health, functional limitations, and health care use among disabled participants in the QMB and SLMB programs. Our estimates imply that about 2.5 million noninstitutionalized individuals were eligible for but not enrolled in the QMB and SLMB programs in 1999. That finding suggests that fewer eligibles may be available for targeting by outreach efforts than was previously believed. Outreach may be more difficult than it would be with a larger eligible population. (ABSTRACT TRUNCATED)