When Do Regression‐Adjusted Performance Measures Track Longer‐Term Program Impacts? A Case Study for Job Corps

The use of performance management systems has increased since the Government Performance and Results Act of 1993. While these systems share the goal of trying to improve service delivery and participant outcomes, they do not necessarily provide information on the causal (value‐added) effects of a program, which requires a rigorous impact evaluation. One approach for potentially improving the association between program performance measures and impacts is to adjust performance measures for differences across performance units in participant characteristics and local economic conditions. This article develops a statistical model that describes the conditions under which regression adjustment improves the performance–impact correlation. We then use the model to examine the performance–impact association using extensive data from a large‐scale random assignment evaluation of Job Corps, the nation's largest training program for disadvantaged youths. We find that while regression adjustment changes the Job Corps center performance measures, the adjusted performance measures are not correlated with the impact estimates. The main reasons are the weak associations between the unadjusted Job Corps performance measures and participants’ longer‐term outcomes as measured by the evaluation, as well as the likely presence of unobserved factors across centers that are correlated with outcomes.     

Assessing the role of vocational rehabilitation in disability policy.

Since 1975, vocational rehabilitation has represented a small and declining component of federal disability policy. This trend is perhaps reflective of the relatively crude assessment techniques that have been applied to the program in the past. Using the Virginia Vocational Rehabilitation (VR) program as a prototype, we outline how the data and methods of assessment can be improved for purposes of directing public policy. The key issues include identifying an appropriate comparison group for VR, analysis of longitudinal earnings data, and methods for refining measures of program cost. The analysis provides "fixed-effects" estimates of net earnings impacts for each of three postprogram years stratified by disability classification and gender. These treatment impacts are compared to total and service-specific costs. In general, this analysis suggests that evaluation of VR can be substantially improved and that these improvements can be attained at relatively modest analytic cost.     

Long‐Run Labor Market Effects of the Job Corps Program: Evidence from a Nationally Representative Experiment

Job Corps is the nation's largest and most comprehensive career technical training and education program for at‐risk youth ages 16 to 24. Using the sample from a large‐scale experiment of the program from the mid‐1990s, this article uses tax data through 2015 (20 years later) to examine long‐term labor market impacts. The study finds some long‐term beneficial effects for the older students, with employment gains of 4 percentage points, 40 percent reductions in disability benefit receipt, and 10 percent increases in tax filing rates in 2015. For these students, program benefits exceeded program costs from the social perspective. This study is the first to establish that a national program for disconnected youth can produce long‐term labor market gains, and can be a positive investment made for society. The results suggest that intensive, comprehensive services that focus on developing both cognitive and noncognitive skills are important for improving labor market prospects for this population.     

Do Job Corps performance measures track program impacts?

Since the 1993 Government Performance and Results Act, performance measurement systems based on short‐term program outcomes have been increasingly used to assess the effectiveness of federal programs. This paper examines the association between program performance measures and long‐term program impacts, using nine‐year follow‐up data from a recent large‐scale, national experimental evaluation of Job Corps, the nation's largest federal job training program for disadvantaged youths. Job Corps is an important test case because it uses a comprehensive performance system that is widely emulated. We find that impacts on key outcomes are not associated with measured center performance levels. Participants in higherperforming centers had better outcomes; however, the same pattern holds for comparable controls. Thus, the performance measurement system is not achieving the goal of ranking and rewarding centers on the basis of their ability to improve participant outcomes relative to what these outcomes would have been otherwise. © 2008 by the Association for Public Policy Analysis and Management.     

HELPING PEOPLE WITH LABOR MARKET PROBLEMS: AN OVERVIEW OF KEY POLICIES

Those in need of employment and training programs face a variety of difficulties. Accordingly, the federal response to labor market problems has encompassed a wide range of initiatives. These include preparing people for work (through skills training, the Job Corps, work experience programs); improving the functioning of the labor market (through matching workers with jobs, establishing a minimum wage, and the Targeted Jobs Tax Credit program); and preventing discrimination. Unresolved issues in these areas are examined. They include who should be targeted for federal assistance, which levels of government can best operate programs, how public versus private efforts should be balanced, how limited resources should be distributed, and how the welfare versus work question should be resolved.     

IDENTIFYING MECHANISMS BEHIND POLICY INTERVENTIONS VIA CAUSAL MEDIATION ANALYSIS

Causal analysis in program evaluation has primarily focused on the question about whether or not a program, or package of policies, has an impact on the targeted outcome of interest. However, it is often of scientific and practical importance to also explain why such impacts occur. In this paper, we introduce causal mediation analysis, a statistical framework for analyzing causal mechanisms that has become increasingly popular in social and medical sciences in recent years. The framework enables us to show exactly what assumptions are sufficient for identifying causal mediation effects for the mechanisms of interest, derive a general algorithm for estimating such mechanism‐specific effects, and formulate a sensitivity analysis for the violation of those identification assumptions. We also discuss an extension of the framework to analyze causal mechanisms in the presence of treatment noncompliance, a common problem in randomized evaluation studies. The methods are illustrated via applications to two intervention studies on pre‐school classes and job‐training workshops.     

Housing Voucher Take‐Up and Labor Market Impacts

Low participation rates in government assistance programs are a major policy concern in the United States. This paper studies take‐up of Section 8 housing vouchers, a program in which take‐up rates are quite low among interested and eligible households. We link 18,109 households in Chicago that were offered vouchers through a lottery to administrative data and study how baseline employment, earnings, public assistance, arrests, residential location, and children's academic performance predict take‐up. Our analysis finds mixed evidence of whether the most disadvantaged or distressed households face the largest barriers to program participation. We also study the causal impact of peer behavior on take‐up by exploiting idiosyncratic variation in the timing of voucher offers. We find that the probability of lease‐up increases with the number of neighbors who recently received voucher offers. Finally, we explore the policy implications of increasing housing voucher take‐up by applying reweighting methods to existing causal impact estimates of voucher receipt. This analysis suggests that greater utilization of vouchers may lead to larger reductions in labor market activity. Differences in take‐up rates across settings may be important to consider when assessing the external validity of studies identifying the effects of public assistance programs.     

Exploring the relationship between performance management and program impact: A case study of the job training partnership act

Programs that involve multiple levels of government may suffer from a principal‐agent problem: Lower levels of government may wish to pursue different objectives than the higher level of government that provides funding. One strategy for dealing with this problem is to establish a performance management system where units operating the program are accountable for meeting performance standards and are rewarded or sanctioned depending on how well they perform. Title II‐A of the Job Training Partnership Act provides training for economically disadvantaged adults and has operated under a performance management system since 1983 when the program was established. The federal government's goal for the program is to maximize impact on the employment and earnings of participants, but because control groups are not available for the 640 local programs, proxy measures of performance must be used. In this paper, data from an experiment in 16 sites are used to determine how closely measured performance corresponds to program impact. It is concluded that there is only a weak correspondence between the two measures and that the Department of Labor should avoid making significant rewards or sanctions based on the current performance management system. © 2000 by the Association for Public Policy Analysis and Management.     

Evaluating the benefits and costs of the job corps

Social programs have a wide variety of effects and often have the explicit objective of improving the economic status of the people they serve. In order to be useful to policymakers, benefit—cost analysis of social programs should explicitly take account of these two important program features. The approach used in this analysis of the benefits and costs of the Job Corps does this and provides a useful methodology for evaluating other social programs. According to the analysis, the program has substantial net value for society as a whole as well as for the average Corpsmember.     

Effect of rehabilitation on employment and earnings of the disabled: sociodemographic factors.

This report analyzes the importance of sociodemographic factors in the effect of rehabilitation services on the employment and earnings of disabled persons after their cases were closed by State vocational rehabilitation agencies in fiscal year 1971. The analysis is based on information about personal characteristics and on 1972 employment and earnings data for all such cases in the linked records of the Social Security Administration and the Rehabilitation Services Administration. It identifies sociodemographic factors that facilitate or hamper the effects of rehabilitation as measured primarily by employment differences between clients who completed and failed to complete a program of rehabilitation services. Rehabilitation appears to provide aid, especially to groups frequently disadvantaged in the labor market because of sex, age, ethnicity, or education. Impact was greater for men--but not for women--who were married and had larger families. These results differe significantly from conclusions based on previous studies of the disabled. Earlier studies often concluded that vocational rehabilitation was less successful for women, older persons, ethnic minorities, and persons with low socioeconomic status. Because those studies lacked information on rehabilitation status or focused only on persons who had received rehabilitation services and because they did not compare those who had completed a rehabilitation program with those who had not, they were unable to examine the effects of vocational rehabilitation completely.     

Lifetime earnings patterns, the distribution of future Social Security benefits, and the impact of pension reform

In order to assess the effect of Social Security reform on current and future workers, it is essential to accurately characterize the initial situations of representative workers affected by reform. For the purpose of analyzing typical reforms, the most important characteristic of a worker is the level and pattern of his or her preretirement earnings. Under the current system, pensions are determined largely by the level of the workers' earnings averaged over their work life. However, several reform proposals would create individual retirement accounts for which the pension would depend on the investment accumulation within the account. Thus, the pension would also depend on the timing of the contributions into the account and hence on the exact shape of the worker's lifetime earnings profile. Most analysis of the distributional impact of reform has focused, however, on calculating benefit changes among a handful of hypothetical workers whose relative earnings are constant over their work life. The earnings levels are not necessarily chosen to represent the situations of workers who have typical or truly representative earnings patterns. Consequently, the results of such analysis can be misleading, especially if reform involves introducing a fundamentally new kind of pension formula. This article presents two broad approaches to creating representative earnings profiles for policy evaluation. First, we use standard econometric methods to predict future earnings for a representative sample of workers drawn from the Survey of Income and Program Participation (SIPP). Our statistical estimates are based on a simple representation of typical career earnings paths and a fixed-effect statistical specification. Because our estimation file contains information on each worker's annual earnings from 1951 through 1996 as reported in the Social Security Administration's earnings files, we have a record (though an incomplete one) of the actual earnings that will be used to determine future benefit payments. Our estimates of the earnings function permit us to make highly differentiated predictions of future earnings for each member of our sample. By combining the historical information on individual earnings with our prediction of future earnings up through the normal retirement age, our first approach produces tens of thousands of predicted career earnings paths that can be used in microsimulation policy analysis. Our second approach to creating lifetime earnings profiles is similar in some ways to the traditional method. For example, it is based on the creation of only a handful of "stylized" career earnings patterns. An important difference with the traditional method, however, is that we define the career earnings patterns so that they are truly representative of patterns observed in the workforce. We use simple mathematical formulas to characterize each stylized earnings pattern, and we then produce estimates of the average path of annual earnings for workers whose career earning path falls within each of the stylized patterns we have defined. Finally, we calculate the percentage of workers in successive birth-year cohorts who have earnings profiles that match each of the stylized earnings patterns. Although this method may seem simple, it allows the analyst to create stylized earnings patterns that are widely varied but still representative of earnings patterns observed among sizable groups of U.S. workers. The effects of policy reforms can then be calculated for workers with each of the stylized earnings patterns. Our analysis of U.S. lifetime earnings patterns and of the impact of selected policy reforms produces a number of findings about past trends in earnings, typical earnings patterns in the population, and the potential impact of reform. The analysis focuses on men and women born between 1931 and 1960. Along with earlier analysts, we find that men earn substantially higher lifetime wages than women and typically attain their peak career earnings at a somewhat earlier age. However, the difference in career earnings patterns between men and women has narrowed dramatically over time. Workers with greater educational attainment earn substantially higher wages than those with less education, and they attain their peak career earnings later in life. For example, among men with the least education, peak earnings are often attained around or even before age 40, whereas many men with substantial postsecondary schooling do not reach their peak career earnings until after 50. Our tabulations of the lifetime earnings profiles of the oldest cohorts (born around 1930) and projections of the earnings of the youngest profiles (born around 1960) imply that the inequality of lifetime earnings has increased noticeably over time. Women in the top one-fifth of female earners and men in the top one-fifth of male earners are predicted to receive a growing multiple of the economy-wide average wage during their career. Women born between 1931 and 1935 who were in the top fifth of female earners had lifetime average earnings that were approximately equal to the average economy-wide wage. In contrast, women born after 1951 who were in the top fifth of earners are predicted to earn almost 50 percent more, that is, roughly 150 percent of the economy-wide average wage. Women with a lower rank in the female earnings distribution will also see gains in their lifetime average earnings, but their gains are predicted to be proportionately much smaller than those of women with a high rank in the distribution. Men with high earnings are also predicted to enjoy substantial gains in their relative lifetime earnings, while men with a lower rank in the earnings distribution will probably see a significant erosion in their typical wages relative to the economy-wide average wage. That is mainly the result of a sharp decline in the relative earnings of low-wage men born after 1950. In creating stylized earnings profiles that are representative of those of significant minorities of U.S. workers, we emphasized three critical elements of the earnings path: the average level of earnings over a worker's career, the upward or downward trend in earnings from the worker's 30s through his or her early 60s, and the "sagging" or "hump-shaped" profile of earnings over the worker's career. That classification scheme yields 27 characteristic patterns of lifetime earnings. Surprisingly, the differnce between men and women within each of those categories is quite modest. The main difference between men and women is in the proportions of workers who fall in each category. Only 14 percent of men born between 1931 and 1940 fall in earnings categories with the lowest one-third of lifetime earnings, whereas 53 percent of women born in those years have low-average-earnings profiles. On the other hand, women born in those years are more likely to have a rising trend in lifetime earnings, while men are more likely to have a declining trend. We find that the distribution of lifetime earnings contains relatively more workers with below-average earnings and relatively fewer with very high earnings than assumed in the Social Security Administration's traditional policy analysis. For example, the "low earner" traditionally assumed by the Office of the Chief Actuary is assigned a level of average lifetime earnings that we find to be higher than the average earnings of persons in the bottom one-third of the lifetime earnings distribution. The stylized earnings profiles developed here can be used for policy evaluation, and the results can be compared with those from the more traditional analysis. That comparison produces several notable findings. Because earnings profiles that are actually representative of the population tend to have lower average earnings than assumed in the traditional analysis, workers typically accumulate somewhat less Social Security wealth than implied in the traditional analysis. On the other hand, because the basic benefit formula is tilted in favor of lower-income workers, the internal rate of return on Social Security contributions is somewhat higher than detected in the traditional analysis. Moreover, the primary insurance amount measured as a percentage of the worker's average indexed earnings tends to be higher than implied by the traditional analysis. Finally, the stylized earnings patterns can be used to compare benefit levels enjoyed by workers under the traditional Social Security formula and under an alternative plan based on individual investment accounts. That comparison shows, as expected, that the traditional formula favors low-wage workers and one-earner couples, while an investment account favors single, high-wage workers. Comparing two workers with the same lifetime average earnings, the traditional formula favors workers with rising earnings profiles (that is, with lifetime earnings heavily concentrated at the end of their career), while investment account pensions favor workers with declining earnings profiles (that is, with earnings concentrated early in their career).     

The implications of differences between employer and worker employment/earnings reports for policy evaluation

Differences in administrative (UI) and survey (S) records on employment and earnings have substantial implications for assessing the impact of a variety of public interventions, such as welfare‐to‐work and employment training programs, and especially the state‐oriented welfare reform legislation of 1996. We use data from the 1998 and 1999 waves of the Child Support Demonstration Evaluation (CSDE) Resident Parent Surveys to explore individual differences between survey and UI employment and earnings reports for a Wisconsin sample of current and former welfare recipients. After exploring the potential causes of misreports from both sources, we document the degree of discrepancy between survey and UI earnings and employment measures and assess the difference between the two earnings measures in estimates of simple human capital (earnings) functions. Last, we evaluate the correspondence of the two measures with “hardship” indicators of economic well‐being.     

高职商务英语口语课堂教学质量的思考与改进

社会的发展以及商务英语专业培养方案的需要均迫使我们重视培养学生的英语能力,特别是口语能力。但受传统教育理念、手段、环境等诸多因素的制约,此前高职商务英语口语课程有其局限性,教学质量整体不能得以很好的提高,从而学生的商务英语水平也不能符合职业发展的需要。通过结合自身教学实际,提出改进商务英语口语教学的建议。     

Achieving Economic Self-Sufficiency Through Housing Assistance: An Assessment of a Self-Sufficiency Program of the Housing Authority of Champaign County,Illinois

ABSTRACT

This study examines the early impact of a Local Self-Sufficiency (LSS) program of the Housing Authority of Champaign County (HACC), Illinois, on recipients’ total annual household income and earnings, and employment. In 2013, HACC, through LSS, mandated work requirements for households with working-age, able-bodied adult members and imposed sanctions on those who did not meet the program requirements. We find that, between 2012 and 2014, the LSS program led to an average increase of $2,283 in earnings for an individual household. In aggregate, this allowed HACC to serve an additional 98 (9%) LSS-eligible households for a year. Also, LSS-eligible households experienced an increase in the employment–adult ratio by 11.6 percentage points. The LSS program also had a larger impact for more economically disadvantaged households with no prior work history.     

Contemporary challenges to the German vocational training system

The German vocational training system has played a central role in sustaining the competitive strength of German manufacturing. This article provides an analysis of contemporary developments in this system to assess its likely future trajectory. I begin by underscoring the differences and similarities of the German system to alternative arrangements that have emerged in other countries. I then turn to recent trends in Germany that have caused concern among policy‐makers about the continued strength and viability of the vocational training system. I discuss reforms undertaken in the past few years that point to incremental, though possibly transformative, changes in the system designed to reduce costs and increase flexibility through renegotiations on two fronts: between general training standards and firm needs and training practices, and between the in‐plant and school‐based components of training.     

Vocational Rehabilitation Services and Labor Market Outcomes for Transition-Age Youth with Disabilities in Maine

Using administrative data from Maine to quantify the effect of vocational rehabilitation (VR) services on employment outcomes for transition-age youth with disabilities, we exploited the variation in VR counselors’ propensity to develop Individualized Plans for Employment (IPEs) for VR youth as an instrument for service receipt. We found that among the estimated 36 percent of VR clients on the margin of service receipt, having an IPE implemented increases quarterly employment rate by 15.4 percentage points and quarterly earnings by $1,442 (2018 dollars). We found heterogeneous program effects with individuals 18 or younger benefiting significantly more than those older than age 18 at the time of VR application. We estimate the annual real rate of return to be 10.2 percent for two years after VR case closure.     

The Political Economy of Growth: Democracy and Human Capital

Democracy is more than just another brake or booster for the economy. We argue that there are significant indirect effects of democracy on growth through public health and education. Where economists use life expectancy and education as proxies for human capital, we expect democracy will be an important determinant of the level of public services manifested in these indicators. In addition to whatever direct effect democracy may have on growth, we predict an important indirect effect through public policies that condition the level of human capital in different societies. We conduct statistical investigations into the direct and indirect effects of democracy on growth using a data set consisting of a 30-year panel of 128 countries. We find that democracy has no statistically significant direct effect on growth. Rather, we discover that the effect of democracy is largely indirect through increased life expectancy in poor countries and increased secondary education in non poor countries.     

Teacher Incentives and Student Achievement: Evidence from an Advancement Program

This paper examines the effects of a comprehensive performance pay program implemented in high‐need schools using administrative data from Louisiana. Exploiting the within‐student variation in the timing of implementation, we find a large and significant effect beginning with the second year of the program on math achievement. Similar but generally insignificant point estimates are observed in social studies. As for English Language Arts and science, there are no effects of the program. We provide evidence against changes in the composition of teacher workforce and postadoption student sorting as potential explanations for our results. We then show aggregate findings from a web‐survey that teachers may have altered their teaching practices following performance‐based compensation. Finally, under certain assumptions, we calculate the net present value of earnings impact to be close to a million dollars per school‐year for math.     

Discouraging Disadvantaged Fathers’ Employment: An Unintended Consequence of Policies Designed to Support Families

Substantial declines in employment and earnings among disadvantaged men may be exacerbated by child support enforcement policies that are designed to help support families but may have the unintended consequence of discouraging fathers’ employment. Disentangling causal effects is challenging because high child support debt may be both a cause and a consequence of unemployment and low child support order compliance. We used childbirth costs charged in unmarried mothers’ Medicaid‐covered childbirths, from Wisconsin administrative records, as an exogenous source of variation to identify the impact of debt. We found that greater debt has a substantial negative effect on fathers’ formal employment and child support payments, and that this effect is mediated by fathers’ prebirth earnings histories.     

Employee Empowerment,Employee Attitudes,and Performance: Testing a Causal Model

The last three decades have witnessed the spread of employee empowerment practices throughout the public and private sectors. A growing body of evidence suggests that employee empowerment can be used to improve job satisfaction, organizational commitment, innovativeness, and performance. Nearly all previous empirical studies have analyzed the direct effects of employee empowerment on these outcome variables without taking into account the mediating role of employee attitudes. This article contributes to the growing literature on employee empowerment by proposing and testing a causal model that estimates the direct effect of employee empowerment on performance as well as its indirect effects as mediated by job satisfaction and innovativeness. The empirical analysis relies on three years of data from the Federal Human Capital Survey/Federal Employee Viewpoint Survey and a structural equation modeling approach, including the use of lagged variables. The results support the hypothesized causal structure. Employee empowerment seems to have a direct effect on performance and indirect effects through its influence on job satisfaction and innovativeness, two key causal pathways by which empowerment practices influence behavioral outcomes.