Would a Value‐Added System of Retention Improve the Distribution of Teacher Quality? A Simulation of Alternative Policies

In this paper, we consider several features of teacher‐retention policies based on value‐added measures of effectiveness under a variety of empirically grounded rules and parameters. We consider the effects of policy design by varying the standard above which satisfactory teachers are expected to perform. We simulate recently adopted policies that remove teachers based on consecutive unsatisfactory performance and compare these to policies that remove teachers based on poor performance on average over a multiyear period. We also consider the precision of the performance measure and the underlying variation in teacher quality on policy effects. Finally, the simulation makes a step forward by incorporating recent empirical findings of a relationship between teacher quality and natural attrition from the profession. Our results indicate that deselection policies based on value‐added measures have the potential to improve teacher quality, although understanding the role of policy design, self‐selected exits, and the underlying variation in teacher quality is essential for determining policy effects.     

Teacher Pension Systems,the Composition of the Teaching Workforce,and Teacher Quality

Teacher pension systems concentrate retirements within a narrow range of the career cycle by penalizing individuals who separate too soon or remain employed too long. The penalties result in the retention of some teachers who would otherwise choose to leave, and the premature exit of some teachers who would otherwise choose to stay. We examine the link between teachers’ pension incentives and workforce quality and find no evidence to suggest that the incentives raise quality. Given the large and growing costs associated with maintaining teacher pension systems, and the lack of evidence regarding their efficacy, experimentation by traditional and charter schools with alternative retirement benefit structures would be useful.     

Do school accountability systems make it more difficult for low‐performing schools to attract and retain high‐quality teachers?

Administrative data from North Carolina are used to explore the extent to which that state's relatively sophisticated school‐based accountability system has exacerbated the challenges that schools serving low‐performing students face in retaining and attracting high‐quality teachers. Most clear are the adverse effects on retention rates, and hence on teacher turnover, in such schools. Less clear is the extent to which that higher turnover has translated into a decline in the average qualifications of the teachers in the low‐performing schools. Other states with more primitive accountability systems can expect even greater adverse effects on teacher turnover in low‐performing schools. © 2004 by the Association for Public Policy Analysis and Management.     

Strategic Involuntary Teacher Transfers and Teacher Performance: Examining Equity and Efficiency

Despite claims that school districts need flexibility in teacher assignment to allocate teachers more equitably across schools and improve district performance, the power to involuntarily transfer teachers across schools remains hotly contested. Little research has examined involuntary teacher transfer policies or their effects on schools, teachers, or students. This article uses administrative data from Miami‐Dade County Public Schools to investigate the implementation and effects of the district's involuntary transfer policy, including which schools transferred and received teachers, which teachers were transferred, what kinds of teachers replaced them in their former schools, and how their performance—as measured by their work absences and value‐added in math and reading—compared before and after the transfer. We find that, under the policy, principals in the lowest performing schools identified relatively low‐performing teachers for transfer who, based on observable characteristics, would have been unlikely to leave on their own. Consistent with an equity improvement, we find that involuntarily transferred teachers were systematically moved to higher performing schools and generally were outperformed by the teachers who replaced them. Efficiency impacts are mixed; although transferred teachers had nearly two fewer absences per year in their new positions, transferred teachers continued to have low value‐added in their new schools.     

How do Principals Assign Students to Teachers? Finding Evidence in Administrative Data and the Implications for Value Added

The federal government's Race to the Top competition has promoted the adoption of test‐based value‐added measures (VAMs) of performance as a component of teacher evaluations throughout many states, but the validity of these measures has been controversial among researchers and widely contested by teachers’ unions. A key concern is the extent to which nonrandom sorting of students to teachers may bias the results and lead to a misclassification of teachers as high or low performing. In light of potential for bias, it is important to assess the extent to which evidence of sorting can be found in the large administrative data sets used for VAM estimation. Using a large longitudinal data set from an anonymous state, we find evidence that a nontrivial amount of sorting exists—particularly sorting based on prior test scores—and that the extent of sorting varies considerably across schools, a fact obscured by the types of aggregate sorting indices developed in prior research. We also find that VAM estimation is sensitive to the presence of nonrandom sorting. There is less agreement across estimation approaches regarding a particular teacher's rank in the distribution of estimated effectiveness when schools engage in sorting.     

Negotiating Education Reform: Teacher Evaluations and Incentives in Chile (1990–2010)

Reforms designed to improve the quality of teaching by reforming personnel practices, such as pay for performance arrangements, usually run into opposition from well‐organized teacher unions that can either block reform in the short run or undermine it over the longer term. The experience of a series of reforms that introduced collective and individual pay incentives for teachers in Chile from 1990 to 2010 provide a rare example of ongoing negotiation with the teacher union that resulted in an institutionalized structure of incentive pay for teachers as well as widespread attitudes of sustained support among teachers for performance pay. Chile offers an important example of how sustained change in incentive pay can be achieved through ongoing negotiation.     

Inconvenient Truth? Do Collective Bargaining Agreements Help Explain the Mobility of Teachers within School Districts?

We utilize detailed teacher‐level longitudinal data from Washington State to investigate patterns of teacher mobility in districts with different collective bargaining agreement (CBA) transfer provisions. Specifically, we estimate the log odds that teachers of varying experience and effectiveness levels transfer out of their schools to other schools in the district in Washington kindergarten through 12th grade (K‐12) public schools. We find little consistent evidence relating voluntary transfer provisions in CBAs to patterns in teacher mobility, but do find evidence that patterns in within‐district mobility by teacher experience and effectiveness vary between districts that do not use seniority in involuntary transfer decisions and those that use seniority as a tiebreaker or the only factor in these moves. In models that consider teacher experience, the interaction between teacher experience and school disadvantage in teacher transfer decisions is more extreme in districts with strong involuntary seniority transfer protections; novice teachers are even more likely to stay in disadvantaged schools, and veteran teachers are even more likely to leave disadvantaged schools. On the other hand, models that consider value‐added measures of teacher effectiveness suggest that more effective teachers are less likely to leave disadvantaged schools in districts that do use seniority in involuntary transfer decisions, that is, seniority transfer provisions could actually make the distribution of output‐based measures of quality more equitable. Taken together, these results suggest that seniority transfer provisions may have differential impacts on the distributions of teacher experience and effectiveness.     

Teacher Incentives and Student Achievement: Evidence from an Advancement Program

This paper examines the effects of a comprehensive performance pay program implemented in high‐need schools using administrative data from Louisiana. Exploiting the within‐student variation in the timing of implementation, we find a large and significant effect beginning with the second year of the program on math achievement. Similar but generally insignificant point estimates are observed in social studies. As for English Language Arts and science, there are no effects of the program. We provide evidence against changes in the composition of teacher workforce and postadoption student sorting as potential explanations for our results. We then show aggregate findings from a web‐survey that teachers may have altered their teaching practices following performance‐based compensation. Finally, under certain assumptions, we calculate the net present value of earnings impact to be close to a million dollars per school‐year for math.     

Cream‐Skimming,Parking and Other Intended and Unintended Effects of High‐Powered,Performance‐Based Contracts

As performance‐based contracting in social welfare services continues to expand, concerns about potential unintended effects are also growing. We analyze the incentive effects of high‐powered, performance‐based contracts and their implications for program outcomes using panel data on Dutch cohorts of unemployed and disabled workers that were assigned to private social welfare providers in 2002 to 2005. We employ a difference‐in‐differences design that takes advantage of the fact that contracts gradually moved from partial performance‐contingent pay to full (100 percent) performance‐contingent contracting schemes. We develop explicit measures of selection into the programs and find evidence of cream skimming and other gaming activities on the part of providers, but little impact of these activities on program outcomes. Moving to a system with contract payments fully contingent on performance appears to increase job placements, but not job duration, for more readily employable workers.     

The Effect of the Child Support Performance and Incentive Act of 1998 on Rewarded and Unrewarded Performance Goals

This paper examines the impact of the Child Support Performance and Incentive Act (CSPIA) of 1998 on child support performance measures that are rewarded financially as well as outcomes that are not rewarded. Three of the five performance measures explicitly rewarded by CSPIA are reconstructed in this analysis, as are two child support outcomes that were considered for financial rewards but were ultimately rejected. Using a panel interrupted time series model with state fixed effects and state‐specific trends, this analysis finds that CSPIA had a statistically positive impact on just one rewarded performance goal, cost‐effectiveness, and negatively impacted an unrewarded child support outcome—collections sent to other states. Effect sizes suggest that CSPIA had little impact on child support performance, on balance. These results provide more evidence to the ongoing debate about the ability of performance incentives to improve public sector performance. It also suggests that reforming performance systems in response to perceived problems may create new gaming responses.     

Cost‐Effectiveness of Early Childhood Interventions to Enhance Preschool: Evidence from a Randomized Experiment in Head Start Centers Enrolling Historically Underserved Populations

We evaluate the cost‐effectiveness of two early childhood interventions that use instructional coaching and parent coaching as levers for improvement. The study design allows us to compare the individual effects of each intervention as well as their combined effect on student outcomes. We find that teachers receiving instructional coaching improve their use of evidence‐based instructional practices, while families receiving parent coaching show increases in numerous responsive parenting behaviors associated with positive child outcomes. Both interventions demonstrate positive impacts on students, but only parent coaching shows statistically significant effects across a range of student outcomes. Instructional coaching alone is substantially less costly and may therefore be the most cost‐effective of the three treatment conditions; however, small sample sizes limit our ability to reach definitive conclusions. Policy simulations suggest that implementing these interventions could raise the overall cost‐effectiveness of Head Start by at least 16 percent.     

Paying for grades: Impact of merit‐based financial aid on educational quality

In contrast to education reform efforts that target teachers and schools, merit‐based financial aid for college increases the incentives for high school students and their families to directly affect the quality of education by investing more time and effort in schoolwork. Large‐scale merit‐based aid programs, such as Georgia's HOPE Scholarship, seek to improve education by encouraging students to meet higher standards, in this case by obtaining a 3.0 grade point average in high school and college. Since the HOPE program began in 1993, the number of high school graduates qualifying for the aid has steadily increased to more than 38,000 graduates in the class of 1998, or 59.5 percent of the graduating class. At the same time, the relationship between grades and achievement has remained consistent or, in some cases, improved since HOPE began. In fact, African–American males and females with a 3.1 high school core course grade point average have increased their average Scholastic Assessment Test (SAT) scores by more than 20 points. This indicates that merit‐based aid has improved the quality of K–12 education in Georgia and reduced racial performance disparities by motivating students and their families to commit greater effort to schooling. © 2002 by the Association for Public Policy Analysis and Management.     

Comparing Inference Approaches for RD Designs: A Reexamination of the Effect of Head Start on Child Mortality

The regression discontinuity (RD) design is a popular quasi‐experimental design for causal inference and policy evaluation. The most common inference approaches in RD designs employ “flexible” parametric and nonparametric local polynomial methods, which rely on extrapolation and large‐sample approximations of conditional expectations using observations somewhat near the cutoff that determines treatment assignment. An alternative inference approach employs the idea of local randomization, where the very few units closest to the cutoff are regarded as randomly assigned to treatment and finite‐sample exact inference methods are used. In this paper, we contrast these approaches empirically by re‐analyzing the influential findings of Ludwig and Miller ( 2007 ), who studied the effect of Head Start assistance on child mortality employing parametric RD methods. We first review methods based on approximations of conditional expectations, which are relatively well developed in the literature, and then present new methods based on randomization inference. In particular, we extend the local randomization framework to allow for parametric adjustments of the potential outcomes; our extended framework substantially relaxes strong assumptions in prior literature and better resembles other RD inference methods. We compare all these methods formally, focusing on both estimands and inference properties. In addition, we develop new approaches for randomization‐based sensitivity analysis specifically tailored to RD designs. Applying all these methods to the Head Start data, we find that the original RD treatment effect reported in the literature is quite stable and robust, an empirical finding that enhances the credibility of the original result. All the empirical methods we discuss are readily available in general purpose software in R and Stata; we also provide the dataset and software code needed to replicate all our results.     

International Comparison of Public and Private Employees’ Work Motives,Attitudes, and Perceived Rewards

This article presents a conceptual perspective on the distinctive characteristics of public organizations and their personnel. This perspective leads to hypotheses that public organizations deliver distinctive goods and services that influence the motives and rewards for their employees. These hypotheses are tested with evidence from the International Social Survey Programme in order to compare public and private employees in 30 nations. Public employees in 28 of the 30 nations expressed higher levels of public‐service‐oriented motives. In all of the countries, public employees were more likely to say they receive rewards in the form of perceived social impact. In most of the countries, public employees placed less importance on high income as a reward and expressed higher levels of organizational commitment.

Practitioner Points

• The findings presented here add to previous evidence that public employees seek and attain more altruistic and public‐service‐oriented rewards than private sector employees. In particular, we add evidence that these differences hold in many different nations and cultural contexts.
• Compensation and incentive system reforms in many governments have often concentrated on financial incentives and streamlining procedures for discipline and removal. Such matters are important but should not drive out concerns with showing public employees the impact of their work on the well‐being of others and on the community and society. Leaders and managers should invest in incentive systems that emphasize such motives and rewards.
• Leaders and managers should invest in the use of altruistic and socially beneficial motives and rewards in recruiting systems.

     

Teacher career paths,teacher quality,and persistence in the classroom: Are public schools keeping their best?

In this paper we examine the mobility of early‐career teachers of varying quality, measured using value‐added estimates of teacher performance. Unlike previous studies that have examined these issues, we focus on the variation in these effects across the effectiveness distribution. We find that, on average, more effective teachers tend to stay in their initial schools and in the teaching profession. But there also appears to be heterogeneity in mobility behavior across the performance distribution and evidence that teacher mobility is affected by student demographics and achievement levels. © 2010 by the Association for Public Policy Analysis and Management.     

The effects of NBPTS‐certified teachers on student achievement

In this study, we consider the efficacy of a relatively new and widely accepted certification system for teachers established by the National Board for Professional Teaching Standards (NBPTS). We utilize an extensive database covering the universe of teachers and students in Florida for a four‐year span to determine the relationship between NBPTS certification and the impact of teachers on student test scores from both low‐stakes and high‐stakes exams. Contrary to some previous studies, we find evidence that NBPTS certification provides a positive signal of a teacher's contribution to student achievement only in a few isolated cases. However, our results do reinforce evidence from previous research that the process of becoming NBPTS certified does not increase teacher productivity. © 2009 by the Association for Public Policy Analysis and Management.     

Means,Motive, and Opportunity – Local Government Data Distortion in a High‐Stakes Environment

Regulatory authorities are increasingly relying upon performance data for developing public policy. However, this reliance necessarily assumes that the data are free from material distortion. This paper provides a conceptual framework for understanding the ‘means’, ‘motive’, and ‘opportunity’ for distorting data employed in high‐stakes performance‐management programmes. We present empirical evidence which suggests that the use of data drawn entirely from financial statements by no means guarantees a distortion‐free depiction of performance. In addition, we provide econometric evidence of some important determinants of performance data distortion. Taken as a whole, the following analysis provides a comprehensive picture of the salient matters which must be addressed to ensure accurate data for public policy‐making purposes.     

A supervisor like me: Race,representation, and the satisfaction and turnover decisions of public sector employees

Studies of race representation in public organizations illustrate the importance of bureaucrat race in determining client‐level outcomes. Building “upward” from this research, this study examines how supervisor race impacts outcomes for street‐level bureaucrats using data from a nationally representative sample of public schools. Employing multiple estimation methods, we find that, consistent with the predictions of representation theory, teachers report higher job satisfaction and turn over less often when supervised by an own‐race principal. We also find that race congruence impacts the tangible and intangible organizational benefits teachers receive, and, moreover, that race congruence impacts white and African American employees differently. Most troubling, we find evidence that black teachers earn substantially less in supplemental pay when they work for a white principal, even when compared to white teachers in the same school. © 2011 by the Association for Public Policy Analysis and Management.     

The Use and Efficacy of Capacity‐Building Assistance for Low‐Performing Districts: The Case of California's District Assistance and Intervention Teams

The theory of action upon which high‐stakes accountability policies are based calls for systemic reforms in educational systems that will emerge by pairing incentives for improvement with extensive and targeted technical assistance (TA) to build the capacity of low‐performing schools and districts. To this end, a little discussed and often overlooked aspect of the No Child Left Behind Act (NCLB) mandated that, in addition to sanctions, states were required to provide TA to build the capacity of struggling schools and Local Education Agencies (LEAs, or districts) to help them improve student achievement. Although every state in the country provides some form of TA to its lowest performing districts, we know little about the content of these programs or about their efficacy in improving student performance. In this paper, we use both quantitative and qualitative analyses to explore the actions taken by TA providers in one state—California—and examine whether the TA and support tied to California's NCLB sanctions succeeds in improving student achievement. Like many other states, California requires that districts labeled as persistently failing under NCLB (in Program Improvement year 3, PI3) work with external experts to help them build the capacity to make reforms that will improve student achievement. California's lowest performing PI3 districts are given substantial amounts of funding and are required to contract with state‐approved District Assistance and Intervention Teams (DAITs), whereas the remaining PI3 districts receive less funding and are asked to access less intensive TA from non‐DAIT providers. We use a five‐year panel difference‐in‐difference design to estimate the impacts of DAITs on student performance on the math and English language arts (ELA) standardized tests relative to non‐DAIT TA during the two years of the program intervention. We find that students in districts with DAITs perform significantly better on math California Standards Tests (CSTs) averaged over both treatment years and in each of the first and second years. We do not find evidence that students in districts with DAITs perform higher on ELA CSTs over the combined two years of treatment, although we find suggestive evidence that ELA performance increases in the second year of treatment relative to students in districts with non‐DAIT TA. Ordinary least squares (OLS) regressions that explore the association between specific activities fostered by DAITs and changes in districts’ gains in achievement over the two years of treatment show that DAIT districts that report increasing their focus on using data to guide instruction, shifting district culture to generate and maintain high expectations of students and staff, and increasing within‐district accountability for student performance, have higher math achievement gains over the course of the DAIT treatment. In addition, DAIT districts that increase their focus on ELA instruction and shift district culture to one of high expectations have higher ELA achievement gains than do DAIT districts that do not have a similar focus. © 2012 by the Association for Public Policy Analysis and Management.     

Organizational form and performance: An empirical investigation of nonprofit and for‐profit job‐training service providers

While research on the distinctions between for‐profit and nonprofit organizations and their changing service sector shares is vast, there is comparatively little empirical evidence on the consequences of their differing attributes for social program outcomes. This article presents research on publicly subsidized for‐profit and nonprofit job‐training service providers, namely whether organizational form influences client enrollment, service delivery activities, or performance, as measured in terms of participant outcomes. The findings show that nonprofit providers were not more likely to serve more disadvantaged clients and that neither for‐profit nor nonprofit service providers were consistently more effective in increasing participants' earn‐ings and employment rates. When performance incentives were included in service providers' contracts, contractors of all forms performed significantly better across all participant outcomes examined. © 2000 by the Association for Public Policy and Management.