The Use and Efficacy of Capacity‐Building Assistance for Low‐Performing Districts: The Case of California's District Assistance and Intervention Teams

The theory of action upon which high‐stakes accountability policies are based calls for systemic reforms in educational systems that will emerge by pairing incentives for improvement with extensive and targeted technical assistance (TA) to build the capacity of low‐performing schools and districts. To this end, a little discussed and often overlooked aspect of the No Child Left Behind Act (NCLB) mandated that, in addition to sanctions, states were required to provide TA to build the capacity of struggling schools and Local Education Agencies (LEAs, or districts) to help them improve student achievement. Although every state in the country provides some form of TA to its lowest performing districts, we know little about the content of these programs or about their efficacy in improving student performance. In this paper, we use both quantitative and qualitative analyses to explore the actions taken by TA providers in one state—California—and examine whether the TA and support tied to California's NCLB sanctions succeeds in improving student achievement. Like many other states, California requires that districts labeled as persistently failing under NCLB (in Program Improvement year 3, PI3) work with external experts to help them build the capacity to make reforms that will improve student achievement. California's lowest performing PI3 districts are given substantial amounts of funding and are required to contract with state‐approved District Assistance and Intervention Teams (DAITs), whereas the remaining PI3 districts receive less funding and are asked to access less intensive TA from non‐DAIT providers. We use a five‐year panel difference‐in‐difference design to estimate the impacts of DAITs on student performance on the math and English language arts (ELA) standardized tests relative to non‐DAIT TA during the two years of the program intervention. We find that students in districts with DAITs perform significantly better on math California Standards Tests (CSTs) averaged over both treatment years and in each of the first and second years. We do not find evidence that students in districts with DAITs perform higher on ELA CSTs over the combined two years of treatment, although we find suggestive evidence that ELA performance increases in the second year of treatment relative to students in districts with non‐DAIT TA. Ordinary least squares (OLS) regressions that explore the association between specific activities fostered by DAITs and changes in districts’ gains in achievement over the two years of treatment show that DAIT districts that report increasing their focus on using data to guide instruction, shifting district culture to generate and maintain high expectations of students and staff, and increasing within‐district accountability for student performance, have higher math achievement gains over the course of the DAIT treatment. In addition, DAIT districts that increase their focus on ELA instruction and shift district culture to one of high expectations have higher ELA achievement gains than do DAIT districts that do not have a similar focus. © 2012 by the Association for Public Policy Analysis and Management.     

The Effects of Tulsa's Pre‐K Program on Middle School Student Performance

As states have upgraded their commitment to pre‐K education over the past two decades, questions have arisen. Critics argue that program effects are likely to fade out or disappear over time, while supporters contend that program effects are likely to persist under certain conditions. Using data from Tulsa Public Schools, three neighboring school districts, and the state of Oklahoma, and propensity score weighting, we estimate the effects of Tulsa's universal, school‐based pre‐K program on multiple measures of academic progress for middle school students. We find enduring effects on math achievement test scores, enrollment in honors courses, and grade retention for students as a whole, and similar effects for certain subgroups. We conclude that some positive effects of a high‐quality pre‐K program are discernible as late as middle school.     

College Enrollment and Completion Among Nationally Recognized High‐Achieving Hispanic Students

Hispanic high school graduates have lower college completion rates than academically similar white students. As Hispanic students have been theorized to be more constrained in the college search and selection process, one potential policy lever is to increase the set of colleges to which these students apply and attend. In this paper, we investigate the impacts of the College Board's National Hispanic Recognition Program (NHRP), which recognizes the highest‐scoring 11th‐grade Hispanic students on the PSAT/NMSQT, as a mechanism of improving college choice and completion. The program not only informs students about their relative ability, but it also enables colleges to identify, recruit, and offer enrollment incentives. Overall, we find that the program has strong effects on college attendance patterns, shifting students from two‐year to four‐year institutions, as well as to colleges that are out‐of‐state and public flagships, all areas where Hispanic attendance has lagged. NHRP shifts the geographic distribution of where students earn their degree, and increases overall bachelor's completion among Hispanic students who traditionally have had lower rates of success. These results demonstrate that college outreach can have significant impacts on the enrollment choices of Hispanic students and can serve as a policy lever for colleges looking to draw academically talented students.     

The Impact of Homeless Prevention on Residential Instability: Evidence From the Homelessness Prevention and Rapid Re-Housing Program

ABSTRACT

Millions of individuals and families in the United States do not have access to stable housing. Recent policies in the United States and the rest of the developed world emphasize programs intended to prevent homelessness through temporary financial assistance. This article explores the impact of the largest homelessness prevention program in U.S. history, the Homelessness Prevention and Rapid Re-housing Program (HPRP), on residential instability, using a national sample of families with children enrolled in school. The identification strategy exploits variations on the location of HPRP providers. Using data on the ratio of K–12 students experiencing homelessness in school districts, we find that HPRP is associated with reductions in the percentage of homeless students for districts closer to an HPRP provider. However, the impacts of HPRP fade out when program benefits end, bringing into question whether homeless prevention can help families achieve self-sufficiency in the long run.     

The Forgotten Summer: Does the Offer of College Counseling After High School Mitigate Summer Melt Among College‐Intending,Low‐Income High School Graduates?

Despite decades of policy intervention to increase college entry and success among low‐income students, considerable gaps by socioeconomic status remain. To date, policymakers have overlooked the summer after high school as an important time period in students’ transition to college, yet recent research documents high rates of summer attrition from the college pipeline among college‐intending high school graduates, a phenomenon we refer to as “summer melt.” We report on two randomized trials investigating efforts to mitigate summer melt. Offering college‐intending graduates two to three hours of summer support increased enrollment by 3 percentage points overall, and by 8 to 12 percentage points among low‐income students, at a cost of $100 to $200 per student. Further, summer support has lasting impacts on persistence several semesters into college.     

Cream‐Skimming,Parking and Other Intended and Unintended Effects of High‐Powered,Performance‐Based Contracts

As performance‐based contracting in social welfare services continues to expand, concerns about potential unintended effects are also growing. We analyze the incentive effects of high‐powered, performance‐based contracts and their implications for program outcomes using panel data on Dutch cohorts of unemployed and disabled workers that were assigned to private social welfare providers in 2002 to 2005. We employ a difference‐in‐differences design that takes advantage of the fact that contracts gradually moved from partial performance‐contingent pay to full (100 percent) performance‐contingent contracting schemes. We develop explicit measures of selection into the programs and find evidence of cream skimming and other gaming activities on the part of providers, but little impact of these activities on program outcomes. Moving to a system with contract payments fully contingent on performance appears to increase job placements, but not job duration, for more readily employable workers.     

Brown Fades: The End of Court‐Ordered School Desegregation and the Resegregation of American Public Schools

In this paper, we investigate whether the school desegregation produced by court‐ordered desegregation plans persists when school districts are released from court oversight. Over 200 medium‐sized and large districts were released from desegregation court orders from 1991 to 2009. We find that racial school segregation in these districts increased gradually following release from court order, relative to the trends in segregation in districts remaining under court order. These increases are more pronounced in the South, in elementary grades, and in districts where prerelease school segregation levels were low. These results suggest that court‐ordered desegregation plans are effective in reducing racial school segregation, but that their effects fade over time in the absence of continued court oversight.     

Direct certification in the national school lunch program expands access for children

Under the policy of direct certification for free school meals, school districts use information shared by state agencies about household eligibility for means‐tested programs in the state in order to determine the potential eligibility for free meals of students enrolled in the district. This information allows districts to automatically approve students in these eligible households for free meals without requiring the household to complete the application process. This paper examines the impacts of direct certification on students' likelihood of becoming certified for free meals, using data from a national survey of school food service directors as well as statelevel administrative data on program participation. A state‐level fixed effects model is estimated to account for the possibility of selection bias. The key finding is that direct certification leads to a statistically significant increase in the number of children getting free school meals. More generally, this finding highlights a promising approach for improving access to means‐tested programs without compromising program integrity. © 2008 by the Association for Public Policy Analysis and Management.     

Examining the Nature and Magnitude of Intradistrict Resource Disparities in Mid‐Size School Districts

While a growing research base has examined resource disparities across schools within large school districts, the literature has largely overlooked resource allocation in the mid‐size school districts that a large portion of the nation's children attend. In this paper, we measure disparities in teacher resources within four New York State districts and conclude that intradistrict equity is not just a big city problem. We also explore the budgeting processes and mechanisms leading to these disparities, including fund‐based budgeting, grant‐based allocation rules, class size formulae, and ad‐hoc mechanisms. We conclude with policy implications and recommendations for improving intradistrict resource equity.     

College Guidance for All: A Randomized Experiment in Pre‐College Advising

Pre‐college advising programs exist in most disadvantaged high schools throughout the United States. These programs supplement traditional advising by high school guidance counselors and attempt to help underrepresented and disadvantaged students overcome the complexities of the postsecondary admission and financial aid processes. Existing evidence on these programs often uses within‐school randomization where spillovers and alternative supports may confound estimates. We provide the first evidence on a whole school intervention resulting from a school‐level randomized controlled trial in the United States. The college access program we study uses a near‐peer model where a recent college graduate works at the school assisting students in the application and enrollment process. Pooled results across the first three years of program implementation find no significant impacts on overall college enrollment. However, subgroup analyses reveal positive, significant effects among the groups most targeted by the intervention: Hispanic and low‐income students. Most of the impact comes through increasing two‐year college enrollment, but this appears to be new entrants rather than inducing students to move from four‐year to two‐year colleges. The observed positive effects for these subgroups attenuate over time. We attribute this drop in the estimated impact to departures in fidelity of the experiment. Even among the cohorts for which we find positive enrollment impacts, we find no significant impacts on college persistence.     

Walking Away From The Wire: Housing Mobility and Neighborhood Opportunity in Baltimore

Families using the Housing Choice Voucher Program rarely experience large gains in neighborhood or school quality when compared with unassisted poor renters. Research on housing mobility programs has reached mixed conclusions about whether vouchers can improve neighborhood and school quality, especially in the long term. We revisit these findings using new data from the partial remedy to the Thompson v. HUD desegregation case in Baltimore, known as the Baltimore Housing Mobility Program (BHMP). Through targeted vouchers, intensive counseling and innovative policy features, the BHMP helped families move to low-poverty, nonsegregated neighborhoods with higher performing school districts. We examine residential outcomes for the first 1,800 families that moved through the program for a period of up to 9 years. We find that BHMP families moved to more integrated and affluent neighborhoods, in school districts with more qualified teachers and fewer poor students—and most families stayed in these neighborhoods beyond their initial lease-up period. Eventually, a small proportion of families moved to neighborhoods that are less white, but still significantly less poor and less segregated than their original communities. We interpret these findings in light of past mobility programs and discuss policy implications for the Housing Choice Voucher Program.     

Charter High Schools’ Effects on Long‐Term Attainment and Earnings

Since their inception in 1992, the number of charter schools has grown to more than 6,800 nationally, serving nearly three million students. Various studies have examined charter schools’ impacts on test scores, and a few have begun to examine longer‐term outcomes including graduation and college attendance. This paper is the first to estimate charter schools’ effects on earnings in adulthood, alongside effects on educational attainment. Using data from Florida, we first confirm previous research (Booker et al., 2011 ) that students attending charter high schools are more likely to graduate from high school and enroll in college. We then examine two longer‐term outcomes not previously studied in research on charter schools—college persistence and earnings. We find that students attending charter high schools are more likely to persist in college, and that in their mid‐20s they experience higher earnings.     

The Effect of Breakfast in the Classroom on Obesity and Academic Performance: Evidence from New York City

Participation in the federally subsidized school breakfast program often falls well below its lunchtime counterpart. To increase take‐up, many districts have implemented Breakfast in the Classroom (BIC), offering breakfast directly to students at the start of the school day. Beyond increasing participation, advocates claim BIC improves academic performance, attendance, and engagement. Others caution BIC has deleterious effects on child weight. We use the implementation of BIC in New York City (NYC) to estimate its impact on meals program participation, body mass index (BMI), achievement, and attendance. While we find large effects on participation, our findings provide no evidence of hoped‐for gains in academic performance, or of feared increases in obesity. The policy case for BIC will depend upon reductions in hunger and food insecurity for disadvantaged children, or its longer‐term effects.     

Paying for grades: Impact of merit‐based financial aid on educational quality

In contrast to education reform efforts that target teachers and schools, merit‐based financial aid for college increases the incentives for high school students and their families to directly affect the quality of education by investing more time and effort in schoolwork. Large‐scale merit‐based aid programs, such as Georgia's HOPE Scholarship, seek to improve education by encouraging students to meet higher standards, in this case by obtaining a 3.0 grade point average in high school and college. Since the HOPE program began in 1993, the number of high school graduates qualifying for the aid has steadily increased to more than 38,000 graduates in the class of 1998, or 59.5 percent of the graduating class. At the same time, the relationship between grades and achievement has remained consistent or, in some cases, improved since HOPE began. In fact, African–American males and females with a 3.1 high school core course grade point average have increased their average Scholastic Assessment Test (SAT) scores by more than 20 points. This indicates that merit‐based aid has improved the quality of K–12 education in Georgia and reduced racial performance disparities by motivating students and their families to commit greater effort to schooling. © 2002 by the Association for Public Policy Analysis and Management.     

Explaining race,poverty, and gender disparities in advanced course‐taking

We use panel data on Florida high school students to examine race, poverty, and gender disparities in advanced course‐taking. While white students are more likely to take advanced courses than black and Hispanic students, these disparities are eliminated when we condition on observable pre–high school characteristics. In fact, black and Hispanic students are more likely than observably similar white students to take advanced courses. Controlling for students' pre–high school characteristics substantially reduces poverty gaps, modestly reduces Asian–white gaps, and makes little dent in female–male gaps. Black and Hispanic students attend high schools that increase their likelihood of taking advanced courses relative to observably similar white students; this advantage is largely driven by minorities disproportionately attending magnet schools. Finally, recent federal and state efforts aimed at increasing access to advanced courses to poor and minority students appear to have succeeded in raising the share of students who take advanced courses from 2003 to 2006. However, secular trends (or spillovers of the policies to non‐poor, non‐minority students) have spurred faster growth for other students, contributing to widening demographic gaps in these years. © 2009 by the Association for Public Policy Analysis and Management.     

Local Government Responses to Fiscal Stress: Evidence from the Public Education Sector

This article investigates local government responses to fiscal stress through the lens of the K–12 public education sector, examining two major policy options available to school districts for managing fiscal hardship: (1) cutting costs, especially through layoffs, and (2) raising revenues locally through voter referenda. The article employs district‐level administrative and survey data from California and Indiana to examine whether school districts exhibit features of a rational or natural system—in which their behaviors largely reflect fiscal pressures only—or whether they exhibit features of an open system in which nonfinancial factors also shape responses. In Indiana, district fiscal characteristics explain differences in cost‐cutting and revenue‐raising behaviors; there is little empirical evidence that school districts exhibit features of an open system. In California, both fiscal and environmental attributes, including poverty characteristics, average student achievement levels, and the enrollment of English learner students, explain school district behaviors.     

Hispanic College Attendance and the State of Texas GEAR UP Program

This article reports the results of an evaluation of the state of Texas GEAR UP project implemented in six school districts from 1998 to 2005. The intent of the program was to enhance preparation for post‐secondary education among low‐income, Hispanic students. The program was comprised of practical interventions including vertical curriculum alignment, training teachers in advanced placement techniques, encouraging students to take rigorous courses, and increasing outreach about college attendance to students and parents. The central dependent variable is reports by parents of graduating seniors about whether their children would be attending college. A multivariate analysis, controlling for household SES, student grades and attendance, household composition, and language spoken in the home indicates that increased student exposure to the GEAR UP program markedly increases the probability that parents will report that their children are going to attend college.     

School Vouchers and Student Outcomes: Experimental Evidence from Washington,DC

School vouchers are the most contentious form of parental school choice. Vouchers provide government funds that parents can use to send their children to private schools of their choice. Here we examine the empirical question of whether or not a school voucher program in Washington, DC, affected achievement or the rate of high school graduation for participating students. The District of Columbia Opportunity Scholarship Program (OSP) has operated in the nation's capital since 2004, funded by a federal government appropriation. Because the program was oversubscribed in its early years of operation, and vouchers were awarded by lottery, we were able to use the “gold standard” evaluation method of a randomized experiment to determine what impacts the OSP had on student outcomes. Our analysis revealed compelling evidence that the DC voucher program had a positive impact on high school graduation rates, suggestive evidence that the program increased reading achievement, and no evidence that it affected math achievement. We discuss the implications of these findings in light of recent policy developments including the reauthorization of the OSP and the enactment or expansion of more than a dozen school voucher or voucher‐type programs throughout the United States in 2011 and 2012.     

The impact of no Child Left Behind on student achievement

The No Child Left Behind (NCLB) Act compelled states to design school accountability systems based on annual student assessments. The effect of this federal legislation on the distribution of student achievement is a highly controversial but centrally important question. This study presents evidence on whether NCLB has influenced student achievement based on an analysis of state‐level panel data on student test scores from the National Assessment of Educational Progress (NAEP). The impact of NCLB is identified using a comparative interrupted time series analysis that relies on comparisons of the test‐score changes across states that already had school accountability policies in place prior to NCLB and those that did not. Our results indicate that NCLB generated statistically significant increases in the average math performance of fourth graders (effect size 5 0.23 by 2007) as well as improvements at the lower and top percentiles. There is also evidence of improvements in eighth‐grade math achievement, particularly among traditionally low‐achieving groups and at the lower percentiles. However, we find no evidence that NCLB increased fourth‐grade reading achievement. © 2011 by the Association for Public Policy Analysis and Management.     

“Robin Hood” and Texas School District Borrowing Costs

The “Robin Hood” system of school financing in Texas takes property tax funds from wealthy school districts and gives them to poorer districts. This paper examines Permanent School Fund‐insured, school district debt and discovers that under the “Robin Hood” system, Texas school districts with either Aa or A1 underlying credit ratings have higher borrowing costs than districts with lower ratings. Also, the borrowing costs of Texas school districts with underlying credit ratings of Aa and A1 are higher than those for non‐Texas, privately insured school districts with the same ratings, while the borrowing costs of A and Baa‐rated Texas school districts are lower.