Connections Matter: How Interactive Peers Affect Students in Online College Courses

Peers affect individual's productivity in the workforce, in education, and in other team‐based tasks. Using large‐scale language data from an online college course, we measure the impacts of peer interactions on student learning outcomes and persistence. In our setting, students are quasi‐randomly assigned to peers, and as such, we are able to overcome selection biases stemming from endogenous peer grouping. We also mitigate reflection bias by utilizing rich student interaction data. We find that females and older students are more likely to engage in student interactions. Students are also more likely to interact with peers of the same gender and with peers from roughly the same geographic region. For students who are relatively less likely to be engaged in online discussion, exposure to more interactive peers increases their probabilities of passing the course, improves their grade in the course, and increases their likelihood of enrolling in the following academic term. This study demonstrates how the use of large‐scale, text‐based data can provide insights into students’ learning processes.     

Marginal Pricing and Student Investment in Higher Education

This paper examines the effect of marginal price on students’ educational investments using rich administrative data on students at Michigan public universities. Marginal price refers to the amount colleges charge for each additional credit taken in a semester. Institutions differ in how they price credits above the full‐time minimum (of 12 credits), with many institutions reducing the marginal price of such credits to zero. We find that a zero marginal price induces a modest share of students (i.e., 7 percent) to attempt up to one additional class (i.e., three credits) but also increases withdrawals and lowers course performance. The analysis generally suggests minimal impacts on credits earned and the likelihood of meeting “on‐time” benchmarks toward college completion, though estimates for these outcomes are less precise and more variable across specifications. Consistent with theory, the effect on attempted credits is largest among students who would otherwise locate at the full‐time minimum, which includes lower‐achieving and socioeconomically disadvantaged students.     

Improving the Implementation and Effectiveness of Out‐of‐School‐Time Tutoring

School districts are spending millions on tutoring outside regular school day hours for economically and academically disadvantaged students in need of extra academic assistance. Under No Child Left Behind (NCLB), parents of children in persistently low‐performing schools were allowed to choose their child's tutoring provider, and together with school districts, they were also primarily responsible for holding providers in the private market accountable for performance. We present results from a multisite, mixed‐method longitudinal study of the impact of out‐of‐school time (OST) tutoring on student reading and mathematics achievement that link provider attributes and policy and program administration variables to tutoring program effectiveness. We find that many students are not getting enough hours of high‐quality, differentiated instruction to produce significant gains in their learning, in part because of high hourly rates charged by providers for tutoring. We identify strategies and policy levers that school districts can use to improve OST tutoring policy design and launch improved programs as waivers from NCLB are granted.     

Evaluating the Effectiveness of Adopting Open Educational Resources in an Introductory American Government Course

In this article, we present findings from a grant-funded initiative to replace traditional, proprietary textbooks with an open content textbook under a Creative Commons license in the introductory American government course (POLS 1101) at Middle Georgia State University. We find that the use of an open content textbook led to somewhat negative effects on student learning outcomes and student course satisfaction, although the associated lower textbook cost increased textbook accessibility to students. We conclude with some suggestions to those adopting textbooks in this course and to the wider discipline regarding measures that may lead to more unequivocally positive outcomes than those experienced in this study.     

THE IMPACTS OF DIFFERENT TYPES OF COLLEGE INSTRUCTORS ON STUDENTS’ ACADEMIC AND LABOR MARKET OUTCOMES

Based on a novel data set that links college administrative information with earnings records from a state college system for both public two‐year and four‐year colleges, this study quantifies the impacts of exposure to different types of instructors during students’ initial semester in college on their subsequent academic and labor market outcomes. To minimize bias from student sorting by type of instructor, we combine course‐set fixed effects with an instrumental variables approach that exploits term‐by‐term fluctuations in faculty composition in each department, therefore controlling for both between‐ and within‐course sorting. The findings suggest that two‐year students, particularly racial minority students, have substantially higher levels of exposure to adjuncts with temporary appointments than four‐year students. Two‐year students taking a heavy course schedule with temporary adjuncts are adversely affected in college persistence and subsequent credit accumulation, and the penalty is particularly pronounced among males and racial minority students with stronger academic potential. Such negative impacts on academic outcomes do not translate into poorer short‐ to medium‐term labor market performance. In the four‐year setting, no significant distinction is identified between different types of instructors on either student academic or labor market outcomes.     

IMPROVING EARLY‐GRADE LITERACY IN EAST AFRICA: EXPERIMENTAL EVIDENCE FROM KENYA AND UGANDA

Primary school enrollments have increased rapidly in sub‐Saharan Africa, spurring concerns about low levels of learning. We analyze field experiments in Kenya and Uganda that assessed whether the Reading to Learn intervention, implemented by the Aga Khan Foundation in both countries, improved early‐grade literacy as measured by common assessments. We find that Ugandan literacy (in Lango) increased by 0.2 standard deviations. We find a smaller effect (0.08) on a Swahili literacy test in Kenya. We find no evidence that differential effects are explained by baseline differences across countries in student test scores, classroom attributes, or implementation fidelity. A plausible explanation that cannot be directly tested is differential effective exposure to the literacy treatment in the tested languages. Students in Kenya were tested in Swahili, which is not necessarily the main language of instruction in primary schools, despite official policy.     

Cost‐Effectiveness of Early Childhood Interventions to Enhance Preschool: Evidence from a Randomized Experiment in Head Start Centers Enrolling Historically Underserved Populations

We evaluate the cost‐effectiveness of two early childhood interventions that use instructional coaching and parent coaching as levers for improvement. The study design allows us to compare the individual effects of each intervention as well as their combined effect on student outcomes. We find that teachers receiving instructional coaching improve their use of evidence‐based instructional practices, while families receiving parent coaching show increases in numerous responsive parenting behaviors associated with positive child outcomes. Both interventions demonstrate positive impacts on students, but only parent coaching shows statistically significant effects across a range of student outcomes. Instructional coaching alone is substantially less costly and may therefore be the most cost‐effective of the three treatment conditions; however, small sample sizes limit our ability to reach definitive conclusions. Policy simulations suggest that implementing these interventions could raise the overall cost‐effectiveness of Head Start by at least 16 percent.     

The Use and Efficacy of Capacity‐Building Assistance for Low‐Performing Districts: The Case of California's District Assistance and Intervention Teams

The theory of action upon which high‐stakes accountability policies are based calls for systemic reforms in educational systems that will emerge by pairing incentives for improvement with extensive and targeted technical assistance (TA) to build the capacity of low‐performing schools and districts. To this end, a little discussed and often overlooked aspect of the No Child Left Behind Act (NCLB) mandated that, in addition to sanctions, states were required to provide TA to build the capacity of struggling schools and Local Education Agencies (LEAs, or districts) to help them improve student achievement. Although every state in the country provides some form of TA to its lowest performing districts, we know little about the content of these programs or about their efficacy in improving student performance. In this paper, we use both quantitative and qualitative analyses to explore the actions taken by TA providers in one state—California—and examine whether the TA and support tied to California's NCLB sanctions succeeds in improving student achievement. Like many other states, California requires that districts labeled as persistently failing under NCLB (in Program Improvement year 3, PI3) work with external experts to help them build the capacity to make reforms that will improve student achievement. California's lowest performing PI3 districts are given substantial amounts of funding and are required to contract with state‐approved District Assistance and Intervention Teams (DAITs), whereas the remaining PI3 districts receive less funding and are asked to access less intensive TA from non‐DAIT providers. We use a five‐year panel difference‐in‐difference design to estimate the impacts of DAITs on student performance on the math and English language arts (ELA) standardized tests relative to non‐DAIT TA during the two years of the program intervention. We find that students in districts with DAITs perform significantly better on math California Standards Tests (CSTs) averaged over both treatment years and in each of the first and second years. We do not find evidence that students in districts with DAITs perform higher on ELA CSTs over the combined two years of treatment, although we find suggestive evidence that ELA performance increases in the second year of treatment relative to students in districts with non‐DAIT TA. Ordinary least squares (OLS) regressions that explore the association between specific activities fostered by DAITs and changes in districts’ gains in achievement over the two years of treatment show that DAIT districts that report increasing their focus on using data to guide instruction, shifting district culture to generate and maintain high expectations of students and staff, and increasing within‐district accountability for student performance, have higher math achievement gains over the course of the DAIT treatment. In addition, DAIT districts that increase their focus on ELA instruction and shift district culture to one of high expectations have higher ELA achievement gains than do DAIT districts that do not have a similar focus. © 2012 by the Association for Public Policy Analysis and Management.     

The effect of attending full‐day kindergarten on English learner students

A significant and growing English learner (EL) population attends public schools in the United States. Evidence suggests they are at a disadvantage when entering school and their achievement lags behind non‐EL students. Some educators have promoted full‐day kindergarten programs as especially helpful for EL students. We take advantage of the large EL population and variation in full‐day kindergarten implementation in the Los Angeles Unified School District to examine the impact of full‐day kindergarten on academic achievement, retention, and English language fluency using difference‐in‐differences models. We do not find signficant effects of full‐day kindergarten on most academic outcomes and English fluency through second grade. However, we find that EL students attending full‐day kindergarten were 5 percentage points less likely to be retained before second grade and there are differential effects for several outcomes by student and school characteristics. © 2011 by the Association for Public Policy Analysis and Management.     

From Birth to School: Early Childhood Initiatives and Third‐Grade Outcomes in North Carolina

This study examines the community‐wide effects of two statewide early childhood policy initiatives in North Carolina. One initiative provides funding to improve the quality of child care services at the county level for all children between the ages of 0 to 5, and the other provides funding for preschool slots for disadvantaged four‐year‐olds. Differences across counties in the timing of the rollout and in the magnitude of the state financial investments per child provide the variation in programs needed to estimate their effects on schooling outcomes in third grade. We find robust positive effects of each program on third‐grade test scores in both reading and math. These effects can best be explained by a combination of direct benefits for participants and spillover benefits for others. Our preferred models suggest that the combined average effects on test scores of investments in both programs at 2009 funding levels are equivalent to two to four months of instruction in grade 3.     

Charter High Schools’ Effects on Long‐Term Attainment and Earnings

Since their inception in 1992, the number of charter schools has grown to more than 6,800 nationally, serving nearly three million students. Various studies have examined charter schools’ impacts on test scores, and a few have begun to examine longer‐term outcomes including graduation and college attendance. This paper is the first to estimate charter schools’ effects on earnings in adulthood, alongside effects on educational attainment. Using data from Florida, we first confirm previous research (Booker et al., 2011 ) that students attending charter high schools are more likely to graduate from high school and enroll in college. We then examine two longer‐term outcomes not previously studied in research on charter schools—college persistence and earnings. We find that students attending charter high schools are more likely to persist in college, and that in their mid‐20s they experience higher earnings.     

Can Financial Aid Help to Address the Growing Need for STEM Education? The Effects of Need‐Based Grants on the Completion of Science,Technology, Engineering,and Math Courses and Degrees

Although workers in science, technology, engineering, and math (STEM) fields earn above‐average wages, the number of college graduates prepared for STEM jobs lags behind employer demand. A key question is how to recruit and retain college students in STEM majors. We offer new evidence on the role of financial aid in supporting STEM attainment. Exploiting a regression discontinuity that allows for causal inference, we find that eligibility for need‐based financial aid increased STEM credit completion by 20 to 35 percent among academically‐ready students in a large, public higher education system. These results appear to be driven by shifting students into STEM‐heavy course loads, suggesting aid availability impacts the academic choices students make after deciding to enroll. We also find suggestive evidence that aid offers increase degree attainment in STEM fields, although we cannot rule out null impacts on STEM degree production.     

Dual-Credit Courses and the Road to College: Experimental Evidence from Tennessee

Dual-credit courses expose high school students to college-level content and provide the opportunity to earn college credits, in part to smooth the transition to college. With the Tennessee Department of Education, we conduct the first randomized controlled trial of the effects of dual-credit math coursework on a range of high school and college outcomes. We find that the dual-credit advanced algebra course alters students’ subsequent high school math course-taking, reducing enrollment in remedial math and boosting enrollment in precalculus and Advanced Placement math courses. We fail to detect an effect of the dual-credit math course on overall rates of college enrollment. However, the course induces some students to choose four-year universities instead of two-year colleges, particularly for those in the middle of the math achievement distribution and those first exposed to the opportunity to take the course in eleventh rather than twelfth grade. We see limited evidence of improvements in early math performance during college.     

The Effect of Providing Breakfast in Class on Student Performance

Many schools have recently experimented with moving breakfast from the cafeteria to the classroom. We examine whether such a program increases achievement, grades, and attendance rates. We exploit quasi‐random timing of program implementation that allows for a difference‐in‐differences identification strategy. We find that providing breakfast in class relative to the cafeteria raises math and reading achievement by 0.09 and 0.06 standard deviations, respectively. These effects are most pronounced for low‐performing, free lunch‐eligible, Hispanic, and low body mass index students. A lack of differences by exposure time and effects on grades suggest that these impacts are on test‐taking performance rather than learning. At the same time, the results highlight the possibility that measured achievement may be biased downwards, and accountability penalties may be inappropriately applied, in schools where many students do not consume breakfast.     

School Starting Age and Cognitive Development

We present evidence of a positive relationship between school starting age and children's cognitive development from ages 6 to 18 using a fuzzy regression discontinuity design and large‐scale population‐level birth and school data from the state of Florida. We estimate effects of being old for grade (being born in September vs. August) that are remarkably stable—always around 0.2 SD difference in test scores—across a wide range of heterogeneous groups, based on maternal education, poverty at birth, race/ethnicity, birth weight, gestational age, and school quality. While the September‐August difference in kindergarten readiness is dramatically different by subgroup, by the time students take their first exams, the heterogeneity in estimated effects on test scores effectively disappears. We do, however, find significant heterogeneity in other outcome measures such as disability status and middle and high school course selections. We also document substantial variation in compensatory behaviors targeted towards young‐for‐grade children. While the more affluent families tend to redshirt their children, young‐for‐grade children from less affluent families are more likely to be retained in grades prior to testing. School district practices regarding retention and redshirting are correlated with improved outcomes for the groups less likely to use those remediation approaches (i.e., retention in the case of more affluent families and redshirting in the case of less affluent families.) Finally, we find that very few school policies or practices mitigate the test score advantage of September‐born children.     

The Effect of R&D Investment on Graduate Student Productivity: Evidence From the Life Sciences

This study examines the role of graduate training and R&D investments on research productivity by focusing on the effect of federal funding for early‐career graduate students. We employ a difference‐in‐differences research design drawing upon a sample of high‐quality life science graduate students who either are award recipients or honorable mentions of the prestigious U.S. National Science Foundation's Graduate Research Fellowship Program. We find that a $91,000 grant over three years has a limited, yet positive impact on the awardee's productivity. These effects are driven by the sample of graduate students without publications prior to applying for the fellowship.     

The Effects of Changing Test‐Based Policies for Reclassifying English Learners

When students labeled English Learners (ELs) are reclassified as Fluent English Proficient, changes often occur in services and settings (e.g., changes in teachers, peers, and ancillary services). Policymakers play an important role in the reclassification process because they establish test‐based criteria that an EL must attain in order to become reclassified. If the criteria established by policymakers are incongruent with the instructional changes that result from reclassification, then services and settings may be denied to students who would otherwise benefit from them. In response to teachers’ and administrators’ concerns that some ELs who were reclassified were not succeeding without additional supports, California policymakers in 2006 to 2007 changed the reclassification criteria. In this paper, we examine the effects of changing these criteria using data on Latino/a students from the Los Angeles Unified School District, the U.S. district serving the largest number of ELs. Using “difference‐in‐regression‐discontinuities” approaches, we find consistent evidence that this policy change, which increased the difficulty of attaining the test‐based criteria for EL reclassification eligibility, had significant effects on high‐school students’ subsequent English language arts achievement (0.18 SDs) and graduation outcomes (11 percentage points). Specifically, when the criteria for reclassification were lower, students experienced negative effects of reclassification; but when the criteria were raised, students no longer experienced these negative effects. Highlighting the complex interplay between assessments, instruction, and policy, these findings demonstrate the important role policymakers play in the academic success of ELs and speak to policymaker considerations when implementing assessment and accountability systems.     

Understanding Organizational Reforms in the Modern State: Specialization and Integration in Norway and France

This article examines the challenge Norway and France face in coordinating specialized government activities after 10 years of comprehensive reforms. The focus is on the tension between territorial and sectoral specialization and between vertical and horizontal specialization. We describe both sector‐specific administrative reforms and more overarching general reforms, looking at similarities and differences in the reorganization choices made by the two countries and also at what drives change. We argue that a combination of factors is required to explain outcomes. These factors include not only home‐grown reforms but also sectoral challenges, diffusion and learning from abroad, adaptation to the financial crisis and budget deficit, and choices made by powerful political executives. Sometimes these factors work together and reinforce each other, producing radical reforms; at other times they have a mutually constraining influence, resulting in only minor changes.