Informing the Debate: Comparing Boston's Pilot, Charter and Traditional Schools

The Boston Foundation

Researchers from the Harvard Graduate School of Education and MIT have released the results of a groundbreaking study that suggests charter school students in Boston outperform their peers at other public schools in Boston. Results for pilot schools were less clear; some analyses showed positive results at the elementary and high school level, while results for middle school students were less encouraging. The study uses an innovative research design based on school lotteries that allowed for a direct comparison of charter and pilot school students with their peers.

The research team, led by Thomas Kane, faculty director of the Project for Policy Innovation in Education and Professor of Education and Economics at the Harvard Graduate School of Education, and Joshua Angrist, Ford Professor of Economics at MIT, used data provided by the Massachusetts Department of Elementary and Secondary Education. The study is published in a report by The Boston Foundation titled, Informing the Debate: Comparing Boston's Charter, Pilot and Traditional Schools. The Rappaport Institute provided additional support.

"Fifteen years ago, the charter school movement in Massachusetts was launched to see if new models could lead to gains in student achievement," said Kane. "The results of this study suggest that charter schools in Boston are making a significant difference."

Positive effects of charter schools on student achievement were found at both the middle and high school levels and across subjects. The impact on middle school math was particularly dramatic; the effect here amounts to half of a standard deviation, an effect large enough to move a student from the 50th to the 69th percentile in student performance in one year. In fact, the effect of a single year spent in a charter school was equivalent to half of the black-white achievement gap. At the high school level, charter students showed stronger performance scores in English Language Arts, math, writing topic development, and writing competition.

"This report speaks to the promise of education reform - and to its potential impact," said Paul Grogan, president and CEO of the Boston Foundation. "Innovation in schools can help us to close the achievement gap that has clouded a stellar record for success among Boston's schools, which have earned a reputation as among the best large urban systems in the country. The charge forward is to apply the 'best practices' of our charter and pilot schools to serve every family that wants to be a part of this culture of success."

The team tracked the outcomes of students who had participated in the school admission lotteries, comparing those who had been offered a slot to those who had been denied. The school lotteries, which are required under the state's charter law when a school is over capacity, provide a way to answer the common complaint that the charter school applicants are 'different' from their peers in the traditional public schools. "This study is unique because it makes use of unusually rich data on all schools in Massachusetts," said Angrist. "The opportunity to use random assignment - the lotteries - to study and compare two important models for school decentralization means that the students we compared were very similar in terms of family background, motivation, and anything else you might think of except for the likelihood of attending a charter or pilot school."

"At the time of admission, the only difference between applicants who were offered admission and those who were not was a coin flip," said Kane. "The fact that there are large differences in subsequent performance suggests that the charter schools were indeed having an impact. The next step is to identify what's working in charter schools that can be transferred back into the traditional public schools to improve student achievement."


Delivery Area
  • school based
Outcome Area
  • academic performance
  • teacher practice
Participant Area
  • school
Demographic Area
  • urban