D.C. needs research for school improvement and audit for oversight, but not from the same source

June 13, 2018
  • Steven Glazerman
  • Chelsea Coffin
  • Yesim Sayin
Featured Image

The D.C. Council is considering an education research collaborative that would carry out priority research on education in D.C. However, its current approach has one major flaw: the Council plans to place this entity under the Office of the D.C. Auditor, where it will also carry out an audit of D.C.’s education data, data management, and data collection. While both audits and research are critical to guide education policy and practice in the District of Columbia, when combined, the research will fail.

D.C. schools have improved over the last ten years of mayoral control by at least one metric:  Families are flocking back to public schools, which gained 10,000 kindergarten to grade 12 students over the last ten years.  However, revelations in the last year about inflated graduation rates, underreported suspensions, and enrollment fraud also exposed weak internal controls at District of Columbia Public Schools (DCPS) and education agencies. As a result, there is great need for more external controls, too—the type of scrutiny the Auditor is already positioned to provide under its current mandate. This office should receive adequate funding to do so and the District of Columbia’s education agencies should receive adequate resources, both financial and technical, to comply with audit requests.

Best practices of research-practice partnerships

An independent research-practice partnership—the commonly used name for research collaboratives—that generates scientific research is also necessary to identify paths for continued improvements. The research-practice partnership needs to focus on information schools need and be completely separate from audits or politics. Successful research-practice partnerships like those in New Orleans, Chicago, and New York, have buy-in from practitioners and trust of the schools and education entities where they conduct research. They collaboratively choose research topics, have an advisory board that focuses on scientific merit, and rely on external funding from foundations or federal sources instead of just the city budget.

A true research-practice partnership can ensure education data integrity, security, and availability for research. Many partnerships are hosted by a research institution or a university with a deep bench of academic researchers and expertise in cleaning, managing, and storing large datasets. For example, in North Carolina, Duke University has housed the state’s education research data center for 20 years and works on behalf of the state agency to make those data available to requestors who meet established guidelines. As a result, hundreds of dissertations, academic papers, and policy briefs have emerged based on data from North Carolina, benefiting the state and the field, and journalists and oversight agencies have had timely access to information as well. Given its core function, research talent is less likely to be recruited by the Auditor than a university or research organization.

What do existing research-practice partnerships look like in D.C. and elsewhere?

The District of Columbia should also integrate lessons learned from previous education research-practice partnerships in the city. For example, since 2011, DCPS has partnered with researchers at University of Virginia and Stanford University to examine the effect of IMPACT and now LEAP, its systems for assessing the performance of school staff and for teacher professional development. DCPS, DC PCSB, and OSSE have also shared data with the Urban Institute to study transportation to school, Mathematica Policy Research to study school choice in D.C., and many academic researchers from out of state to generate important insights in the field. In 2012, a group of researchers formed the D.C. Education Consortium for Research and Evaluation (EdCORE) based at George Washington University as a partnership between independent research firms and university-based faculty, with support from federal government and philanthropy. EdCORE released five reports on D.C.’s 2007 school reform, known as PERAA. The Auditor served as the fiscal agent for EdCORE’s work, which was mandated by the Council. DCPS and OSSE were compelled to provide data to the study and were not partners in the effort. Without strong agency buy-in and consistent financial support, EdCORE became dormant when its commissioned work ended.

Looking at successful research-practice partnerships outside of D.C. in the table below, the proposed research collaborative differs in ways that weaken its independence. It would be the only one to have an oversight and audit role in addition to carrying out research, and the only one where elected officials can directly request studies by policy. It is also unique in that it receives all of its funding from the city instead of grants from federal sources and foundations. Lastly, it doesn’t incorporate a research institution or university as a partner or on its Advisory Board. The proposed collaborative will make some datasets available to the public, which would be a great service. In addition, it should consider a system to allow third party researchers to access data under certain conditions like the Massachusetts Department of Elementary and Secondary Education’s Office of Planning and Research, or the North Carolina Education Research Data Center, which primarily focuses on creating a data warehouse and making it available to requestors who meet established guidelines.

For research to succeed, it must be separate from an audit

Rigorous collaborative research can inform how educators and policymakers improve their practice; independent audits can empower oversight over such decisions—both functions are sorely needed, but best kept separated. If these two functions are combined, schools will be reluctant to participate in research wrapped up as audit and oversight. The research agenda will be shaded towards compliance rather than learning lessons for improving D.C. education outcomes. Unfortunately, the Council’s proposed path will undermine the role of research in examining what works and what positive paths D.C. can build towards providing every student with an excellent public education.

Authors

Steven Glazerman

Chief Research and Methods Officer
Innovations for Poverty Action

Steven Glazerman has expertise in methods for evaluating the impact of social programs and in teacher labor markets, including issues of teacher recruitment, professional development, alternative certification, performance measurement, and compensation. He is an expert on student achievement growth models and value added. His recent research has focused on school choice, especially consumer demand and the role of information in school choice behavior. Glazerman is the director of state and local education partnerships and also directs the Educator Impact Laboratory.

Glazerman’s early research included large-scale national impact evaluations of high-profile programs such as Teach For America and Job Corps. More recently, he was the principal investigator for federally funded national studies of preschool curriculum, teacher induction, and teacher pay. He completed an impact evaluation of the Talent Transfer Initiative, an effort to identify high value-added teachers and attract them with monetary incentives to low-performing schools. He led a five-year randomized study of the impacts of the Teacher Advancement Program in the Chicago Public Schools and was a principal investigator for a national evaluation of the Teacher Incentive Fund, in which more than 130 schools were randomly assigned to offer performance-based pay or serve in a control group. His research on school choice relates to consumer demand, modeling how individual preferences influence aggregate outcomes like enrollment patterns and segregation. He currently leads a factorial experiment to examine the impacts of different ways of presenting school profile information to parents (as in school report cards or school shopping websites).

In addition to his work in the United States, Glazerman has advised government officials and researchers in El Salvador, Guatemala, Honduras, Nicaragua, Peru, Mexico, Ethiopia, and Tanzania through his work with the World Bank, USAID, the Millennium Challenge Corporation, and the Global Development Network. He currently is the principal investigator for a series of multi-country rigorous impact evaluations of education interventions in Latin America and the Caribbean.

Glazerman served on the Brookings Institute Task Force on Teacher Quality and as an adjunct professor of economics at Georgetown University. His work has been published in books and journals, including the Journal of Policy Analysis and Management and Education Finance and Policy, and has been quoted frequently in Education Week, and in the Washington PostBaltimore SunHouston Chronicle, and the Arizona Star. He holds a Ph.D. in public policy from the University of Chicago, Harris School of Public Policy.

Chelsea Coffin

Director of the Education Policy Initiative
D.C. Policy Center

Chelsea Coffin joined the D.C. Policy Center in September 2017 as the Director of the Education Policy Initiative. Her research focuses on how schools connect to broader dynamics in the District of Columbia. She has authored reports on diversity in D.C.’s schools, the D.C. schools with the best improvement for at-risk students, and the transition after high school in D.C. Chelsea has also conducted planning analysis at the D.C. Public Charter School Board, carried out research at the World Bank, and taught secondary school with the Peace Corps in Mozambique.

Chelsea holds a Bachelor of Arts from Middlebury College and a Master of Arts from Johns Hopkins University (SAIS) in International Economics and Development.

You can reach Chelsea at chelsea@dcpolicycenter.org.

Yesim Sayin

Executive Director
D.C. Policy Center

Yesim Sayin is the founding Executive Director of the D.C. Policy Center.

With over twenty years of public policy experience in the District of Columbia, Dr. Sayin is recognized by policymakers, advocates and the media as a source of reliable, balanced analyses on the District’s economy and demography.  Yesim’s research interests include economic and fiscal policy, urban economic development, housing, and education. She is especially focused on how COVID-19 pandemic is changing regional and interregional economic interdependencies and what this means for urban policy. Her work is frequently covered in the media, including the Washington Post, the Washington Business Journal, the New York Times, the Wall Street Journal, WAMU, and the Washington City Paper, among others.

Before joining the D.C. Policy Center, Dr. Sayin worked at the District of Columbia Office of the Chief Financial Officer leading the team that scored the fiscal impact of all legislation the District considered. She frequently testified on high profile legislation and worked closely with the executive and Council staff to ensure that policymakers fully understand the fiscal implications of their proposed legislation. Yesim also has worked in the private sector, and consulted with international organization on a large portfolio of public finance topics.

Yesim holds a Ph.D. in economics from George Mason University in Fairfax, Virginia, and a bachelor’s degree in Political Science and International Relations from Bogazici University, located in Istanbul, Turkey.