The Numbers on School Choice

Brian Frederking

January 18, 2006

            Beware arguments that begin “Studies show…”

            In our current political system, it is often more important to maintain ideological purity than to base decisions on evidence. So should we not welcome arguments based on evidence from “studies”?

            Well, sometimes, yes. Here is how to decipher the good stuff: 1) the authors are not affiliated with or funded by ideologically predisposed institutions; 2) the results are published in academic journals after peer review; 3) the evidence is publicly available and others have replicated the findings.

Many “studies” have none of these characteristics; they are paid for by interested parties and used as ammunition to win public policy battles. Truth is not a concern. The privatization of research distorts findings in all areas of science, like pharmaceutical companies paying for studies to test their own drugs (surprise – they work!). A great example is the debate over school vouchers.

Advocates of vouchers rely on market ideology (choice, competition, etc., are good) for their claims. If schools were subject to market pressure, then they would perform better. It is an article of faith. Evidence is secondary. Yet public debates are easier to win if you can begin your argument with “studies show….” So you get your side to fund studies on school choice to bolster your case.

            The vast majority of these studies are ammunition rather than evidence. They meet none of the three characteristics above. The authors are affiliated with or funded by pro-voucher organizations like the Heritage Foundation, the Manhattan Institute, the Olin Foundation, the Bradley Foundation, or the Friedman Foundation. The liberal side plays this game too, with studies sponsored by groups like the People for the American Way or the American Federation of Teachers (surprise - public schools work!).

            Very few studies on school choice end up in peer reviewed journals. For example, General Accounting Office reviews of 36 voucher studies during 2001-2002 included only four that were published in peer reviewed journals. And yet the media report the findings of these studies. By the time scholars at academic institutions dispute their findings (replication and peer review takes time), the damage is done. The ammunition, but not the slow-arriving evidence contradicting the ammunition, ends up in the New York Times.

            The available scholarly evidence on school choice programs is summarized by Kevin Smith (Nebraska) in the June 2005 Perspectives on Politics (a peer reviewed journal):

1) Choice programs have a trivial effect on student achievement. We have little evidence that students in voucher programs perform better than students in public schools.

            2) Choice programs lead to higher parental satisfaction and higher parental involvement.

            3) Choice programs have modest benefits on the quality of the public schools.

4) Choice programs increase the chances of segregation by race, class, and religion.

5) There are no significant differences between different types of choice programs; those that include choice only within public schools perform as well as programs that also include private and religious schools.

            His conclusion: “The scholarly record shows mixed and inconsistent support for the broader claims of choice, especially in its more laissez-faire variants.” He argues that carefully crafted choice programs may make sense in certain contexts where public schools are failing. But the evidence does not support widespread adoption of voucher programs. And the few benefits that come from choice programs can be achieved within the public school system, thus avoiding the constitutional issues of publicly funding religious schools.

            And yet the drumbeat rolls on. Voucher advocates continue to tout choice as a panacea for all the ills of public education. Ideology trumps evidence every time.

            I tell my students that they must generate solid evidence and be willing to let the evidence change their minds. This is what distinguishes political scientists from spin doctors. It is the hallmark of rational, modern, scientific discourse. The premise of this space is that it should also be the hallmark of democratic discourse. Elections are, in this sense, a peer review. But for citizens to adequately evaluate arguments, they need to distinguish evidence from ammunition.