Today's episode of Rationally Speaking is sponsored by Give Will Give All Takes, a data driven approach to identifying charities where your donation can make a big impact. Give all spends thousands of hours every year vetting and analyzing nonprofits so that it can produce a list of charity recommendations that are backed by rigorous evidence. The list is free and available to everyone online. The New York Times has referred to give well as quote, the spreadsheet method of giving give. Those recommendations are for donors who are interested in having a high altruistic return on investment in their giving.

Its current recommended charities fight malaria, treat intestinal parasites providing vitamin A supplements and give cash to very poor people. Check them out at Give Weblog.

Welcome to, rationally speaking, the podcast, where we explore the borderlands between reason and nonsense. I'm your host, Julia Gillard, and I'm here with today's guest, Nirav Queensland.

Nirav was the on the founding team of and then later the CEO of New Schools for New Orleans, which essentially rebuilt the public school system in New Orleans after Hurricane Katrina and created the first public school system in the country that was almost entirely charter schools.

So it's now one of the big, maybe highest profile charter school success stories in the United States now. Nor is managing partner of the city fund, which has recently raised two hundred million dollars to work with a select group of cities to create more innovative public school systems around the country.

So today, Nyrup and I are going to be talking about what he's done, what he's learned and the evidence for and maybe against the effectiveness of charter schools. You're welcome to, rationally speaking. Great.

I'm excited to be here. Thanks for having me.

So you were in New Orleans in the aftermath of Hurricane Katrina, essentially rebuilding the public school system. What was that like? Was it chaotic? Did you have to essentially start from scratch?

Yeah, it was extremely chaotic and incredibly tragic. I mean, most of the city was underwater. Over a thousand New Orleans lost their lives. Kids were suffering from high degrees of trauma. But out of that strategy, tragedy, we had a chance to rethink public schooling in the city. And a group of leaders went around the country to try to figure out what to replicate. What city in the United States was knocked out of the park for low income minority kids?

And unfortunately and one of the wealthiest countries in the world, we didn't have one city, which are kind of the hubs of wealth and talent in our country that was doing great things in public schools for low income kids. And so instead of adopting what another city was doing, we kind of took our own path. And that ended, as you noted in the intro, with creating the country's first basically 100 percent charter school system.

I mean, that seems like such a radical step. What was the do you happen to know what the previous record holder in terms of like percentage of charter schools in the U.S. at that time was and probably closer to 40 percent.

And I would say it was what it's radical in hindsight, but was pragmatic at the time. I don't think anybody really at the outset knew we were going to 100 percent charter schools. I would say maybe in year four, year five, after the reforms began looking at the data or the charter schools were dramatically outperforming the traditional system that was being built. A group of us and parents who are choosing these schools wanted to grow what was working. And so I'd say about halfway through the reforms, the vision of, well, maybe this is just a better way to do public education started to form and then we had to build a regulatory system around that.

So what would it mean if a whole public school system was all charter? What does the government do then? How does it oversee those charters? And that was really phase two of the reforms, thinking about how to govern a new type of public school system.

Was it hard to get permission to do this? Like who did you have to get to the green, the you know, the go ahead from?

You know, we had high levels of political support. I think that was for two reasons.

One, the system, it really bottomed out before, you know, there was a story about the FBI renting out a floor in the building where the school district was setting up wiretaps and convicting over twenty people out of five fraud, basically stealing from public school students like teachers or superintendents in the administrative office, dead people on the payroll.

I mean, it was it was really, really bad. And I think it was shockingly bad. And a lot of people understood that. So that created a space, I think, to move quicker. And then second, I think being in a post disaster situation, people are willing to try things that they maybe wouldn't if you were following the inertia of the current system. Yeah, and then we were under state takeover. So the state was overseeing at the time about 80 percent of the schools in the city and the state was supportive.

We also got an infusion of federal funds. So we were able to hold pretty strong political coalition for about seven or eight years straight, which is really important. And then it's worth noting now that the system is back under local control. What does that mean? So I think some people mistake charter schools as a way to avoid democratic oversight of public schools. Yeah, and New Orleans is now married, though. So there's a local school board, locally elected, but the locally elected political entity, instead of running schools, now oversees them.

So all the schools in New Orleans are governed by nonprofit oriented. Nations and the elected government institution as a regulator like the FCC or FCC, things like that, so they don't the difference then would be they don't like to which teachers to hire or choose, you know, which curriculum was to implement or things like that. They just sort of monitor for, I don't know, fraud or like whether they're meeting the standards of the state or things like that.

Exactly. They have you know, the government really has too many rules in New Orleans. One is to oversee the system for performance. So to let the schools that are doing the best expand and to selectively replace the schools that aren't meeting kids needs. And the second is around equity. So are public schools serving kids with special needs or any schools expelling too many kids? There's a lot of issues where you do need some government oversight to make sure all the schools are playing by the same rules.

Can you clarify the difference between charter schools and school vouchers? Yeah, I was confused about that for a long time.

Sure. So the easiest way to think about charter schools is they're basically public schools that are governed by nonprofit boards. For the most part, about 80 percent of them are non-profit rather than by the locally elected school board. So most cities have traditional government monopolies that run the public school system. Charters are a way to allow non-profits to operate within the public confines, but with the different governance level. And they think and I can go in deeper into this, that's probably the most important thing about charter schools, is the nonprofit governance has really been a breakthrough in terms of allowing for greater efficacy.

Vouchers are different and that they're outside the public system. So you're getting public funding as a family that's using a voucher, but you can attend, let's say, a Catholic private school so that Catholic school can be religious affiliated. It might not need to take the state test. It's really outside the confines of what you think of as traditional public schooling. And a voucher system allows families to use public funds to kind of exit the system and access private schools.

I see. But you're still I mean, in the charter school model, you're still getting to choose which of the charter schools you want to send your kids to. Exactly.

So you have a choice. It's just a little more highly regulated. Charter schools have to take the state tests the same way the traditional public schools do. They can't pick and choose which kids they're going to serve, which sometimes. And private schools. You can do that. So you're operationally decentralized with the charter school model. The charter school can pick the curriculum, they can pick the teachers, they can pick the length of the school day. They can pick their calendar.

But they're still plugged in to the overall regulatory apparatus of the government with state standards and state testing and so forth.

In practice, how do charter schools end up being different from the standard public schools like along? What dimensions do they vary? Are they teaching in different styles, like more of a Montessori thing, or are they trying smaller classrooms or what are they doing differently?

Yeah, this has been a big change for me. So I think the people who started the charter school movement and I think myself when I got into this thought that the greatest I have charters would be innovative practice. Yeah. And in a certain sense, I think the true innovation of charter schools is in the governance model. So a charter school that's governed by a nonprofit can have strategic consistency over decades. So some of the best charters that have gotten to scale like the KIPP schools are now have been in operation over twenty years.

They've iterated on the model, but they've held the same core beliefs and then they've grown. So KIPP serves almost 100000 kids across the country. They're not bound by geography and they're not bound by the turmoil, which sometimes happens through elected boards. And so with the nonprofit governance, you can really scale an effective organization within the traditional model. You know, the school boards elected every four years, the new politicians come in and they say we're going to change things.

They hire a superintendent. That superintendent says, you know, my way is the highway and then all the teachers roll their eyes and here we go again. And you just constantly reinventing the wheel. And it's very hard to do anything consistently over time. So governance is an extremely important and perhaps the most important innovation of charter schools. The second thing that's really important is it allows for entrepreneurship. So as we've seen in so many other sectors, like a rogue person with a great idea of starting something new, is often how the world's changed.

And that's just really hard to do. And a monopolistic bureaucracy and a charter school allows an educator who has an amazing idea to open up a school that families think and choose. So I'll get to your question in a sec. But I do think the major innovation has been around governance and entrepreneurship. Now, once you unleash that upon the world, you just see a wide variety of models. So there's charter schools that are very progressive, Montessori oriented students follow their own learning path.

And then you have charter schools that are military schools. You know, the. A very strict and regimented, so I think we've seen a wide breadth of how schools operate under the more flexible regulatory regime.

Does that mean that we then also have data telling us which of these No. Thousand blooming flowers are an improvement over the norm?

We do know I'll stick to what's really working with low income kids who are at risk. That's our line of work. Roland Fryer, econ professor at Harvard, looked at charter schools in New York and tried to figure out what the commonalities were of those that were increasing test scores the most. And he found five core components. One was a culture of high expectations. So things like every kid is expected to go to college, every kid's expected to get good grades and really pushing kids hard on academic to was rapid cycles of teacher feedback.

So instead of your evaluator coming in once a year and saying good or bad principals in their every day or every week, giving you very iterative feedback so you can grow as a professional three with the use of data. So very frequent assessments daily and weekly. So you're constantly checking in on kids are at rather than waiting to the end of the year. Fourth is tutoring either in a one on one or in small groups pulling kids out or further behind and giving them very close one on one attention.

And fifth is extended time, longer school day, longer school year. So when he looked at schools in New York, that really knocked me out of the park on test scores. Those were the five practices. And I would say around the country, we see some of the higher performers sticking to some version of that model. I think what we don't know yet is how much those higher test scores lead to longer term life outcomes. So this is a really important part of education.

Research is the easiest thing to figure out is test scores. But the connection between test scores and life outcomes is pretty murky. And we are still learning about what that means for high school graduation, college graduation and ultimately wages and income growth.

Going back to the impressive results of implementing charter schools, New Orleans. Well, first off, can you just summarize the empirical what we mean when we say there were impressive results? Yeah.

So Doug Harris, a researcher out of Tulane, studied the reforms and he found roughly a half standard deviation effect size on test scores, which is very, very large in the education literature. I think the quote from his report was he'd never seen a city improve this much in such a short time to put that quite apples to apples. But to give some layman's context, the black white achievement gap in America is about 098 standard deviations. So making a point five standard deviation jump is a pretty big jump for a whole city to move in about a seven year period.

Yeah, that's helpful. We're just starting to get data back on high school graduation, college enrollment, college completion. That has been modestly positive to date, but we're still in the early cohort's. And so my hope is over the next five to 10 years will get a lot more positive data on longer term outcomes. I'm curious how much you trust that evidence or how much weight you put on it like. I mean, rebuilding a school system after a hurricane is kind of a weird situation, like, first of all, there's the problem of like before after comparison isn't, you know, a randomized control trial.

But then, like specifically in the case of New Orleans after Hurricane Katrina, it's just unusual in so many ways. And I could sort of imagine or generate off top of my head various possible reasons we might see this huge improvement that don't stem from the charter school element specifically, like maybe the just like rebuilding schools. You know, having new schools instead of old schools was good. Or maybe you thought there was an influx of federal funding. I don't know how much that change the spending per student, but that could have something to do with it.

Or, you know, maybe the survivors had a new lease on life and that, you know, they resolved to buckle down and commit to education or I don't know, I could make up a lot more stories. But I know you're a pretty careful thinker when it comes to to evaluating evidence. I'm just I'm just curious, you know, how you think about that with the case of Orleans in particular, if two quick response on that.

One is we've thought a lot about how much should we trust the evidence. And so a lot of controls were put in around a lot of the things you talked about, who are the returnee's? How do we compare them with other kids across Louisiana? Funding did go up by, I think, a little over a thousand a kid, you know, what does that mean? A bunch of different teachers came in like this wasn't an experiment and causations really hard to draw.

I think you need to marry the research with lived experience a little bit. I lived in New Orleans before the storm. I tutored in public schools before the storm.

Did you get caught in the storm? No, I was away at law school when the storm actually hit and then moved back down. And the classroom I tutored in in when I was living there before the storm was to this day probably the worst public school classroom I've ever been in my life. A year after I was done tutoring, there was a gang style drug execution at a middle school campus at the campus. I was tutoring it.

Wow. During the school day or during the school day.

And so. I think there is both lived experience of the people who were there before and after the storm, that shows that's just kind of hard to refute if you were there on both sides, that the schools are just fundamentally better. And I think the test score data adds to that belief. And so there's multiple reasons why I think things are better. All that being said, it is a very unique circumstance and causation is hard to draw. So the organization I'm working at now, the city fund, our goal basically over the next five to seven years is to try a version of this model and another 10 cities or so and see if we get similar results at a greater level of scale.

You know, that is in the bigger questions of the role of philanthropy, the role of evidence, how quickly or slowly should you move these things? And I think we're trying to take the pragmatic and somewhat conservative approach of we think we have a couple of good data points and we want to try it at one more level of scale and see what happens. There are other cities in the country that have that have implemented charter schools on varying levels. What what have we learned from those other examples?

Like what what cities have seen success with charter schools? Which cities have and do you understand why we see the different etc.?

So I am a big, big believer in urban public charter schools.

Why that specifically there are just very few public policy interventions that have shown good results at any level of scale. And so the national data on Urban Charter Schools CREDO, a research group out of Stanford, has done the biggest research. So they're looking at, you know, the sample sizes in the tens of thousands of kids, if not larger, might be in the hundreds of thousands. And the test score results are positive. I think the last one I saw is over a three year period.

It was about a point, one standard deviation jump, give or take between point six and point one. And the results improved over the course of the study. So urban charters are scaling. They're getting good results. In the course of that actual five year study, the effect size nearly doubled from the beginning of the study to the end. They cost less than the thing they're competing with replacing on the average urban charter schools get about 25 to 20 percent less public resources than the traditional system.

They also don't hurt the traditional system. So you could be worried you're scaling this and then the traditional schools are losing money and kids. It is common.

In fact, the teachers in Los Angeles who are striking right now, I feel like that was one of their main grievances, was that charter schools were leeching funding away from them. Why do you think that's not true?

There's a lot of research on this. So the last meta study I looked at, they looked at 16 different cities and regions. And out of the 16, only one had a negative effect on the existing system. And it was very slightly negative and everything else was neutral to positive. So I think we have pretty decent evidence that you don't see massive drops in academic learning as charter schools.

Oh, I was thinking just narrowly of the funding to the schools. Well, that's true, though. I don't think the point of a public school system is to increase its revenue. It's to increase learning. And I think we should think about the public dollars being more associated with children than any one given institution. So I don't view movements and dollars as a moral positive or negative, much more concerned about. I think if we were watching charters grow and then that was causing, you know, crazy academic drops and then traditional system, then we'd have to think about are we hurting some of the people that aren't getting it?

And we just don't see that as interesting, though, if if the public schools have less funding but academic outcomes haven't dropped, that's kind of an interesting result. It's funding per student.

So maybe that's the same thing I read from the teachers was that, you know, our schools have this. A lot of it is a fixed cost. And so if you take away funding per student for each student, that leaves to go to a charter school that like we're actually more we suffer more than just like proportionately for the students who leave the school.

So I think I have some empathy with, like one kid leaves the school. You can't easily cut, you know, the ten thousand dollars overnight. I I think the union often assumes that labor is a fixed cost and the predominant cost of public education are labor costs, and in most industries, laborers are assumed as fixed costs. Right. So in aggregate, I'm pretty skeptical of that argument. And I think the evidence is bearing that out. Like, it just doesn't we don't see that anywhere that charter schools are causing a negative impact on the traditional system.

Interesting, because it suggests, like, if you follow that logic, it seems to suggest that we should maximize class sizes and so on because, you know, we want to get more benefit per dollar.

Exactly. So so, yeah, I just you know, when you think about public interventions, I think you should start with the baseline that almost nothing works in here with urban charter schools. We have something that works, something that's getting better over time, something that works more cheaply and is higher productive, something that is scaling fairly rapidly and something that doesn't hurt the existing system that's like is close to you get is like a miracle in public policy and have just become a huge believer.

And I think the politics of it are unfortunate. So, you know, to your original question, what gives me some optimism in these other cities? I think we have a pretty long 20 year track record that urban charter schools are doing really positive things for low income kids. Now, I think the thing we don't know is does that hold true at higher levels of concentration and says the city gets to 30, 40, 50, 60 percent charter.

Do those results still hold on all fronts?

Why might they not other than just the general? When things scale up, who knows what happens?

So I think, A, that who knows what happens when things scales are very real. I do think one thing I saw in New Orleans and I think the charter community needs to be a little more honest about this, is every time we started serving more kids, the kids were getting a little harder and harder to serve. And so the first 10 percent of kids who go into charters, the parents may have more resources and so forth to find those schools.

When you're at the 80 percent kid or, you know, you're going from 90 to 92 percent student share, you're getting the kids whose families, for some reason or another, have kept them in schools that are struggling a lot. Yeah. And so it could be that, you know, maybe the model falls apart as you're serving harder and harder to reach kids. Now, I think New Orleans has strong evidence that we went all the way in those part of the reason I was morally compelled to try to get the city all the way as I wanted to show that it could work and I thought it could work.

And I actually think New Orleans right now is one of the most equitable public school systems in the country because it went all charter and what we're doing for the hardest to reach kids. Isn't it surprising, though, that if students if as more and more students go to charter schools, the ones who increasingly take that step are come from families that are maybe more motivated to, you know, find the best education for their child, they're more involved, etc.

, maybe different socioeconomically. That seems like it would imply that the students left behind in the non charter schools would be, you know, have less involved or less motivated parents. How could it be that we don't see educational outcomes dropping in the regular public school system?

It's a great question. I'd say partly very few cities have gotten to scale. So I think this is one of those things where testing like that effect probably wouldn't hit in the aggregate until you're getting to 30, 40, 50, 60 percent. Now, I think one thing that's positive that is going in the opposite direction of what you're saying is the effect of competition. And so if you take a city like Washington, D.C., charters, there are about 45 percent of total students served.

And as the charters were growing, the district kept on getting better and better. And we really got into an all boats rising situation. And so that's my hope, is that the competition creates more nimbleness in the traditional system and that's better for the kids and educators.

And that system, too, is the competition, mostly in the form of charter schools that are underperforming get shut down, whereas in the previous traditional system, underperforming schools were shut down, or is it more in the form of parents looking at their options and choosing to send their kids to the schools that perform better?

I think both of those are really important and I would add a third of having proof points that things can be different. So if the government's running all the schools and they're all, you know, a lot of them are mediocre, it's hard to know that things could be better. And when you have a couple of charters that are doing amazing things for low income kids, it raises the political question of why are not schools like this?

Right. Do we have any evidence about how parents make these school choice decisions? Like do they actually tend to look at the stats and, you know, use test scores as their main deciding factor or what?

Yeah, we're starting to get a lot of evidence, which is really exciting, I think, both for policymakers and then both for families that are using more transparent system. So in about five of the cities we work in, they've created online unified enrollment systems that are very easy to enroll. So if your kid starting kindergarten, you look at every school in the city, it's on one website you can compare and then you rank your top seven and then the government gets all that data aggregated so we can see what schools are being ranked highest.

So in economic terms, we have revealed preference rather than focus group and polling preference, which is amazingly useful. And what we're seeing is there's three to four main factors in these cities, which are generally urban environments. I don't want to, you know, the external validity be different environments. They care about the government performance labeling system. So if there's a through F system that the state gives every school or a star system, they look at that and the higher performing, the more they have that matters to them.

And it's revealed preferences, this like coefficients and a regression.

So you're looking at association. So we take all the schools, label them on their performance, extracurriculars they offer and so forth. And then you what what characteristics of schools are correlated with high parent demand we kind of look at. And so the correlations are a high score on the state regulatory system based on test scores, usually distant parents, all things equal would rather send their kid down the street, extracurricular activities, football band, things like that, sibling preference.

If your kids in the school you want to send your other kid right. And you can also come up with kind of interesting equations on how far parents are willing to make these trade offs. So I think in New Orleans, it was roughly parents were willing to travel an extra mile or two for every letter grade interest in the school. Yeah, and I think that really made it. You know, there's a human element where families have complicated lives and they're making a bunch of tradeoffs.

And through these systems, you can get a better understanding of the things they're trading off. It will say one thing that came out of a study in New York, which has a similar system, that. Is either interesting or troubling, depending on your perspective, as parents cared more about absolute student achievement than academic growth. So there's a set of schools in New York that are kind of famous. You know, they'll knock it out of the park.

But there's also a bunch of schools in New York that are taking kids in at a lower place, but moving them much faster. Yeah, and the high demand schools where the famous absolute achievement schools. Now, it's an interesting question of whether families are drawn to those schools because they actually think they're the best. But had they known that there are these other schools that have high growth, they'd pick them. Or if families care so strongly about peer effects, they're actually saying we actually don't care how much our kids are going to learn, we just want them around a bunch of other high fliers.

Or this could also be evidence for Bryan Caplan signaling model of education, where the median value of the education is not the increase in human capital, what you learn, but instead, you know, having that on your resume that you went to that school to employers, then we would expect to see the parents valuing the schools with like like the the star schools. Exactly.

And so there's a couple of ways you can respond to this. As a policymaker, you could say this is what parents want. So we should give them this information and let them choose these, you know, high achievement schools. You could also say actually parents should value how much their kids are learning. And so let's create accountability system that gives a threat scores. That's mostly based on growth. Right.

And yet proposed a nudge system where we try to nudge parents to pick the schools but actually improve scores more.

So there are experiments being run right now about how you show information to parents in these systems and figuring out what that means. Now, again, you want to balance. Do you know what the right thing to show is like? Because if you're nudging, that means you have an opinion on how parents should be making decisions, right? Versus do you want to be a little more humble and say we're just going to show the information as neutral as we can, you know?

Yeah. Are pretty complicated decisions for policymakers on how to design choice systems.

Yeah, I'm just trying to think about other objections that I've heard to charter schools that we haven't yet covered. Well, one of them is the claim that charter schools increase segregation or that they're bad for diversity because people are going to choose to send their kids to schools with other people of the same race ET or maybe just white parents are going to try to send their kids to charter schools that just have white kids, etc.. Is that something you're concerned about?

And if not, why? Out of all of the things that have caused segregated public schools in America, I would put charter schools at the far bottom of the list. We have a very unfortunate history of segregated schools in this country. Brown v. Board took a big dent into that problem. And then we saw white flight where a lot of white people moved to the suburbs to basically get segregated public schools. Public schools, unfortunately, this country has been a bastion of segregation, not much in solving the problem.

So I, I just find it really disingenuous when people say charter schools are the reason we have segregated public schools rather than a long history of racism in our country. In terms of how a I think there's two questions. How much should this matter? In other words, should we just focus on making schools better regardless? Or is integration in of itself a value? And then two, to the extent you believe integration to value, how would you move further in that direction?

Speaking for myself personally, I do find integration to be a value. I want that for my own child and I think it's healthy for society. We're all in this together in some sense, and having kids grow up together I think is an important component of that. I think what Brown v. Board showed us is the limits of what government can do to force segregation if people don't want it. And so it's going to be a much longer path. But I think we have to choose integration and I think charter schools can be a model of that.

So one small anecdote. There's a school called Bricolage Academy in New Orleans. It was a teacher who worked at a KIPP school and thought that school was just phenomenal for the kids who are serving. He had two daughters. He was living in New Orleans. He put them in kind of, you know, one of the elite private girls schools in the city, which is a phenomenal school, and then was just trying to resolve this dichotomy of I think there's this amazing school, KIPP, my daughter's on in that school.

Is there a better way? And so he decided to found a school that would be intentionally diverse. And he went to wealthy white families across the city and black wealthy families and said, will you pull your kids out of private school? If I can also recruit low income kids and we can be this in this together, he successfully did that about a 50 50 mix. And I think last year it was one of the top three most in demand schools in New Orleans.

So I think that's the future is we have to figure out a way to choose this together and say it's a value. I'm highly skeptical that we can force our way into integrated systems.

And that is exciting as a model for how charter schools could could reduce segregation. I'm just curious if we if your position is a there's no reason to think charter schools are making the problem worse, but like, here's how they could make it better. Or, B, charter schools do make segregation worse, but that's sort of inevitable for the present and like not the main reason segregation is bad. Yeah, got it.

I don't know enough outside of cities to answer the evidence on integration within most of the cities we work. It's kind of a moot point in that they're all low income minority systems anyways. And so we are working in cities that have unfortunately already been segregated and we're just trying to make the schools better. Got it.

I guess the last objection that I can think of off my head to charter schools that have haven't covered is just parents who put their kids in charter schools, then having less stake in the public school system and being probably less inclined to, you know, vote for superintendents who are going to do a good job, etc.. Is that something you're worried about? If forcing people to attend crappy schools is a way to increase civic engagement, to then fix the schools, we've run that playbook and it hasn't worked.

So I think I'd question the premise that that is the fastest and best way to improve public school because it could always get worse and worse could possibly be.

Yet I have found through relatively small experience in New Orleans and other cities that charter schools can actually be a way to increase civic engagement on a couple levels. I think the act of actually having to choose a school puts parents in a position of more civic power. Having choice, having the ability to exit is a form of political activism that I think is empowering. And so I view it as a civic activity. I think having a bunch of nonprofit boards that are made of community members increases the amount of people who are actively engaged.

But in the charter school of governance.

Yeah, but, you know, I'm working trying to scale the charter school governance to talking to New Orleans. And I would just look at the voting turnout rates of school boards. They are not very high. I have not found school board elections to be the high point of civic engagement. I'd encourage you to go to school board meetings. They can be very disillusioning. So I think that's an idea that might be a good talking point, but it doesn't really hit reality when you look at the unfortunate lack of democratic accountability and civic participation, do you think that American public schools in general have gotten better or worse or stayed roughly the same in the last, let's say, 40 years?

I think it's a generally positive story in that they are getting better in the sense that so there's a test called NAPE, which is the only nationally norm tests that we have that has a large enough sample across the country. And in younger ages, we have seen gains in reading and math over the past 40 years. In 12th grade we have not. So that is still positive to me in that that's a sign to me that more people are gaining basic literacy and math skills, even if they're not, you know, knocking it out of the park and Algebra two or some of the harder high school things, we've seen even more improvement in Hispanic and African-American students, which, depending on the grade and test, have roughly closed half the achievement gap with their white peers.

And so when I think of our work, I think of our work as the next wave of improvement in public schools, not as we're on a 40 year decline that we're trying to reverse. I will say in the cities we're working, not all of them benefited from those gains. So places like New Orleans and D.C., unfortunately, still had extremely awful public schools, you know, 10 to 15 years ago. We talked a little while ago about how you have changed your mind about what the men, the secret sauce of charter schools was going to be, and you come around to thinking that it's more about the governance than you thought and less about the innovation.

Is there anything else that you've changed your mind about with respect to charter schools or just education in general since you started working in this field?

Yeah, a couple of things. And this probably goes to another critique of our work, which we try to grapple with. I think I came into the work thinking charter schools are best used as pockets of innovation that would then do all this amazing stuff and then the traditional system could adopt them because I didn't think they could scale at the level to reach every kid. I have since changed my mind on that. And I think they're much more scalable if you think across appropriate time horizons than I originally thought.

You know, there's a give or take 10 to 15 million kids in poverty and public schools in America over a 30 year period. If it works, we shouldn't scale it if it doesn't work. But if we follow the evidence and they continue to do great things, I think there's no reason charter schools couldn't serve the vast majority of those low income kids.

Why so relevant to a critique of your work that you're the sounds like a good thing?

I the critique is that charter schools aren't worth doing because they're not scalable. And I, I used to agree with that critique. And so I have now changed my mind and that I do think they are scalable over a over a time horizon of 20 to 30 years. But when you're thinking about changing something as big as public American education, I think that's reasonable. And again, only should expand if the results hold. Other things I've changed my mind about, and I think I've decreased my belief in the importance of test scores.

I just used to think that was the way you evaluate public schools. And now I'm much more of that's one piece of the puzzle. What else would you look at? I'm a big believer in parent demand, I think for a couple of reasons. One, you know, again, the connection between test scores and long term outcomes in some sense is intuitive. Like if you can't do basic reading and math, that's probably going to hurt you at some point.

But that that linear relationship continues, you know, up to being it's unclear to me that going from like pretty decent test scores to absolutely amazing at test scores is the best margin to work on once you reach a certain threshold, rather than things like character, values, conscientiousness, agreeableness that seem to be pretty important in the modern economy. Yeah, and so I think I have now a more holistic understanding of the things schools could potentially work on. Those things are also really hard to measure.

So I'm less likely to say we should have government metrics on some of the more important aspects of schooling. So that leads me to parent choice as another way to understand, as schools are doing something good, not perfect in of itself, but useful.

I mean, I also feel that rings true to me too. But doesn't that lead to this problem where if we want to know whether charter schools are working, we just like it seems kind of circular where like, well, parents are choosing these schools and that's our metric of whether they're working. Therefore, the schools are working.

But parents would you know, they choose some schools.

And so it could be like whatever whatever happens that would lead that metric would lead to the conclusion that charter schools are working.

Yeah, I think I'm a pluralist on this. So I think we should look at both test scores and parent demand. In other words, have parents role all choosing schools that had very negative test score effects. That would make me very curious and a little hesitant. Now, it could be the parents are seeing something that the test scores aren't picking up, but it would make me uneasy and make me worry that the parents might be missing something. But it would be complicated.

I think with urban charters, we're seeing both. We're seeing positive test score effects and parents are choosing them like a charter school can exist and lets parents make an affirmative choice to go there. Yeah, and so they're passing both tests, the research test and the parent demand test. And so both of those things together make me increase my confidence.

Would it do you do you think that a good metric of whether parents, how much to trust parent choice would be how robust parent choice is to the kind of like reorganizations of how we present the information about schools? Like maybe if, you know, parents are really selecting carefully or thoughtfully or whatever, then they're going to be more likely to pick whatever schools listed at the top or, you know, something like that.

Right. I think. And this is a hard part and working complicated public systems, you really have to let that unfold for many years to have hard opinions like, yeah, you're building a market, so to speak, and it's going to take a while for parents to figure out what they want and then they're going to tell their friends. And then based on that, you know. Yeah.

So I don't think it's you're going to look at the outcomes of the school after it's been around for. Exactly. And so I think you just have to be thoughtful and give information as best you can. Yeah. Be a little humble and assume that over time you'll get into a good equilibrium. Cool, well, that's probably a good place to wrap up before I let you go. I like to ask my guests to nominate some resource, be it a book or website or even a person like an author who they have substantial disagreements with.

But nevertheless, respect or think is worth reading or engaging with. Anyone comes to mind or anything come to mind for you about education or otherwise?

Definitely a lot. One line of thinking that I try to grapple with, and that's something I respect, but I'm probably not solely in the camp. So think of charter schools as generally central left, you know, a little technocratic, still saying we want to be a part of a public system, but a little entrepreneurial, a little free market, but still kind of in the centrist camp, the libertarian critique of, you know, as long as you're taking state tests, that means you're basically dictating what should be taught and you're putting schools.

And, you know, if you want real innovation, it's not going to happen through the charter model. If you want real parent choice, you need to let it rip a little bit more. You need way more risk taking and opening and closing schools. And so, you know, Jay Green out of the University of Arkansas is a big proponent of that thinking and then kind of interrelated into that of the Bryan Caplan line of thinking of the thing you're maximizing for is foolish.

Like, you know, I think Brian has a much more sober take on what schooling can actually do at scale. And I've moved a little bit on that to say towards Brian's.

Yeah, I think and luckily this, you know, in certain synth lines up with the core of our work. Yeah, I think of a deeper belief based on Brian's work that getting high levels of basic math and literacy is one of the more important things public schooling can do. And being a healthy babysitter so parents can work and being a safe place with good values. And so that's really the pre-K through part of the world. You know, I think Brian would be a little more skeptical of whether we should teach advanced mathematics to all these kids and make them suffer in junior and senior year.

Right. And I do think the bulk of our work in the most important of our work is really in so many kids and so many of these cities have just been not given an educational opportunity where they can be prepared for the workplace. And the lead choice, meaningful lives in Brian's work is, I think, refocused my emphasis along those lines a little bit. So the Brian Capling, Jay Green line of thinking, I think anybody who starts with my set of beliefs would do well to grapple with those ideas and be influenced by them.

Great will link to. Do you have a particular work by Jay Greene or should I?

He runs a blog so we can link to his blog and Brian Kaplan's book on education as well. Worth reading. Great.

And I'll also link to your blog Relinquishment. Is it dotcom or Dog? Great Relinquishment Dog, which has some great blog posts and and links to various other interviews and writing. It's been great having you. Thanks so much for coming on the show.

Thank you. It's been wonderful to be here.

This concludes another episode of Rationally Speaking. Join us next time for more explorations on the borderlands between reason and nonsense.