School Accountability Grows Up

Education Sector published a new report today that Rob Manwaring and I wrote, about using growth models in K-12 accountability systems. The long and the short of it is that when NCLB was written 10 years ago, most states had neither annual tests nor statewide data systems that track the movement of individual students among schools and grades over time, which are two of the things you need to calculate how much individual student learning is growing (or not), as measured by standardized tests. Today, most states have (or soon will have) those things, which means that so-called “growth models” can and should be incorporated into K-12 accountability systems.

A number of states have already done this via a waiver process developed by the last administration, and we can learn from what’s worked and what hasn’t. Using growth for accountability purposes can be complicated, as shown by Bruce Randolph School in Colorado, the school that President Obama chose to single out for praise in his State of the Union address earlier this year. According to the state of Colorado, the largely poor and Hispanic school has made unusually strong growth while helping many students graduate and go to college. But the state data also suggest that students at Bruce Randolph aren’t growing fast enough to meet state proficiency standards by the time they finish high school. Figuring out how to think about schools like this is one of the challenges of designing and implementing growth models.

One of the accountability experts who reviewed a draft of the paper noted that it’s light on very specific recommendations for how growth models should be designed and incorporated into K-12 accountability systems under ESEA. Since I just got finished dinging Rick Hess for not getting specific, let me explain my reasoning here.

I think the federal government should insist on the creation of useful education information. States that take federal education dollars should be required to have standards and administer annual tests in core subjects, all the way through 12th grade. They should also be required to develop high-quality longitudinal data systems that connect K-12 school information to labor market data and post-secondary information systems. They should be obligated to use those systems to calculate and publish sophisticated growth estimates, among other things, including measures of how people who attend K-12 schools ultimately fare in the labor market and higher education. All of this information should be open and available, in ways that protect student privacy to be sure, but are otherwise user-friendly and actively provided to the public, not just under FOIA duress. Doing these things should be mandatory for receipt of federal money. With information systems being what they are today, there are just no plausible administrative burden arguments against data anymore.

But I don’t believe that Congress or anyone else can design a wholly rules-based accountability system that slices and processes and evaluates that much information into mechanistic judgements of success that are sufficiently accurate and credible to plausibly serve as the foundation for educational improvement. There are just inescapable tensions between the amount of information needed to render a reasonably accurate judgement of school success, the inherent complexity of any rules-based system for processing such information, and the transparency and credibility necessary for people to believe in and constructively react to the judgement of the system–and, as such, for the system to work. The more variables you add to the accountability equation, the more complicated and opaque–and, therefore, less useful–the equation becomes. Accountability systems aren’t aircraft engines, where we don’t mind that we can’t understand them because they manifestly get us safely from here to there. They don’t work if people don’t understand how they work.

That means we need to rely on human judgement to interpret accountability information and decide when and how to act upon it. There is no doubt that some state and local officials will fail in this responsibility, due to incompetence or bad faith. But here’s the thing–those same officials are perfectly capable of subverting a rules-based system to the same ends. That’s the story of NCLB implementation over the last 10 years. We need better officials, not an official-proof system.

This isn’t the case with all aspects of federalism and education policy. It’s a very good thing that the Supreme Court banned racial discrimination and it’s a very good thing that some state courts have required legislatures to distribute school funding fairly. Students with disabilities everywhere depend on IDEA. Preventing discrimination, providing procedural and educational rights, and redistributing money are all things that governments can and should do via force of law.

What they can’t do is force people to build educational environments that are really good. Shutting down really bad schools is a different matter and we shouldn’t discount the difference between terrible and mediocre. But for the most vulnerable students, only really good environments will do. There’s no grudging path to excellence. People really need to buy in. If you’ve created a compliance mentality, you’re doomed from the start.

So from a policy standpoint you fight like hell for real information and transparency and you try to feed it into policy and market environments that can benefit from it. You use the information to inform scholarship, influence public opinion, identify best practices, guide parental choice, and hold elected officials to account. But you don’t pretend that rules will suffice when only people will do.

[Cross-posted at the Quick & the Ed ]

Kevin Carey

Kevin Carey directs the Education Policy Program at New America.