Last night PBS’s Frontline featured a documentary on Michelle Rhee, the controversial former superintendent of Washington, DC schools.

Rhee, famous for instituting top-down accountability based on results from standardized tests, ran D.C. public schools from 2007 to 2010. She closed more than 20 schools during her time on the job, and fired 36 principals and almost 250 teachers. She resigned when DC mayor Adrian Fenty, who hired her, lost his bid for reelection.

The documentary outlined her time in office, the anger she generated, particularly among the city’s teachers union, and allegations of test erasures among schools. She was a take no prisoners sort of administrator, once explaining that “I think if there is one thing I have learned… it’s that cooperation, collaboration and consensus-building are way overrated.”

But did it work? Was Michelle Rhee effective? In fact, we don’t know. And we never will.

In 2008 Clay Risen wrote of Rhee for The Atlantic that the mayor hired her because he was “a data-focused decision maker, less interested in politics as usual than a politics of results.” But Rhee generated so much ire, from parents and the teacher union, that she left too soon to really provide useful data about the effectiveness of her reform ideas.

The reaction to the Rhee documentary was intense. Education historian Diane Ravitch writes of Rhee that:

The public schools in D.C. improved “slightly” on national tests but “are still among the worst in the nation,” and its high school graduation rate is dead last. We learn that her relentless focus on test scores produced allegations of widespread cheating, not better education. Her policy of firing teachers and principals did not turn around the schools; it created turmoil. Every year, about 20% of the teachers (including those she hired) leave, and most of the principals she hired have moved on.

The only logical conclusion from this documentary is that states and districts should not do what Michelle Rhee did. It didn’t work. It failed. Rhee, however, remains unfazed. She’s taken her reform agenda to the national stage and is now urging states to follow her lead.

Another critic, commenting on a CNN piece about the controversy Rhee created, said that:

When Rhee came to DC plenty of people did not agree with her methods and what she was saying. Some teachers even filed lawsuits to fight her. But I can say one thing she did improve DC public schools and the proof is in test scores and achievements of some former failing schools. Just because the thought of some teachers being the problem hurts a few feelings doesn’t mean she isn’t correct in that assessment.

It’s inappropriate to draw either of those conclusions. Two school years are nothing in terms of trend data. Her reforms were certainly controversial, but she wasn’t around long enough for us to know whether or not she succeeded or failed at what she set out to do: improve student achievement.

Perhaps her greatest accomplishment is that she got people thinking about education, and particularly the public education system of Washington, DC (historically an institution that most of the DC policy community, which is itself largely concentrated in Upper Caucasia, thought about much the way it thought about the public bus system: it’s probably bad, that’s what the cleaning woman says anyway).

What I find particularly annoying about this sort of discussion of Michelle Rhee though, is that the program didn’t present, and indeed probably could not have presented, a full picture of what it looks like to be a urban school superintendent.

It is, in fact, one of the worst positions in city government to have. That’s because the chances of success are so slim, and the barriers to achievement so great, that it’s hard for anyone to do a really good job, and a really well recognized good job.

In fact, it’s not possible to make dramatic leaps in test scores and graduation rates in a year. That’s not the way progress in education works. The only way to improve schools is steadily and with a lot of focus on what students and teachers need to achieve their goals. That doesn’t often happen in urban schools because the pressures are so great, and the job is so political, that it’s usually easier to just fire the superintendent (when Rhee was hired in 2007 she was DC’s fifth school head in decade; like her processors she didn’t make it for four years) than really see whether or not his reforms are working, whether or not he can lead. In order to really achieve meaningful success, he probably needs at least five years in the job, and he almost never gets it.

In the long run I suspect the “eliminate the teachers unions and focus relentlessly on the bubble tests” probably isn’t going to result in dramatic improvement in the education of America’s poor students, but critics of Rhee need to stop saying that Rhee “failed.” She wasn’t there long enough to succeed.

Daniel Luzer

Daniel Luzer is the news editor at Governing Magazine and former web editor of the Washington Monthly. Find him on Twitter: @Daniel_Luzer