Why WNYC’s District-Charter Attrition Comparison Has Gotten So Little Love

According to last week’s WNYC series, NYC Charters Retain Students Better Than Traditional Schools — though different networks fare differently (see below).

Yep, that’s right.

In contrast to the findings from anecdotal coverage published by other outlets — and the understandable instinct to believe that tough discipline and suspension practices would cause parents to pull their kids from charters in large numbers — WNYC found that, “citywide, across all grades, 10.6 percent of charter school students transferred out in 2013-14, compared to 13 percent of traditional public school students.”

In elementary school, district attrition is 40 percent higher than charter attrition. In middle and high schools, charter attrition is “very slightly higher than district attrition,” according to the WNYC story — a finding that’s confirmed by other studies, it says.

Rather than being picked up and passed along in a flurry of social media sharing, as is common with eye-opening journalism like this, the WNYC story was largely ignored, according to the journalists who produced it.

Why is that, and what if any issues are there with the WNYC piece? Read on to find out. 

Charter Network Students Who Transferred Out Expected Transfers* Percent of District Attrition
Kipp 74 290 25.6
Uncommon Schools 172 543 31.7
Icahn 72 209 34.5
Achievement First 217 575 37.8
Ascend 164 385 42.6
Explore Schools 116 255 45.5
National Heritage Academies 150 328 45.7
Success Academy 609 1061 57.4
Democracy Prep 81 96 84.3

WNYC chart comparing charter network retention rates. 

In a phone interview with WNYC education reporter Beth Fertig and data gurus John Keefe, and Jenny Ye (pictured below), Fertig described how the FOIA process required for obtaining the data went back to 2014 – it took the DOE a year to respond – and that the data they received back comes from an antiquated DOE system called ATS complete with glowing green letters on a black screen (think “Wargames”).

Asked why they decided to go to all the bother, Fertig responded that “There’s been lots of press and noise about charter school retention rates.” Part of what they wanted to figure out was whether, with all the growth in charters and the increased scrutiny, “Has there been a change? Is there going to be more attrition now than in the past?”

Jenny Beth John -- photo by Richard Yeh

Left to right: WNYC’s Jenny Ye, Beth Fertig, and John Keefe.

ANECDOTES VS. STATISTICS

In addition, one one of the main goals of the story was to get beyond anecdotal reports, according to Fertig and her colleagues.

The New York Times has focused on reports of tough disciplinary practices at some charter schools, and Chalkbeat NY has reported on charter school suspension rates.

But it wasn’t known whether, on the aggregate, tough discipline and even suspensions at charters would correlate with parents choosing to pull their kids out of these schools.

“You might expect there to be more attrition,” said Fertig. “We do hear from principals [about charter school kids arriving on their doorsteps].”

That’s not what they found, however. 

“It helps to look at numbers dispassionately,” said Keefe about the dataset. “That doesn’t mean peoples’ individual experiences are wrong, but it puts them into perspective.”

REACTIONS

However, it may be that perspective is something that readers and colleagues don’t always want to hear.

Given the usual human response to ignore information that doesn’t confirm pre-existing beliefs (or previous reporting), it’s not so surprising that the story hasn’t gotten picked up as much as it might otherwise have done. (Google “confirmation bias” if you’re curious — or doubtful — about this phenomenon.)

In addition, there’s the ever–present challenge of cultural and racial bias: We all would like to believe that other people would make the same decisions we would, regardless of differences in culture and background, and tend to discredit or diminish decisions others make that don’t match our own. 

Other than a NY Daily News editorial that ran last weekend, the story was largely ignored by other local media outlets, according to Fertig. (The story was discussed on an in-house talk show here.)

Asked if part of the reason the story didn’t get picked up more broadly because it contradicts the dominant narrative about charter schools, Keefe chose his words carefully. “If our results showed something different, would that headline be picked up more broadly? I think that’s a good question.”

MERROW CRITIQUE

A few folks like former PBS NewsHour education correspondent John Merrow have gone well beyond ignoring the WNYC story.

In a blog post titled Eva’s Offensive, Merrow (a friend) calls WNYC education reporter Beth Fertig a “surprising ally” in a PR effort by Success Academy head Eva Moskowitz and describes the WNYC story as a “silly and meaningless exercise.”

According to Merrow, the comparison between charter school and district decisions isn’t relevant, given differences between charter and district parents. It’s “like comparing the kids who go to the playground to toss a ball around with the kids whose parents enroll them in the karate program at the Y, buy them uniforms and accompany them to practice and competitions.”

Indeed, the WNYC story doesn’t explicitly consider differences in charter school and district school parents’ socioeconomic status. (Others have studied these demographic differences — CREDO, IBO — and WNYC links to these.)

The WNYC story doesn’t also get into why students are leaving the schools that they leave. (There’s no data coded for that. The ATS system was set up to make sure the system knew when students left one school and arrived at another.)

However, the comparison between district and charter schools in the same geographic attendance zones is something of a proxy for parent and student demographics, and it’s unclear how much if at all a more elaborate accounting for demographics would change the overall picture.

In its comparison between district and charter schools (and among charter networks), WNYC controlled for differences in district attrition rates by “accounting for the neighborhoods that the charters were in,” said Ye.

[Though they worked closely with the district in the process of developing the story, WNYC didn’t ask the DOE for a response to the story. “It’s a data story,” said Fertig. “I didn’t feel the need to run up to everyone and ask them what they thought of this.” The DOE did not respond to emailed requests for comment.]

On the phone, the WNYC team stands by its findings but made clear that it makes no claim that its study is comprehensive or answers all questions about charter schools. 

Fertig declined to comment specifically on Merrow’s blog post. 

If any notable education researchers have taken the WNYC methodology to task (or praised it, for that matter), I have yet to see it. 

Alexander Russo

Alexander Russo is a freelance education writer who has created several long-running blogs such as the national news site This Week In Education, District 299 (about Chicago schools), and LA School Report. He can be reached on Twitter at @alexanderrusso, on Facebook, or directly at alexanderrusso@gmail.com.