Can a teacher’s worth be measured by how much hisÂ students’ test scores improve? Â And should teachers who don’t move that needle very much be fired? These are two of the most controversial questions in education.
Some school districts have plunged ahead with “yes” answers to both. Whether that strategy — which leads to high teacher turnover — has improved student learning has yet to be fully analyzed. But aÂ first stab at it has just been released by a quartet of researchers from the University of Virginia and Stanford University.
First, some background.Â It might seem pretty obvious that firing ineffective teachers and replacing them with better ones would be good for students. But identifying which teachers are ineffective is tricky. And it’s even trickier to dismiss unionized employees. Whether to pursue a dismissal has often been left up to the whims and energy of school principals. That’sÂ why some policymakersÂ came up with the idea of finding objective measures, and thenÂ adopting new firing policies.
Those who oppose measuring a teacher’s worth by test scores argue that the multiple-choice tests don’t capture a lot of what students are learning, and can underestimate great teaching. Even those who agree that the tests are somewhat good at identifying the worstÂ teachers find other problems. For example, some teachers react by dumping creative lessons and teaching only topics that are on the standardized test.
And then there is the collateral damage from firings. It’s not just the teachers who are let go who suffer; it’s also disruptive to the entire school, possiblyÂ harming students. All the firings and new hirings devour principals’ energyÂ and take time away from curriculum improvement and teacher training. And it can take months, even years, for new teachers to adjust to a new school. PreviousÂ researchers have found that schools with higher teacher turnover have lower student achievement.Â Even more troubling, a system that ranks teachers by test scores can stoke fear and resentment in the heart of any teacher.Â The stress might cause a school’s very best teachers to leave. Morale can plummet.
The academics from the University of Virginia and Stanford University set out to study how a system that increases turnover by firing low-rated teachers affectsÂ student learning. Specifically, they looked at the Washington, D.C., public school system from 2011 through 2013, after a new ratings-and-firing policy had been put in place there. The authors documented an 18 percent teacher turnover rate, one of the highestÂ teacher departure rates among big cities in the country.
Despite the high turnover, the authors found thatÂ student performance, especially in math, improved. Roughly 400Â good teachers left their jobs — about 13 percent of the those rated “effective” or “highly effective”. But more than three times that rate, or 46 percent, of low performers left. That amounted to 130 teachers who were either automatically fired, because their ratings were so low, or quit of their own accord.
“They wereÂ replacing them with teachers who appear to be substantially better,” said Thomas Dee, an education professor at Stanford University, and one of four co-authors of theÂ working paper, “Teacher Turnover, Teacher Quality and Student Achievement in DCPS,”Â published on the website of the National Bureau of Economic Research in January 2016. “Rigorous evaluation of teachers and then trying to replace consistently low-performing teachers appears to be effective.”
In the study, Dee and his research colleaguesÂ looked at the test scores of everyÂ grade from fourth through eighth in each school.Â If a fourth-grade teacher was rated low-performing and removed, they compared the test scores of fourth graders before and after the removal in that same school. The same was done for all departing teachers. Â After a high-performing teacher left, student performance usually went down. ButÂ after a low-performing teacher left, student performance usually went up.
When you net out these two effects, there was an overall positive gain in math scores, although one that might seem rather small (“8 percent of one standard deviation”). But Dee described itÂ as equivalent to 10 percent of the black-whiteÂ achievement gap, as measured by national tests. Reading scores rose less, and were only “weakly” significant from a statistical perspective, meaning that it’s unclear whether reading outcomes improvedÂ or notÂ after the firing policy.
At the time of this study, a teacher’s rating was composed of several parts. Growth in student test scores accounted for 50 percent. The remainder was based on subjective teacher observations, principals’ ratings and the entire school’s test score improvements.
One critic of using student test scores to rate teachers said the study was “interesting” but not persuasive.
“D.C. was putting in place a lot of reforms during this period. Curriculum. Coaching. There are a lot of things that could account for the gain in scores,” said Linda Darling-Hammond, also a professor of education at Stanford, but not involved with this study. “When you see a change in scores, you have to have a way to evaluate all the policies; you can’t just say that the policy you’re interested in accounted for it.”
Darling-Hammond’s research has shown that rating teachers by student test scores, known as value-added measures, is flawed and volatile. Often a teacher with a high score one year will get a low score the next, and vice versa. Teachers of gifted students canÂ find it hard to show test score growth because their students’ test scores start so high. Similarly, teachers of special-needs students or English language learners can have students who are so far behind that even if they catch up two grade levels, the test doesn’tÂ capture that.
Still, it’sÂ possible that the D.C. district was correctly identifying weak teachers and replacing them with better ones, and that their superior teaching skills were the reason student test scores improved. But it’s unclear if other school districts could replicate this, or even if D.C. could replicate this now. A big factor is the teacher job market. This study took place after the recession, when there were a lot of teachers looking for jobs. The city had the luxury of choosing from a great pool of applicants. With the economy’s rebound, there are now teacher shortages. And in other regions of the country, it might be hard to recruit good teachers altogether.
[Cross-posted at The Hechinger Report]