Perhaps I’m reading the story wrong, but a quick look at the NYT’s new piece about Common Core test scoring suggests that reporter Motoko Rich has provided perhaps the most emphatic, critical take on the scoring process among the several other outlets that have also covered it in the past few weeks. But the underlying assumption behind the piece — that classroom teachers would make better scorers than college-educated outsiders — isn’t really fleshed out or fully examined, and may not stand up to scrutiny.
The Times story (Grading the Common Core: No Teaching Experience Required) is datelined San Antonio and focuses on the absence of classroom teachers among those scoring the new Common Core tests that were administered this spring in many places:
On Friday, in an unobtrusive office park northeast of downtown here, about 100 temporary employees of the testing giant Pearson worked in diligent silence scoring thousands of short essays written by third- and fifth-grade students from across the country. There was a onetime wedding planner, a retired medical technologist and a former Pearson saleswoman with a master’s degree in marital counseling.
There are complaints and concerns from classroom teachers who think they could do better:
Some teachers question whether scorers can grade fairly without knowing whether a student has struggled with learning difficulties or speaks English as a second language…. Experienced teachers also say that some students express themselves in ways that might be difficult for noneducators to decipher.
Sometimes students say things as a student that as a teacher you have to interpret what they are actually saying,” said Meghann Seril, a third-grade teacher at Broadway Elementary School in Venice, Calif., whose students took the Smarter Balanced test this year. “That’s a skill that a teacher needs to develop over time, and as a grader, I think you need to have that as well.”
Rich asks a Pearson executive whether the process is comparable to McDonalds or Starbucks and — unfortunately for Pearson — gets a response:
“McDonald’s has a process in place to make sure they put two patties on that Big Mac… We do that exact same thing. We have processes to oversee our processes, and to make sure they are being followed.”
The use of temps and non-educators is a familiar and understandable complaint about test scoring procedures for pretty much every standardized test other than Advanced Placement.
I’ve highlighted these concerns any number of times in the past. See for example: Putting Testing Flaws In Context; The Elephant In The Room; A Scathing, Humorous Look Inside The Testing Industry; Dan Rather Examines Test Scoring …; “The Truth About Testing”; Testing Companies “Streamline” Scoring, Oversight.
One of Rich’s predecessors at the Times (Jacques Steinberg) wrote a scathing series on testing and the scoring process way back in the day (The Test Industry’s Failures; When a Test Fails the Schools; Right Answer, Wrong Score ).
If anything, the process Rich describes is an upgrade over how other, previous standardized tests are often scored (other than the AP).
And indeed, in and amongst are some details suggesting the recruitment, training, and supervision of the scorers isn’t quite as loose as McDonald’s:
To get the job, like other scorers nationwide, they needed a four-year college degree with relevant coursework, but no teaching experience. They earned $12 to $14 an hour, with the possibility of small bonuses if they hit daily quality and volume targets…. About half of those who go through training do not ultimately get the job.
But again the focus of the piece (in terms of what’s described and who’s quoted) is on the potential downsides. Given the chance to explain why it doesn’t use teachers for scoring, a Smarter Balanced executive gives a pretty unconvincing response:
“Having classroom teachers engaged in scoring is a tremendous opportunity,” said Tony Alpert, executive director of Smarter Balanced. “But we don’t want to do it at the expense of their real work, which is teaching kids.”
Does it matter if teachers score the tests or not?
At a visceral level, sure. We want those closest to the learning process to be doing everything. Teachers are the best. But classroom teachers don’t tend to have the strongest academic backgrounds, there are obvious conflicts of interest when it comes to having teachers score tests of students in their own classrooms, schools, or districts, and we don’t even know for sure that teachers are better at scoring standardized tests – the fundamental unexamined assumption behind the Times piece.
Teachers claim they could do better, but the claim isn’t confirmed with research or countered with another equally credible voice suggesting that they might not make such good scorers as they think. The piece would have been much stronger if it had backed up its central claim.
For comparison, here’s the EdWeek story about the process from last month (Thousands of Scorers Take On the Common-Core Tests) and the package of stories from the Cleveland Plain Dealer (Graders can shoot for 60 answers per hour, A look inside a Common Core test grading center, Who’s grading your child’s Common Core tests?)
Related post on Rich’s kindergarten playtime story.
