What we did
Lynn and I were able to do something that, to our knowledge, has not been done before: estimate the effects of advertising and the ground game simultaneously. We took Dave Leip’s county-level data on Obama and Romney’s vote share and combined it with two measures of campaign activity. First, we relied on advertising data that we and John Geer purchased from the Nielsen Company. These data are measured in gross rating points (GRPs), which help to capture not only the total number of ads but also their likely reach. Second, we relied on the location of Obama and Romney’s field offices. Thus, for each county, we know how many GRPs were purchased at any point before the election and we know something about the extent of the ground game. Our approach allows us to look at the effects of ads and GRPs simultaneously. In our statistical model, we accounted for other features of these counties—including how well Obama did there in 2008 and various demographic characteristics.
Did the ads matter?
Yes, but in very circumscribed ways. The larger the advertising advantage that either candidate had, the more vote share he received. However, the effect of ads appeared to decay quickly. Ads on the day before the election appeared to produce small but measurable gains in vote share. The largest advantage that Obama had in any one county—the equivalent of an additional 300 GRPs on the Monday before Election Day, relative to Romney’s advertising—translated into an additional three-tenths of a point of vote share. The largest advantage that Romney had on the same day—the equivalent of 900 GRPS—translated into almost an additional point of vote share. Romney aired much more advertising than Obama in this last day of the race, so any advertising effect would tend to benefit him overall.
But ads aired before that Monday—even the calendar week before the week of the election—had a much smaller association with vote share. This is consistent with what Michael Franz noted and what previous studies have found: the apparent effects of ads simply do not last long. At one point, Obama advisor David Axelrod argued that early advertising was more important than late advertising, suggesting that Romney’s ad blitz, “back-loading,” was wrongheaded. Axelrod said: “By September, people are disregarding ads. They back-loaded. We front-loaded.” We find that back-loading—airing ads close to the election—was actually more effective than front-loading—airing ads early in the campaign—if the goal was to influence voters on Election Day.
Another thing constrained the effects of ads: it was hard to get a large advertising advantage. The 900-GRP advantage that was Romney’s largest was exceedingly rare close to the election. In fact, despite his late ad blitz, he had a 300-GRP advantage (or more) in only 15% of battleground state counties.
Did the ground game matter?
Yes, but to an extent far smaller than Obama’s winning margin. Other things equal, Obama’s vote share was about three-tenths of a point higher in counties where Obama had one field office and six-tenths of a point higher in counties where Obama had two or more field offices. (With relatively few counties having more than 2 offices, we did not try to estimate the effect of additional field offices beyond 2.) Romney’s field offices, by contrast, had an effect that was only half this size and could not be estimated with as much statistical confidence. This is consistent with the impression that Obama’s field operation was more effective than Romney’s. Essentially, our best guess is that Romney would have needed two offices in a county to match the effects of one of Obama’s offices, all else equal. (See Seth Masket’s post for a few more tidbits.)
We simulated a counterfactual in which Obama had no field offices but nothing else changed. In that scenario, we estimate that he would have lost about 248,000 votes nationwide. Given where those votes were located, we estimate that Obama would have lost Florida by a very narrow margin in this scenario. This estimate parallels estimates from the 2008 election. Joshua Darr, Luke Keele, and Matthew Levendusky have estimated that Obama’s 2008 field operation won him about 275,000 votes in total and could have been responsible for his victory in North Carolina. A similar analysis by political scientist Seth Masket found that Obama’s 2008 field operation may have been consequential in North Carolina, Florida, and Indiana.
Thus, Obama appeared to have an advantage on the ground, but not necessarily an advantage large enough to change many outcomes. We think this conclusion is consistent with how Obama advisor David Plouffe described a field operation in 2008: as a “field goal unit.” His comment suggests that field organization is valuable but not necessarily the game-winner unless the margin is close. In 2012, the election did not appear close enough for either the ads or field organization to be decisive.
What are the caveats in this analysis?
There are at least two. One is that, unlike Franz and Enos and Fowler, we did not attempt to leverage “spillover”—that is, the fact that media markets spill across state boundaries—to isolate the effects of ads (Franz: by examining non-battleground counties in a battleground media market) or the ground game (Enos and Fowler: by examining battleground counties and non-battleground counties in a battleground media market). So our inferences may depend even more on whether we have accounted for all other relevant factors that determine placement of ads and offices. We think we have done a reasonable job, but in the absence of a well-controlled experiment—which a presidential campaign never is—we cannot be certain.
Second, our measure of the ground game is rough. Field offices are far from the full story. Much organizing is done outside of field offices, such as in the “staging locations” that the Obama campaign used (although the presence of these locations is likely correlated with the presence of field offices). Moreover, like other analyses of field offices, ours assumes that the effect of a field office would be felt only in the county in which it’s located. But obviously, field organizing often crosses county and even state boundaries—for example, if people in Obama’s Montana offices were calling into Nevada or Colorado. To the extent that such activity was effective, we are underestimating the effect of field offices, although by how much it is hard to tell.
With that said, however, it is worth considering a thought experiment. In Ohio, Obama beat Romney by 166,272 votes. Assume, very generously, that the Obama field organization increased turnout by 9 points in 2012. Such an effect is at the upper bound of the effects of get-out-the-vote drives documented in the field experimental literature. (This would likely presume face-to-face contact between an Obama field staffer and each targeted voter, since other forms of contact are less effective.) To generate a winning margin of over 166,000 votes assuming a 9-point boost from voter mobilization efforts, the Obama campaign would have had to contact 1.8 million voters in this labor-intensive face-to-face fashion. That is a massive number, equivalent to a third of all of voters in Ohio in 2012. Perhaps that is possible: it would entail something like face-to-face contact with 11,300 people every day from June 1 until Election Day, under the assumption that the effectiveness of contact is constant across these months. I’m skeptical, but I am not privy to the Obama campaign’s internal data and cannot speak to this with certainty.
On the other hand, 1.8 million does seem like a far greater number than the Obama campaign would have ever intended to contact in the first place. Their overall strategy seemed to be base mobilization: Darr et al. show that as counties grew more Democratic, the chance of having an Obama field offices increased sharply (whereas the placement of Romney’s field offices was much less strongly related to a county’s partisanship). And thus the Obama campaign likely targeted people who were quite likely to vote Democratic but not necessarily likely to vote. This group of voters strikes me as a far smaller number than 1.8 million.
Moreover, the real effect of canvassing is probably not 9%. In this (gated) meta-analysis of field experiments by Donald Green, Mary McGrath, and Peter Aronow, they find that the average is about 2.5%. If that effect were true in 2012, obviously the Obama campaign would have had to contact a far greater number of people in order for his field organization to be the decisive factor in Ohio and elsewhere.
What’s the takeaway here?
In The Gamble, Lynn and I argue that electioneering in a presidential general election can matter. But that is different from saying that it always matters. In 2012, the electioneering was not pivotal because neither side had a large advantage in campaign resources and because Obama had a large enough advantage from the economic fundamentals. I think this is consistent with what Michael Franz, Ryan Enos, and Anthony Fowler wrote in their posts. The effects of electioneering were visible, but they were not large enough and lopsided enough to determine the winner. Of course, if one side had not campaigned for some inexplicable reason (or the economy had worsened during the election year) then the outcome might have changed.
In reality both sides spent lots of money and ran reasonably effective campaigns. It may seem strange to say that, given how much time and energy is spent critiquing the loser’s campaign and praising the winner’s. But when both sides have a billion-plus dollars to spend, each one’s campaign will often neutralize the other’s. One side may still be able to eke out a narrow advantage because of superior campaigning, but in a year like 2012, when the election was really not that close going into the campaign, that narrow advantage was something less than a “game-changer.”
[Cross-posted at The Monkey Cage]