Why Do People Think the Race Is a Tossup?

By James Kwak

There’s been a minor controversy in the blogosphere not about whether Obama or Romney should be president, and not about whether Obama or Romney is ahead in the polls, but about the esoteric question of whether one should interpret the polls to mean that Obama is the favorite or that the race is a “tossup.” This debate has largely swirled around Nate Silver, who aggregates polling data, recalculates confidence intervals, and incorporates other factors (drawn from analysis of previous elections), and for the past few weeks has rated Obama as having about a 60–80% chance of winning the election. In response, various members of the pundit class have argued that the national polls show a tied race, polls can’t predict the future, or even that since both sides (supposedly) think each has a 50.1 percent chance of winning, their chances must be equal. (See Felix Salmon for a summary.)

Silver has responded to all of the coherent objections that might be made to his forecast, in detail. But what’s at work here isn’t a reasoned debate about how to interpret polls. It’s sheer innumeracy, pure and simple. The statement that Obama has about a 75–80 percent chance of winning is roughly equivalent to the statement—which no one contests—that his average lead in Ohio is about 2–3 points, once you take the confidence interval into account. As Silver has said, it’s analogous to the statement that a team that’s ahead by a field goal deep in the fourth quarter has a better chance of winning than the team that’s behind; no one would call that game a “tossup,” even though either team could win. Even if you can’t predict the next turnover or breakaway running play, that wouldn’t lead you to believe the three-point lead is irrelevant.

It’s the same thing we saw in Moneyball—people who can’t understand numbers claiming that numbers have no practical value. Unfortunately, in political journalism the sample size is so small and the monetary stakes are so low that the incoherent innumerates will never be drummed out of the marketplace.

13 thoughts on “Why Do People Think the Race Is a Tossup?”

  1. If you’re listening to someone other than Nate Silver’s 538 blog, you’re realizing an inferior product.

    One week from today, we’ll have joy in just how perspicacious this dude has been all along.

    Obama Biden wins.

  4. The Republicans according to Pacifica News control the govornor and secratary of state offices in 8 swing states. If you believe Bush beat Gore and that he beat Kerry in Ohio than this may sound like a conspiracy theory . But then again those are key positions.

  5. I have been involved with consumer behavior research for over a quarter of a century. That includes political polling. (In fact, I created exit polling for a Congressman named Alfonso Bell.) The thing that strikes me most about all of these current polls is just how much about how they were done that can make a huge difference in the results. Here are a few examples.

    1) How well trained are the interviewers? A couple of week ago, some girl called and she could barely read the questions off the questionnaire.

    2) How are the interviewers paid? If they are paid by the completed interview, there is a huge incentive to fill in all the questions, and that is related to #3.

    3) How long is the questionnaire? Every additional question increases the likelihood of a hang up and a partial interview. See #2.

    4) How skilled is the questionnaire writer? I have had recent surveys that included questions that were so poorly written that they just embarrassed me.

    5) How was the sample drawn? A long time ago, we could draw geographically valid samples and send interviewers out to interview people in their homes. Now interviews (at least the valid ones) are conducted by telephone. So what is the sample? Land lines? cell phones? In what proportions? What time of the day?

    6) What is the sample size? Generally, a larger sample size will be more likely to be accurate than a smaller one. However, everyone lives with a budget. If you have to have weekly polls, your budget demands smaller sample sizes.

    7) What is the purpose of your survey? Is it to predict the final outcome or is it to entertain the media? Our original exit poll was designed to entertain the media. If it is to predict the outcome of the election with precision, then any money spent in California or Texas is utterly wasted.

    I could go on but I am sure you are bored by now. The point is that nobody can evaluate the meaning of any poll without knowing the the details of how it was conducted. Which brings me to that charlatan Nate Silver. He aggregates garbage, does some phony number crunching and captivates the media by producing garbage.

    Remember that old say from the early days of computers, GIGO?

    Yep, that is what you get from Nate. Garbage In, Garbage Out.

    And finally, this is only a personal, arms length opinion, but the only poll I trust is the Pew polls. They show every characteristic of having been done my serious professionals.

  7. Actually Peter, the only conclusion I have reached is that most political polls can’t be trusted. However, I could make an argument that there is the one sided bias you suggest, and it goes like this. One party’s supporters are older, have land lines and are easier to reach. The other party’s supporters are younger, use cell phones exclusively and are harder, e.g., more expensive, to reach. Since almost all of the media polls are closely budgeted, I would suggest that there is possibility that one kind of voter could be over sampled, and estimates constructed to make up for the missing interviews with all kinds of errors possible. I also read the other day that completion rates on political polls can be as low as 9%!! In that case, 91% of the original sample is never interviewed. Re-read the comments above about budgets and earlier remarks about completed interviews.

    I can tell you from experience that the actual work involved here is always under time and budget pressures, and it is all conducted on top of a slippery slope, often with semi-clean hands. It is not done by detached political scientists sitting around a computer getting all the math just right.

    I have absolutely no knowledge that any of this is true, but it could be! And that is why I suggest you know a whole lot more about the conduct of the poll before you trust it. And Nate knows nothing about his material.

  8. Nobody’s mentioned some sources of asymmetric bias…

    * “Surprise” disenfranchisement — a voter who counts themselves likely to vote but gets stopped by ID issues or aggressive Repub challenges. This happens to a lot more Dems than Repubs.

    * Last-minute Repub ad blitz. The Repubs have more money at this point and it’s pretty obvious where to apply it. Ad blitzes work — it did in 2000.

    * Voting machine fraud — the Salon article on Ohio makes it pretty clear that something strange is happening with Ohio’s voting machines. There are already many credible reports from the last two or three presidential elections. If you believe that Romney and the Repubs have any moral qualms about doing this if they can get away with it, then I gotta bridge to sell ya.

    So maybe the folks at InTrade “know” more than Nate Silver about real-world elections, beyond abstract poll numbers.

  9. Every one of David Lewis’s points are spot on!! However, they are all beyond the control of any researcher. Remember all those folks in Chicago who used to vote from the cemetery? They would have been really hard to interview, pre or post.

  11. I’m not sure any of my points is beyond the reach of research. They are, however, almost beyond the reach of repair or reform, and if the Repubs win the presidency, they probably will be beyond that for the forseeable future. In fact, when the Repubs gain control of the presidency, house, senate and supreme court (2 down, 2 to go) I think we will see permanent one-party rule — as opposed to our current 1.5-party rule. governorships and state legislatures will follow in quick succession, and it won’t be long until it works its way down to the local level.

    What happens after that depends on the recipe for boiled frog.

  13. An astonishing win for President Obama, in light of what he was facing.

    For 4 years, the object of the exercise for DC republicans was to do NOTHING in aid of the president, and thereby block re-election.

    The entire strategy was proven deficient, in my estimation.

Comments are closed.