Tag Archives: journalism

Maybe Nate Silver Was Wrong

By James Kwak

I think Nate Silver does a good job aggregating polls to make meaningful quantitative predictions about upcoming elections. But as he said himself shortly before the election, if the polls he relies on are systematically biased, then his forecasts are going to be off.* Many people have noted that Silver (and other quantitative poll aggregators like Sam Wang) correctly predicted an Obama victory and the outcomes in most if not all states.

But the fact remains that Obama did modestly better than the polls, and hence the poll aggregators, expected (not to mention than the Romney campaign expected). We shouldn’t read too much into this, as even where Obama significantly overperformed—like in Iowa, where Silver forecast a 3.2 percentage point victory and the actual came in at 5.7 points—the results were within the confidence intervals. But it’s also possible that the polls really were systematically biased, only they were biased against Obama—not against Romney, as conservative pundits were claiming in the last days.

Continue reading

Why Do People Think the Race Is a Tossup?

By James Kwak

There’s been a minor controversy in the blogosphere not about whether Obama or Romney should be president, and not about whether Obama or Romney is ahead in the polls, but about the esoteric question of whether one should interpret the polls to mean that Obama is the favorite or that the race is a “tossup.” This debate has largely swirled around Nate Silver, who aggregates polling data, recalculates confidence intervals, and incorporates other factors (drawn from analysis of previous elections), and for the past few weeks has rated Obama as having about a 60–80% chance of winning the election. In response, various members of the pundit class have argued that the national polls show a tied race, polls can’t predict the future, or even that since both sides (supposedly) think each has a 50.1 percent chance of winning, their chances must be equal. (See Felix Salmon for a summary.)

Silver has responded to all of the coherent objections that might be made to his forecast, in detail. But what’s at work here isn’t a reasoned debate about how to interpret polls. It’s sheer innumeracy, pure and simple. The statement that Obama has about a 75–80 percent chance of winning is roughly equivalent to the statement—which no one contests—that his average lead in Ohio is about 2–3 points, once you take the confidence interval into account. As Silver has said, it’s analogous to the statement that a team that’s ahead by a field goal deep in the fourth quarter has a better chance of winning than the team that’s behind; no one would call that game a “tossup,” even though either team could win. Even if you can’t predict the next turnover or breakaway running play, that wouldn’t lead you to believe the three-point lead is irrelevant.

It’s the same thing we saw in Moneyball—people who can’t understand numbers claiming that numbers have no practical value. Unfortunately, in political journalism the sample size is so small and the monetary stakes are so low that the incoherent innumerates will never be drummed out of the marketplace.