Tag Archives: statistics

By James Kwak

Many people have spilled far more words on this topic than I can read, but I wanted to point out a few things that seem clear to me:

• As Daniel Engber pointed out, the fact that Obama won (and that Silver called all fifty states correctly) doesn’t prove that Silver is a genius any more than Obama’s losing would have proven that he was a fraud.
• In fact, Silver appears to have gotten a couple of Senate races wrong, but that still doesn’t prove anything, since his model spits out probabilities, not certainties.
• To my mind, the crux of the debate was between: (a) people who believe that it is meaningful to make probabilistic statements about the future based on existing data (both current polls and parameters estimated from historical data); and (b) people who believe that there is something ineffable about politics that escapes analysis and that therefore there is something fundamentally wrong, or misleading, or fraudulent about the statistical approach. Silver, through no fault of his own, because associated with (a). To my mind, (a) is right and (b) is wrong because of logic and math, so the idea that one election could have settled the question was crazy to begin with.
• Within camp (a), there are certainly valid methodological debates, and it’s by no means clear that Silver is the state of the art. Whether, in the last days of an election, he is any better than simple averages is an open question. The value Silver adds or doesn’t add can’t be judged by the final forecast, because one point of his model is to incorporate factors that are not incorporated in current polls (e.g., economic conditions). (Another aspect of the model is to not overreact to short-term trends—but that aspect also largely vanishes by the night before.) So the superiority of the  model, if it is superior, would appear months before the election, not the night before. But that is even harder to verify by ultimate results. Ideally you would have many elections and for each one you would have a Silver forecast six months before and a simple poll average six months before and you would see which had a higher batting average. I would bet on Silver, but we’ll never have enough data to resolve that question.

If the outcome makes people take statistics more seriously and pundits less seriously, that’s a good thing, but it’s not why you should take statistics more seriously.

By James Kwak

At least when it comes to statistical issues:

(Courtesy of Nate Silver.) Gallup is the huge outlier among the tracking polls, which shows Romney leading by 6–7 points. (On average, the national polls show an exactly tied race.)

This news is a few days old, but the general principle it illustrates is timeless. Reporting tends toward the dramatic and the surprising. In some cases, that’s probably fine—like if you read the paper for entertainment. When it comes to statistics that suffer from measurement error, it’s journalistic malpractice.

Yesterday Maine voted to restrict marriage to opposite-sex couples. It’s a sad day for people who believe that all couples who love each other should be allowed to marry, full stop.

But the chart below may cushion the blow a tiny bit. It’s from a paper by Jeffrey Lax and Justin Phillips, “Gay Rights in the States: Public Opinion and Policy Responsiveness,” recently published in the American Political Science Review (via The Monkey Cage). What you are seeing is support for same-sex marriage in 1994-96, 2003-04, and 2008-09; solid dots indicate that same-sex marriage is equal, hollow dots that it is not.

Continue reading →