Modeling Everything, Public Plan Edition

Ezra Klein and Paul Krugman are both highlighting Nate Silver’s analysis of campaign contributions and the public health plan option. The quick summary? Campaign contributions matter – in this case, by about nine senators. Mainly I’m impressed and encouraged that people can use publicly-available data to quickly whip together plausible models answering questions that otherwise we would all just pontificate about.

Coincidentally, I was getting my car inspected this morning and picked up an October 2008 copy of New York Magazine in the waiting room, which had an article about . . . Nate Silver. The article includes a picture of the presidential electoral map as Silver predicted on October 8, in which he called every state correctly except Missouri (which, remember, took a few weeks to figure out whom it had voted for). Most of the article is about how the empirical approach to baseball turns out to be useful in other areas, like politics and public policy.

Update: Mark Thoma points out this counterargument by Brendan Nyhan (who long ago wrote a blog with the brother of one of the best developers at my company). Nyhan says “studies have typically found minimal effects of campaign contributions on roll call votes in Congress,” and cites a Journal of Economic Perspectives paper as backup.

OK, Nyhan may be right. But he may not be.

I looked at that paper. First, it cites a stack of papers that support Silver’s view (that campaign contributions do influence policy). (Of course, it’s common to cite the papers you are trying to refute.) Then it describes a logical argument against Silver’s view (“Tullock’s Puzzle”), which makes no sense to me, at least as summarized there. The argument is that campaign contributions are pitifully small given the amounts of money at stake, and so firms cannot possibly see contributions as an investment in policy. For example:

Dairy producers, who since 1996 have had to have subsidies renewed annually, gave $1.3 million in 2000 and received price supports worth almost $1 billion in the Farm Security and Rural Investment Act of 2002.

I dont’ see the puzzle; if I can get $1 billion in subsidies by paying $1.3 million, why would I pay any more? It seems to me that the explanation here has to do with special-interest politics; no other constituency is sufficiently mobilized to fight against dairy producers, so they get their subsidy. Here’s the conceptual argument: “The figures above imply astronomically high rates of return on investments. In a normal market, with such high rates of return, existing donors should want to increase their contributions.” But this assumes that the “investment return” on campaign contributions is a smooth, monotonic (always increasing) function, which seems fundamentally at odds with the way Congress works. But as I said, maybe Tullock did a better job explaining his puzzle.

Finally, they do a regression of “roll call” votes in Congress against campaign contributions and find little influence. But this analysis has serious limitations in the present context.

First, the dependent variable is the aggregate rating of each legislator by the U.S. Chamber of Commerce. (They got similar results using other organizations.) That is, it’s an average of a large number of votes made in the course of a session on a large number of issues. So the finding is that corporate contributions only pull a legislator a couple of points toward the Chamber of Commerce’s positions overall; but that doesn’t mean that on a given issue, he might switch his vote because of one or two large campaign contributors from the affected industry.

Second, it ignores the complexities of the legislative process. On many key issues, there is no roll call. For example, whether the public plan even comes to a meaningful vote will depend on Harry Reid – who may not even want the public plan – and whether he thinks he has the votes. The power of some members of Congress goes well beyond their individual votes.

Third, the data are from 1978-1994. I’m not a student of American politics, but casually reading The New York Times indicates a few changes in politics since 1994: there is more money; there is more money that is not controlled by political parties (weakening bonds to party, which historically explained a lot of votes); and there is more money that gets spent directly on advertising and is not contributed to political campaigns. This last factor could cut either way, but I think it’s fair to assume that if a company gave you $50,000, they are probably also donating to soft-money groups that take the same positions.

So here we have: on the one hand, a quick-and-dirty model on this specific question by a guy without a Ph.D. in anything (I think), which correctly picks the positions of 87 out of 99 senators (but it’s possible that the model would have done just as well without campaign contributions); on the other hand, a guy with a Ph.D. in political science and a research fellowship in health policy at a very good university, citing a paper by three MIT professors that’s more or less on the same general topic, arguing that campaign contributions don’t affect policy.

By James Kwak

25 responses to “Modeling Everything, Public Plan Edition

  1. I am not an expert on statistics, but…

    Several of the claims are assuming causation, when all he has shown in correlation. In particular, “If we removed the lobbying money, X Senators would switch their votes” is not even remotely demonstrable from his analysis.

    The three independent variables — ideology, PAC money, state health care spending — are not obviously independent at all. I am not sure what effect (if any) this has on the results of a logistic regression, but a brief search suggests it does affect the reliability of the result.

    Finally, either I am missing something, or this statement is totally specious: “The model guessed the senator’s position correctly in 87 out of 99 instances.” Well, duh. The model was calibrated by the Senators’ positions. I could build a model that predicted all 99 Senators’ positions if you let me calibrate it to those positions first… That would not imply that the factors in my model meant anything at all.

    What would be interesting is if he calibrated the model on (say) 50 Senators chosen at random, and then tested it on the other 50.

    It is an interesting piece, and almost certainly getting at a kernel of truth. But Silver is overplaying his hand and throwing around statistical terms that I am not convinced he fully understands.

  2. Prediction and estimation of causality are two entirely different statistical exercises. Nate’s monte carlo simulations of state election outcomes were excellent. His model of campaign contributions was pretty lousy.

    1) It doesn’t account for reverse causality (aka, PACs contribute to senators who favor their positions in order to keep them in power and/or reward them). In other words, PAC receipts are a signal of senatorial alignment with an interest, NOT of senatorial MOVEMENT. Nate does use an ideology control (DW-nominate), but this does not address the issue.

    This is what Ansolabehere et. al. mean by “simultaneity”. They attempt to correct for this by using an instrumental variable (here, a basic two stage least squares model). Using such a model, they fail to find many statistically significant parameters, but I wouldn’t give this too much weight… 2SLS models are notoriously weak when the r-squared in the first stage is low (I didn’t catch them reporting this value) and when the instruments aren’t that powerful.

    2) Nate doesn’t account for importance of the senator. Notably, the two senators who are predicted by Nate’s model to be the most swayed by money are Max Baucus and Harry Reid… Who just happen to be the Chair of the Finance Committee and the Senate Majority Leader. So, Nate is predicting Baucus has a 0.7% chance of voting for the public option. Wow. That’s, uh, pretty low…

    Ansolabehere et. al. actually use importance of senator as one of their instruments (and closeness of senate race as another). I’m not sure this is a great specification, since it presumes senatorial importance (and closeness of race) have a fixed linear effect on spending. In actuality, one might imagine that an interaction term is required. That is to say, projected return on a lobbying “investment” ~ senatorial influence _times_ money spent.

    With regard to Tullock’s argument (Tullock was one of the granddaddies of the Public Choice school, which basically argued that politics is driven by economic self interest and people act in an economically rational manner in the political sphere – http://en.wikipedia.org/wiki/Public_Choice_Theory), the paradox is simple. Past data suggest that PACs were spending way too little money than was economically optimal, with little return (except in some key areas, such as agriculture).

    The argument may be slightly out of date in that campaign expenditures broke the long term trend relative to GDP, having grown MUCH faster in the last few election cycles than GDP. In that sense, Ansolabehere et. al.’s paper is dated. BUT, MOST OF THIS GROWTH came from INDIVIDUALS with ideological goals, NOT from economically self-interested PACs. Consider George Bush’s Texas Rangers, or the individuals who bankrolled the 509s, or Obama’s mass internet appeals…

    3) Nate’s paper leaves out countervailing lobbying efforts. He acknowledges this, but it’s not a trivial matter. It’s a big deal. Lobbying efforts almost always trigger opposition lobbying. The fastest way for the NRDC to raise funds is to send out an email arguing that the opposition is on the move! (I get them every couple weeks.)

    Of course, this issue cuts two ways… In one sense, it suggests that Nate’s simple model overestimates the net effect of lobbying. On the other hand, it suggests Nate’s simple model could underestimate the marginal effect (if PAC dollars were balanced by opposing PAC dollars, then the effect would have been even larger than the estimate if the opposing PAC dollars had not been spent).

    The issue of countervailing lobbying also wasn’t handled in the Ansolabehere et. al. paper. I can’t recall whether it was handled well in Steven Levitt’s 1994 paper. (Levitt’s basic argument was that high-quality candidates attracted more money, and they also tended to win; so regressing win-rate on campaign earnings yielded a spurious effect due to failure to control for candidate quality. And yes, Levitt is the guy who wrote Freakonomics.)

    But before you write off Ansolabehere et. al. as a bunch of apologists for the current political system (I assure you, they are not), do read their list of issues to explore. They note, for example, that:

    - PAC contributions may simply buy access, but the real money (10 times as much) is spent on the lobbying itself.

    - PACs may help coordinate individual donations (so that PAC donations are only the tip of an unmeasured iceburg)

    So the summary is: Nate’s model is repeating stuff that was done 30 years ago, and has since been pretty much proven irrelevant. (But his monte carlo election projections… top notch!)

  3. You know, I am very very pro education. My father was the first person in his family to get a college degree, and he got his Master’s in Education, in 1957, when having a college degree (even for a male) wasn’t so crucial. Everyone in my family has at least their Bachelor’s. And I took 2 University Math classes, at least 5 Econ classes, and 3 Statistics classes. I’ve done (many years ago) regressions and standard deviations etc. etc. And I know that’s nothing special to “brag” about. My point is, I have nothing against numbers and graphs and using them to analyze things. But anyone who thinks Nyhan’s assertion that campaign contributions have minimal effects is…… well I know this site has word police, so I will say—people who believe Nyhan’s assertion that campaign contributions have minimal affects are “NOT VERY SMART”. I would LOVE to see “Mike” over at the “Rortybomb” blog take a look at this and just destroy it. I would definitely get a sadistic thrill out of seeing Mike Rortybomb annihilate this assertion. And I KNOW he has the skills with numbers.
    Mr. Kwak, next time you see Brendan Nyhan or his brother, ask him if he’s ever heard of “THE KEATING FIVE”. It might be slightly instructional to him.
    Mr. Kwak, I don’t like this PARTICULAR journal paper, but I do really like when you put up links for the journal papers. They go more in depth and I can learn more things. I really learned a lot from that Gary Gorton journal paper. Like what a “haircut” means and many things I learned. So I want to say James Kwak is doing a great job, and hope we can get more finance/economics journal links. Great stuff.

  4. I am not going to comment too much on the statistics, as I want to make another point. However, statistics first.

    I did not buy Silver’s argument. For one thing, I do not know what DW-NOMINATE means. Modern political ideology is a hodge-podge. Both Reps and Dems are coalitions, but the Reps have better party discipline. He presents graphs of his models, but no data points. I don’t know what he means by liberal Dem, mainline Dem, and centrist Dem, nor what he means by centrist Rep. Since there are only 100 senators, he can’t have a whole lot of them in each classification. (Presumably the models are descriptive for the 80 some odd senators who have expressed their opinion on the public option.) I think that I see where he gets his causative explanation from. The contributions were presumably made without discussion of the specifics of the looming health care bill, so the contributors could not tailor the amount of their contribution by how the senator might vote on the public option. Therefore causality could not flow from the upcoming vote to the contribution. If the senator had a history of voting on similar proposals, then the contributors could predict from that the senator’s vote. However, there has not been a comprehensive health care reform to vote on since the early 1990s. Furthermore, if such a voting history existed, Silver could have (would have, he assumes) used it as a predictor. So if there is a causal connection, either both the contribution and the vote are caused by something else, or the contribution is one of the causes of the vote.

    Whew! That is more than I meant to say. ;)

    One more thing. If the other researchers are using standard, Fisherian statistical tests, the stats cannot indicate that contributions do not affect policy, they can only fail to show that they do.

  5. Now, for what I really want to talk about. The key fact of Tullock’s puzzle is that, if campaign contributions buy votes, the legislator is underpaid. By basic economic theory, the legislator should be able to get more money.

    There is a common sense reason for underpayment. Unlike in the 19th century, U. S. politicians today cannot be frank about their corruption. Patronage was big back then, and if you were not sufficiently corrupt, your patronage could not be counted on. Even fifty years ago, Harry Reid could have casually accepted cash in the hallways of the capitol and stuffed it in his pockets without anybody blinking an eye. These days that would be a no-no. Politicians cannot afford to accept sufficient pay for their votes, unless it is kept a secret. (One way to keep it a secret is to pay later. For instance, with a well paid position in industry after leaving the gov’t.)

    There is another reason, which may be termed the principle of underpayment. Politicians understand this principle quite well, which is why they employ so many volunteers. It is not just that volunteers are cheap — they may not receive money, but they do not receive nothing for their help — it is that through their volunteering efforts they become even more ardent supporters. That is, the underpayment of volunteers increases the value (for them) that they give to their volunteer work, and the value that they give to the politician.

    In psychology this goes under the name of cognitive dissonance. It may seem irrational from the standard economic perspective, but it is actually quite logical. The syllogism goes like this: They are not paying me enough to do what I did, unless I am really gung-ho for this politician. Therefore I must be really gung-ho for him (or her). Cognitive dissonance relies upon the fact that our preferences and values are not static, and that we continually rediscover and reinvent ourselves. We also want to be consistent.

    If you wish to corrupt a politician, and you pay him what he is worth, you run the real danger that he will turn on you for someone who pays him more. But if you pay him less than what he is worth, you may make a convert, who appreciates your contribution, but believes that he votes your way because it is the right thing to do. That will make it harder for someone else to buy his vote. Of course, you must pay enough that he is not insulted. That is why, as StatsGuy points out, you pay Harry Reid more than others.

  6. Also, I would like to ask: Does anyone on God’s green earth believe that the Insurance business and Pharmaceutical industry make campaign contributions to Congressman/Presidential candidates for “minimal effects”? No…Yes…?? Oh I forgot, the Insurance and Pharmaceutical industries just want to make sure government is run “efficiently”. Thanks for reminding us about how concerned the Insurance and Pharmaceutical companies are about clean government Mr. Nyhan. Thanks for reminding us Mr. Nyhan, how benevolent the Insurance and Pharmaceutical companies are. Otherwise we might think they just got some masochistic thrill out of throwing money at politicians.

  7. One more thing I can’t resist saying: I hope Mr. Brendan Nyhan would take the “extensive literature on this subject by political scientists and economists” and hurry over to K Street in Washington D.C. Hurry over to K Street as fast as you can Mr. Nyhan. All those people with extremely high salaries on K Street DON’T KNOW that they’re wasting their time.

  8. One more thing I can’t resist saying: I hope Mr. Brendan Nyhan would take the “extensive literature on this subject by political scientists and economists” and hurry over to K Street in Washington D.C. Hurry over to K Street as fast as you can Mr. Nyhan. All those people with extremely high salaries on K Street DON’T KNOW that they’re wasting their time.
    BTW I love your blog!

  9. PAC contributions and lobbying expenses are two different things. Lobbying consumes 10 to 20 times as much money as PAC contributions. So PAC contributions may buy access, but lobbying is where the real money is spent (and presumably the real influence occurs).

    K Street are the lobbyists.

    But thank you for raising issue #4 with Nate’s model: The model overestimates the effect of campaign contributions by PAC because it fails to control for lobbying effort/expenses, which are probably highly correlated with PAC contributions (but could be doing the real work).

    It’s also difficult, btw, to separate _implied_ threat from past donations/lobbying. A lot of lobbying by entities like AARP (probably the most powerful lobby on the Hill) is about _implied_ threats in future campaigns. They wield power through their ability to communicate to members. Likewise the AMA, for example.

    Shutting down campaign contributions isn’t the same as shutting down lobbying.

    But no one is saying campaign contributions don’t have an effect… just that the effect is not nearly as large as what Nate is estimating (particularly for senators like Baucus, for whom Nate sees a >60% shift in likelihood of support).

  10. Pingback: Business Items :: Health Care and Whimsy

  11. Not exactly true. If the study is designed to be of a sufficiently great statistical power (N sufficiently large, controls variables or stratification sufficiently well applied) it can rule out effects less than the minimum detectable effect (see Bloom, 1995).

    But I would like to add that there are papers that assess roll call votes and come up with positive results on campaign contributions, e.g. Brooks, Cameron, and Carter (1998), Gordon (2001), Fellowes and Wolf (2004). Most political scientists think the issue was settled in the 1980s because they were trained in the 1980s and stopped reading then.

    The advances in the literature owe to better measures of contributions and measurement. Silver does not add to the debate on measurement, and that is his real sin in this case.

  12. By less than an MDE, I mean greater than.

  13. Does lobbying money matter? Don’t know. Too bad Spirou Agnew isn’t still around. We could ask him.

    This entire discussion about whether or not K Street matters is kind of funny. Maybe I should be more diplomatic. Not funny, entertaining.

    When the vote’s going to be close moving one vote matters a great deal. And the “lobbying” money being spent might just be a nice job for a nephew.

    People arguing that all this lucre sloshing around D.C. has little effect are funny. Ooops. Entertaining.

  14. Campaign contributions and K Street are not different groups. Just different wallets in different pockets in the same pants.

    Following campaign contributions is a pretty good “tell” on where K Street’s going too.

  15. StatsGuy, you know I hesitate to quarrel with you because I agree with 99% of your posts, and I can’t find any falsehoods in your argument. But would you deny the very good chance, that these lobbyists are slipping an awful lot of money under the table (literally cash in envelopes, which is the way smart people do it because cash can’t be traced) to these Congressman?? Not to mention future jobs promised after they leave Congress, jobs to the 3 interns Congressman X is banging, etc….etc. I don’t see a lot of difference between the campaign contributions and the lobbying.

  16. Come now, this is 2009; complaining you don’t know what something is without having googled it is just plain lazy (it’s the #1 result when you just google “DW-NOMINATE”).

    http://voteview.com/dwnomin.htm

  17. I’m really not disagreeing with you – you are certainly correct that money is a signal for other types of influence, some of which are unmeasured and probably not ethical. (But then, it’s really these _other_ things that are driving influence, and are inflating the estimate on legal PAC contributions.)

    But Nate’s model is plainly bad. It is almost certainly massively overestimating the impact of PAC contributions. And it implies that eliminating PAC contributions would by itself massively swing senate votes in favor of a public option. (Seriously, a 60% projected swing for Baucus?)

    On the other hand, the Ansolabehere et. al. paper has some issues. At a “crude” level, the notion that campaign contributions are a consumption good does not stand well against the (so far journalistic) data suggesting a sharp increase in lobbying expenses resulting from the Stimulus Package.

    http://www.boston.com/business/articles/2009/06/09/dc_sees_sharp_spike_in_lobbying_costs_by_mass_firms/

    I also take issue with Ansolabehere et. al.’s argument that just because most political donations come from individuals indicates that those donations are a consumption good (e.g. an expression of ideology, rather than something intended to have an effect). That effect might be ALTRUISTIC, in that I do not anticipate capturing the benefits of my donation, but most donors surely intend those donations to have an impact on likelihood of a candidate to win.

    The Public Choice doctrine has a very narrow understanding of individual utility (it’s entirely economic), and thus gives a lot of weight to issues such as collective action. Public Choice theory, of course, has a really hard time explaining how Obama managed to pull in >300 million dollars from small/medium donors, many of whom (especially wealthier educated folks) stand to pay a higher share of taxes.

  18. Who is arguing that money has no impact? However, PAC money is a small fraction of the money being spent, and of the influence being peddled. I very much doubt that cutting out PAC money alone would have swing 9 votes in the Senate in favor of the public option. Nate’s model overestimates its impact.

    Having said that, I am rarely accused of being funny or entertaining. So, er, thanks.

  19. @peatey

    Thanks for the URL. Now could you please explain what DW-NOMINATE scores mean? Many thanks. :)

  20. Could some bitter soul please write an in-depth article with details on how to buy influence. Actual case-studies, please. Don’t need to name names. Disenfranchised campaign workers, please step up and give us an earful. Maybe this has been done already and we just need a retrospective?

  21. I think the intial point is the most important one. Some guy doings pretty basic statistics work with public data post an interesting model on his blog. Of course the standard list of shortcomings exist. I’m willing to bet he spent three days tops on a toy model which adds to the national debate. Nate Silver won’t be publishing in the Journal of Economic Perspectives any time soon.

    The beauty of the blog is you can post something unpolished but interesting for wider consumption. This could be theory, data or a simple model. Other can consume, discuss and expand on those ideas. Sometimes it is best to leave perfected papers for the AER. A second year grad student can tear almost any model to shreds, it takes a few more years to appreciate the insights of a simple model despite its limitation.

  22. maybe the research fellow is being lobbied too.

  23. This is a fantastically ridiculous debate. There are so many books which have been written which essentially prove the effects of campaign contributions, both hard and soft.

    I would point out that, according to polls, somewhere between 62% and 75% of both patients and physicians favor a single payer universal health plan, but it is hard to find it being seriously discussed anywhere on Capital Hill or in the popular media. The effect there, and in the financial markets is so patently obvious, and the “soft” disinformation programs are so strong, that you just can’t miss the effect.

    No one will ever convince me that these contributions are made on the basis the the contributor strongly feels that the receiving party is incredibly well disposed to govern with superior general effectiveness.

    All the more reasons to have publicly financed political campaigns!!!!

  24. No problems, :)

    The answer is in the #2 result in the google search:

    http://www.slate.com/id/2060047

  25. @peatey

    Thanks again. :)

    BTW, I usually use a different search engine, which did not have this URL in the first pages.