By now everyone knows about this past year’s Wall Street bonuses: $18.4 billion total, the fifth-highest total ever; the $4 billion in bonuses rushed through by Merrill Lynch before its acquisition by Bank of America; and John Thain’s demand for a personal $10 million bonus (which was initially a demand for $30-40 million, according to Felix Salmon). This has, not surprisingly, unleashed a torrent of rage against Wall Street, up to and including Barack Obama, who called the bonuses “shameful.”

The usual defense of this sort of behavior is that you have to pay the market price for talent, the bonuses for top people are only a small fraction of the value they contribute (not a particularly good argument this year), and so on. And this is, not surprisingly, what John Thain was able to muster up in his defense on CNBC:

If you don’t pay your best people, you will destroy your franchise. Those best people can get jobs other places, they will leave. . . . you have to– pay market prices at the time.

Yes, there is a market for labor, and compensation is the price set by that market. And maybe it’s even a free market. But it’s certainly not a well-functioning market (one where price = marginal cost, for example, or where the surplus is divided between the parties, or where the right incentives are created).

Lucian Bebchuk and Jesse Fried have a basic overview of some of the problems with the market for executives (in which the price is executive compensation) in “Executive Compensation as an Agency Problem,” which deals primarily with CEO compensation. The basic problem is the old principal-agent problem: how do you get an agent to act on behalf of his principals, instead of looting them for his own gain? Bebchuk and Fried spend most of the paper criticizing an “optimal contracting” model, in which “boards [of directors] are assumed to design compensation schemes to provide managers with efficient incentives to maximize shareholder value,” arguing instead for their preferred “managerial power” model, in which managers use their power over the board to maximize their own compensation while simultaneously weakening its links to their performance and making it as hard to understand as possible, in order to minimize shareholder outrage. Having observed the way CEOs get selected and compensated, and having read Rakesh Khurana’s book on CEO searches, and most importantly having a pulse, I’m surprised there is even a debate about this, but the paper is from 2003, so maybe the debate is over by now.

According to Bebchuk and Fried, the basic dynamic at work is that directors like being on boards (it’s a lot of money for not much work, and it’s prestigious), CEOs control who is on the board of directors, CEOs control the information that goes to boards, and board members have weak incentives to act on behalf of the shareholders (they generally don’t own much stock). The only real checks on CEO pay are public outrage (hence the usage of hard-to-understand things like deferred compensation and pension benefits) and large and powerful shareholders. This leads to certain outcomes that are hard to justify on the theory that boards are negotiating in the interests of the shareholders, most strikingly the tendency to give large, gratuitous “goodbye payments” on top of already-generous negotiated severance packages. Note that John Thain’s demand for a bonus was only withdrawn after it was leaked to the Wall Street Journal (cue the public outrage).

Now, this paper primarily applies to compensation of CEOs (and their close friends, whom the CEO can take care of). But a similar problem applies to all Wall Street compensation. Just like CEO compensation depends on the myth that there is a small group of people with the ability to be CEOs, Wall Street compensation depends on the myth that there is a small group of people with the ability to work on Wall Street. (A myth that is pretty well belied by the fact that every year a flood of college and business-school graduates whose only common trait is that they all want to make money comes to Wall Street, and during the boom they all made lots of money.) That compensation is set by top executives and approved by the board, all of whom are bought into the myth of their own uniqueness; the shareholder, be he a teacher on Main Street or a mutual fund manager in Greenwich, doesn’t have a seat at that table. Put another way, compensation should theoretically be determined by the owner of the company – the person who gets the profits after salaries and bonuses are paid – but that person has been cut out of the negotiation by the weakness of our corpoorate governance practices.

Theoretically the market for labor could be what forces prices up; if one company paid below-market bonuses, the story goes, its top people would defect for competitors. But there are problems with this argument. First, all that means is that you have a market failure: when you have a small number of players, it’s easier and cozier for everyone to continue paying the same large bonuses (at the shareholders’ expense) than to pay the level a free market would ordinarily dictate. Second, what would be wrong with top people defecting? Wall Street’s most prestigious investment bank, Goldman Sachs, is also the one that was least willing to hire from the outside and most likely to promote from within – which is one way of saying that you think that people are overpriced on the open market. Third, if bonuses are a function of the threat of people leaving, why are bonuses this year (when there are no job opportunities) the same level as in 2004 (when they were plentiful)?

Weak shareholder control over executive compensation is, of course, common to all industries. The big difference is that while the CEO of Tyson Chicken (for example) doesn’t devote much energy to enriching the people who work on his chicken farms, the CEO of Merrill Lynch did devote energy to enriching the people on his trading desks. While most companies are run for the benefit of a few senior executives, Wall Street firms are unusual in that they are run for the benefit of a large class of professionals. It’s almost a form of sharing the wealth. Except this year there wasn’t much wealth, and what little there was arguably wasn’t theirs to share in the first place.

Update: Bebchuk has an article in the Wall Street Journal arguing that the executive cap limits announced this week aren’t strict enough.

Update: Lloyd Blankfein, CEO of Goldman, has an op-ed in the FT recommending restrictions that go beyond what the Treasury Department proposed, including long-term vesting schedules and addition limits on when shares can be sold. A friend of mine who used to work at Goldman tells me that Goldman already has some of these vesting and delivery restrictions in place.

• Commentary