For a complete list of Beginners articles, see the Financial Crisis for Beginners page.
Joe Nocera has an article in today’s New York Times Magazine about Value at Risk (VaR), a risk management technique used by financial institutions to measure the risk of individual trading desks or aggregate portfolios. Like many Magazine articles, it is long on personalities (in this case Nassim Nicholas Taleb, one of the foremost critics of VaR) and history, and somewhat light on substance, so I thought it would be worth a lay explanation in my hopefully by-now-familiar Beginners style.
VaR is a way of measuring the likelihood that a portfolio will suffer a large loss in some period of time, or the maximum amount that you are likely to lose with some probability (say, 99%). It does this by: (1) looking at historical data about asset price changes and correlations; (2) using that data to estimate the probability distributions of those asset prices and correlations; and (3) using those estimated distributions to calculate the maximum amount you will lose 99% of the time. At a high level, Nocera’s conclusion is that VaR is a useful tool even though it doesn’t tell you what happens the other 1% of the time.
naked capitalism already has one withering critique of the article out. There, Yves Smith focuses on the assumption, mentioned but not explored by Nocera, that the events in question (changes in asset prices) are normally distributed. To summarize, for decades people have known that financial events are not normally distributed – they are characterized by both skew and kurtosis (see her post for charts). Kurtosis, or “fat tails,” means that extreme events are more likely than would be predicted by a normal distribution. Yet, Smith continues, VaR modelers continue to assume normal distributions (presumably because they have certain mathematical properties that make them easier to work with), which leads to results that are simply incorrect. It’s a good article, and you’ll probably learn something.
While Smith focuses on the problem of using the wrong mathematical tools, and Nocera mentions the problem of not using enough historical data – “All the triple-A-rated mortgage-backed securities churned out by Wall Street firms and that turned out to be little more than junk? VaR didn’t see the risk because it generally relied on a two-year data history” – I want to focus on another weakness of VaR: the fact that the real world changes.
Even leaving aside the question of which distribution (normal or otherwise) to use, VaR assumes the likelihood of future events is dictated by some distribution, and that that distribution can be estimated using past data. A simple example is a weighted coin that you find on the street. You flip it 1,000 times and it comes up heads 600 times, tails 400 times. You infer that it has a 60% likelihood of coming up heads; from that, you can calculate the probability distribution for how many heads will come up if you flip it 10 more times, and if you want to bet on those coin flips you can calculate your VaR. Your 60% is just an estimate – you don’t know that the true probability is 60% – but you can safely assume that the physical properties of the coin are not going to change, and you can use statistics to estimate how accurate your estimate is. But another way, your sample (the 1,000 test flips) is drawn from the same population as the thing you are trying to predict (the next 10 flips).
By contrast, imagine you have two basketball teams, the Bulls and the Knicks, who have played 1,000 games, and the Knicks have won 600. You follow the same methodology, bet a lot of money that the Knicks will win at least 5 of the next 10 games – and then the Bulls draft Michael Jordan. See the problem?
Now, are asset prices more like coin flips or like basketball times? On an empirical level, they may be more like coin flips; their probability distributions aren’t likely to change as dramatically as when the Bulls draft Jordan, or the Celtics trade for Kevin Garnett and Ray Allen. But on a fundamental level, they are more like basketball teams. The outcome of a coin flip is dictated by physical processes, governed by the laws of mechanics, that we know are going to operate the same way time after time. Asset prices, by contrast, are the product of individual decisions by thousands, millions, or even billions of people (when it comes to, say, wheat futures), and are affected as well by random shocks such as the weather. We have little idea what underlying mechanisms produce those prices, and all the simplifying assumptions we make (like rational profit-maximizing agents) are pure fiction. Whatever the underlying function for price changes is, if it winds up distributed in a manner similar to some mathematical function, it’s by accident; and more importantly, no one tells us when the function changes.
Going back to asset prices: To estimate the probability distribution of price changes, you need a sample that reflects your population of interest as closely as possible. Unfortunately, your sample can only be drawn from the past, and your population of interest is the future. So you really face two different risks. You face the risk that, in the current state of the world (assuming you can estimate that perfectly), an unlikely event will occur. You also face the risk that the state of the world will change. VaR, at best (assuming solutions to Smith’s criticisms), can quantify the first risk, not the second.
Let’s say you are just interested in your VaR for tomorrow. The chances that the real world will change significantly from today to tomorrow are small, but you still have the question of deciding how far back to draw your sample from. Is tomorrow’s behavior going to be most similar to the behavior over the last 30 days, the last 30 months, or the last 30 years? It depends on when the real world last changed – and you have no good way of knowing that (although there are statistical ways to guess). And when you try to look at your VaR for the next quarter, or year, you have the additional risk of the world changing under your feet.
To put it another way, what happened in the last two years? One explanation is that the models were intrinsically faulty (wrongly specified). One explanation is that the models didn’t go back far enough to incorporate data about steep falls in housing prices. And one explanation is that no amount of data would have helped, because the world changed.
I want to apply this thinking to a question that has annoyed me for years. You often hear personal finance types saying, “over every 30-year period, no matter what year you start in, stocks always outperform bonds.” Their data usually go back about 100 years. So this sounds like you have 70 data points (you don’t have the results for the last 30 starting years), right? Nope. If that were the case, you could start your 30-year period on every single trading day in those first 70 years, which would give you about 17,500 data points. Maybe you have 3 data points, because you have 3 non-overlapping (and hence arguably independent) 30-year periods. But this all assumes that during the 30 years starting right now, the stocks basketball team and the bonds basketball team have the same relative strengths that they did over the last 100 years, which is a big assumption. There are other reasons to believe stocks will have higher returns than bonds, but the fact that for ten years everyone has been assuming stocks must do better than bonds leads me to believe it may not happen this time – at least if you take, say, 2000 as your starting point. (I suppose I should mention that about 63% of my non-cash financial assets are in stocks, more if you include REITs.)
There was one part of Nocera’s article that I liked a lot:
At the height of the bubble, there was so much money to be made that any firm that pulled back because it was nervous about risk would forsake huge short-term gains and lose out to less cautious rivals. The fact that VaR didn’t measure the possibility of an extreme event was a blessing to the executives. It made black swans [unlikely events] all the easier to ignore. All the incentives — profits, compensation, glory, even job security — went in the direction of taking on more and more risk, even if you half suspected it would end badly. After all, it would end badly for everyone else too. As the former Citigroup chief executive Charles Prince famously put it, “As long as the music is playing, you’ve got to get up and dance.” Or, as John Maynard Keynes once wrote, a “sound banker” is one who, “when he is ruined, is ruined in a conventional and orthodox way.”
This, I think, is an accurate picture of what was going on. If you were a senior executive at an investment bank, even if you knew you were in a bubble that was going to collapse, it was still in your interests to play along, for at least two reasons: the enormity of the short-term compensation to be made outweighed the relatively paltry financial risk of being fired in a bust (given severance packages, and the fact that in a downturn all CEO compensation would plummet); and bucking the trend incurs resume risk in a way that playing along doesn’t.
If you were an individual trader, the incentives might have been the opposite: shorting the market was an opportunity to make a name for yourself and open your own hedge fund, while buying more mortgage-backed securities would just keep you in the same bonus tier as everyone else. But it’s the CEOs who called the shots, and their personal risk aversion was what mattered. Or, in the brilliant words of John Dizard (cited in the naked capitalism article):
A once-in-10-years-comet-wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses.
The Baseline Scenario 
VAR is scientism. Financial engineers want the public to think they are a bunch of Einsteins. What nonsense. Stock price changes are not like trying to measure Planck’s constant. Economists try to impress people by attempting to quantify that which is inherently unstable. Stock market returns are not like coin flips. Coin flips of a “fair coin” have no risk, just “uncertainty”. Risk is change.
@ Independent Accountant – “Coin toss is not risk, only uncertainty” – so what is risk if not for uncertainty? Suppose you had bet a million dollars on the outcome of a coin toss? Is that not risk?
Your second statement – “Risk is change” is also not devoid of pitfalls. The correct statement would be -risk is that change which affects you in a negative sense.
I believe it is time to incorporate more qualitative research into finance rather than merely quantitative research. The combination of the two could handle better the events that seems unlikely to happen. For example an investor could change the distribution of his/her portfolio depending on the season of the year, an expected change in politics, or so.
My favorite appraisal of Value at Risk comes from David Einhorn (President of Greenlight Capital, a hedgefund). He said that VaR “is like an airbag that works all the time, except when you have a car accident.” The quote can be found in its context here:
Click to access 2012.pdf
As Mr. Kwak points out, VaR has a number of mathematical inadequacies but its principle problem is its inability to to tell you what happens the other 1% of the time (the tails). But the fact that it was easy to use caused it to be excessively relied on by senior executives and regulators, and it ultimately led (ironically) to excessive risk taking.
Mathematical tools are nice, even inadequate ones, but one shouldn’t get excessively dependent on any one thing when trying to make forecasts or predictions. Asset bubbles always seem to be a product of a psychological positive feedback loop that leads to a herd mentality. There was plenty of evidence from the past that both the dotcom bubble and the housing bubble were going to bust eventually. But in both cases the herd mentality had caused otherwise rational people to conclude that this time was somehow different, that we were living in a new era where things always go up and never come down. Sometimes a little common sense trumps a somewhat sophisticated but highly flawed mathematical tool.
Just to clarify common misconceptions about VaR:
* VaR models do not typically assume normality except maybe for short-term (daily) market risk. Not credit risk, not oprisk
* VaR is typically not approached as above. It is typically one of: historical, monte carlo, or parametric. Or some hybrid of the two.
* Historical and monte carlo do not require a distributional assumption
* The point of the article, time varying distributions, is a really great point. So, we maybe can’t solve it. However, there are VaR models that can at least try to model conditional VaR or VaR based on conditional (time-varying volatility). So, again, the VaR framework is quite simple and allows for addressing many of this perceived shortcomings.
David Harper
Having begun trying to understand stock investing in 1968 and learning over the years how to buy high and sell low, the market has proven to me it is irrational. At the same time, it is the only game around. In my work life as a regional planner, by the late 1980’s I came to view all life ask risk management. Nassim Taleb’s books provided an important perspective which led me to be more conservative recently. I made a few small bets, which may pay off. In the 35 years of my career, those that took a business administration rather than a public administration track have spent, it seems, the majority of their time trying to make money with money. They followed the OPM dictum – you get wealth by using Other Peoples’ Money. I, on the other hand had to create value using OPP – Other Peoples’ People – e.g. networking among and between communities. In planning for the long term I had to use economic and demographic data, as well as pay attention to markets and the impact of changing technology and globalization. These discussions about formulas and theory consume brain cells, but the net effect of all that work was that it was an illusion. Quantification is ideal for the theoretical world. We will always be beginners at risk management, no matter how much experience we have. Numbers do numb. There’s always tomorrow to pay off the credit debt, providing the music does not end. As much as we may enjoy Ravel’s Bolero, it does end. I wish you all well trying to be risk managers. The meteor with our name on it could be flying though space at this moment, or we may step off the curb at the wrong moment. There are losses in life, just as there are gains. To gain some objectivity within our environment is a challenge, but perhaps the only defense. A fish may only recognize water once taken out of it. Experiencing other simultaneous mental environments, cultures – places where the history, logic and underpinning rules are different, can add to our ability to make better decisions.
Guys:
Another great blog. I really enjoy when you break down some of these fundamental concepts and explain it in ways that your audience can understand.
Articles of this quality which stimulate really good feedback from the rest of us. Thank you for stimulating our minds!
Financial risk (VaR) model flaws go much deeper than just the shape of asset price distributions because they rely on the Efficient Market Hypothesis (EMH) to model markets as efficient, always in equilibrium and self correcting where security prices simply react to random news releases. The EMH models day-to-day security price movements as “independent random variables” where purchase or sale is a “zero net present value transaction,” resulting in neither the buyer nor seller having an advantage. Consequently, security trading is modeled like casino gambling where price volatility determines risk assessment. The EMH concludes that because prices always fairly reflect intrinsic value and it is impossible to know what markets are going to do, consequently, fundamental analysis isn’t cost effective. The EMH model is both naïve and specious, i.e., it relies on incorrect premises and, therefore, cannot correctly model market risk. A discussion follows.
Markets are not always in equilibrium nor self correcting but rather are a discounting mechanism, i.e., professional traders look ahead and bid prices either up or down prior to earnings and/or economic news announcements; that is why prices can go up on bad news and down on good news. Security prices from day-to-day seem random, however, portfolio diversification cancels out unsystematic risk, therefore, markets as a whole only have systematic risk. Using a diversified market portfolio, monthly rather than daily data, trend lines and conditional probabilities based on fundamental analysis correctly models systemic market risk; thereby disqualifying the flawed EMH’s reliance on securities’ price frequency and the use of price volatility as a proxy for portfolio risk which is not sufficient information when hedging positions.
(Eric L. Prentis is the author of “The Astute Investor” and “The Astute Speculator,” http://www.theastuteinvestor.net )
“@ Independent Accountant – “Coin toss is not risk, only uncertainty” – so what is risk if not for uncertainty? Suppose you had bet a million dollars on the outcome of a coin toss? Is that not risk?”
@Greg — No, it’s not. In common speech, “risk” has the meaning you ascribe — an event whose outcome is as yet unknown but may be negative. Risk v. Uncertainty in an investing or economics context have distinct meanings.
A risk, investing wise, is the known probability of an event, particularly an undesired event. In the coin toss example, there are only two possible outcomes, heads or tails. Whichever result I bet on, there is a 50/50 chance I might be wrong — that’s a risk.
The bad thing could happen but I know the chances of it happenning, and I can plan around them, and more importantly in an investing context, I can put a price on how much I’d be willing to pay for taking on a certain amount of risk. An investor with a $1 billion to invest, might be willing to risk $1 million on a coin flip, as that sum is only .1% of their total assets. And investor with $2 million would probably not be willing to take on such a risk.
Uncertainty, however, that’s a whole ‘nother thing. Uncertainty is a situation where the likelyhood of a given outcome is unknown and cannot be determined. That’s more like betting a million on whether or not humans will ever find intelligent life on another planet. We just don’t know enough about what the rest of the universe is like to figure out precisely what the true chances are of such a thing happening.
These two things can shade into each other — For instance a million-dollar bet on whether or not a woman will ever become president of the United States. In the past hundred years, it would seem clear that the chances of such a thing have clearly changed, and we might be able to quantify them somewhat by looking at other countries where there have been female heads of state, the circumstances surrounding their elections, the demographics and political systems of the societies involved, etc. But there’s still a lot of unknown unknowns there, a lot of uncertainty.
The real problem is when you think you’re dealing with risk and it turns out you’re dealing with uncertainty….
I’m glad this discussion has continued. I have been involved in trend analysis for planning over 35 years. In no circumstance would two years of demographic, economic or environmental data give a usable trend. In that time frame it would barely be possible to see a spike. For traders, small moves, can generate opportunity for profit, but that alone should not validate short term data sets The fact that everyone is doing it, sub-prime lending in this case, increased the overall magnitude of risk in the system and made it possible for the triggering event to be smaller. The down feather of a Black Swan set off the dominoes. I believe it was the Fed fighting inflation and increasing the bank rate that triggered the ARMs and tipped so many households into foreclosure. The economy had not grown enough for those individuals to cover the risk they were allowed to take/encouraged to take, particularly if they’d taken an ARM over a fixed rate – making more money for the broker. Increasing foreclosures were seen by those in the trenches on Wall Street, communicated to Jim Cramer and, eventually, led to his rant. The seriousness of a family not being able to meet the mortgage payment, can’t really be reflected in numbers and obviously wasn’t when “the brains” were doing their math. If those that created the formulas never had the experience – they could not relate to it. Probability has to be related to the current environment and the risk formulas can’t handle that. In the present, the probability of there being a female president of the U.S. is very high, as is the probability of there being males other than white. Gender or race will not disqualify competent individuals. As for technologically advanced intelligent life, the probability may be of them finding us. We’ve sent out signals, but would they respond without careful investigation of this noisy planet? As irrational as we are, we might be unwelcome in the neighborhood. Finance apparently has lost its sense of the long term. Are we at a point where there is only trading, no investing? When any asset can hold within itself a hot potato, how will trust be restored? Making money with money has hit a wall.