This guest post is contributed by Ricardo Fernholz, a professor of economics at Claremont McKenna College. Some of his other work was profiled on this blog here.
The rise of high-frequency trading (HFT) in the U.S. and around the world has been rapid and well-documented in the media. According to a report by the Bank of England, by 2010 HFT accounted for 70% of all trading volume in US equities and 30-40% of all trading volume in European equities. This rapid rise in volume has been accompanied by extraordinary performance among some prominent hedge funds that use these trading techniques. A 2010 report from Barron’s, for example, estimates that Renaissance Technology’s Medallion hedge fund – a quantitative HFT fund – achieved a 62.8% annual compound return in the three years prior to the report.
Despite the growing presence of HFT, little is known about how such trading strategies work and why some appear to consistently achieve high returns. The purpose of this post is to shed some light on these questions and discuss some of the possible implications of the rapid spread of HFT. Although much attention has been given to the potentially destabilizing effects of HFT, the focus here instead is on the basic theory behind such strategies and their implications for the efficiency of markets. How are some HFT funds such as Medallion apparently able to consistently achieve high returns? It is natural to suspect that such excellent performance is perhaps an anomaly or simply the result of taking significant risks that are somehow hidden or obscured. Indeed, this is surely the case sometimes. However, it turns out that there are good reasons to believe that many HFT strategies are in fact able to consistently earn these high returns without being exposed to major risks.
To understand how this works, let’s consider the S&P 500 U.S. stock index. Suppose that we wish to invest some money in S&P 500 stocks for one year. Currently, Apple has a total market capitalization of roughly $500 billion, making it the largest stock in the S&P 500 and equal to approximately 4% of the total capitalization of the entire index. Suppose that we believe it is very unlikely or impossible that either Apple or any other corporation’s capitalization will be equal to more than 99% of the total S&P 500 capitalization for this entire year during which we plan to invest. As long as this turns out to be true, then it is actually pretty simple to construct a portfolio containing S&P 500 stocks that is guaranteed to outperform the S&P 500 index over the course of the year and that has a limited downside relative to this index. In essence, we can construct a portfolio that will never fall below the value of the S&P 500 index by more than, say, 5% and that is guaranteed to achieve a higher value than the S&P 500 index by the end of the year.[1]
This is not a trivial proposition. If we combine a long position in this outperforming portfolio together with a short position in the S&P 500 index, then we have a trading strategy that requires no initial investment, has a limited downside, and is guaranteed to produce positive wealth by the end of the year. According to standard financial theory, this should not be possible.[2] Furthermore, the assumptions that guarantee that our portfolio will outperform the S&P 500 index appear entirely reasonable. After all, not for one day in the more than 50-year history of the S&P 500 has one corporation’s market capitalization come anywhere close to equaling even 50% of the total capitalization of the market. A 99% share of total market capitalization would essentially amount to there being only one corporation in the entire U. S. for an entire year. This seems like neither a likely outcome nor one that investors should take seriously when constructing their portfolios.
What does a portfolio made up of S&P 500 stocks that is guaranteed to outperform the S&P 500 index look like? There are many different ways in which such a portfolio can be constructed, but one feature common to all such portfolios is that relative to the S&P 500 index itself, they place more weight on those stocks with small total market capitalizations and less weight on those stocks with large total market capitalizations. The weight that an index such as the S&P 500 places on each individual stock is equal to the ratio of that stock’s total market capitalization relative to all stocks’ total market capitalizations taken together. In the case of Apple, then, the S&P 500 index would place a weight of roughly 4% in this individual stock while those portfolios that use HFT to outperform this index would instead place a weight of less than 4% in Apple stock.
The second key feature of these outperforming portfolios is that they must be constantly rebalanced to maintain the chosen weights for each stock. This implies that any time the price of one stock increases relative to the price of other stocks, some of this stock must immediately be sold to maintain that stock’s prescribed weight. This constant rebalancing is what makes these trading strategies part of HFT. Consider, for example, an equal-weighted portfolio that invests the same dollar amount in each of the 500 stocks that make up the S&P 500 index.[3] If the price of one of those stocks increases relative to the others, it is necessary to sell some of that stock and purchase a small amount of the other 499 stocks in order to rebalance the portfolio and maintain the equal weights for all stocks. Trading frequently in order to rebalance the portfolio in this way plays a crucial role in outperforming the market.
The two portfolio characteristics described above – more weight on smaller stocks and high-frequency rebalancing in order to maintain those weights – are not particularly complex and do not rely on information that is not readily available to the public. Of course, the true strategies behind most HFT are more sophisticated and must address real-world issues such as trading costs.[4] Despite its simplicity, however, this discussion describes a valid method of investing that achieves very high returns without major exposure to risk. This point is clearly demonstrated in Figure 1. The figure plots the log return of a portfolio that combines long positions in stocks that decrease in value with short positions in stocks that increase in value and that is rebalanced every minute and a half, much like in the previous discussion.[5]Based on a simulation that uses data from U.S. stock prices in 2005, this portfolio earns a compound return of more than 100% over the course of the year.Surely, the ability of HFT strategies to achieve high returns by exploiting the relative movement that is natural among stock prices in this way explains much of both the rapid spread of HFT and the consistent success of prominent HFT funds such as Medallion. Barring a significant change in financial regulation, then, there is little reason to think that the spread of HFT will reverse itself.
Figure 1: The compound return of a portfolio that combines long positions in stocks that decrease in value with short positions in stocks that increase in value and that is rebalanced every minute and a half.
Nothing about our discussion of HFT strategies and high returns is inconsistent with a market that is efficient in the sense that stock prices reflect the public’s full knowledge about fundamentals. According to standard financial theory, anytime an individual stock price does not reflect that stock’s fundamentals, rational investors will buy or sell that stock and earn high returns until the stock price shifts to a value that does reflect fundamentals so that such returns are no longer possible. The HFT strategies that achieve high returns with limited risk, however, do not rely on deviations between prices and fundamentals. Indeed, as long as fundamentals are such that no one stock dominates the entire market for a full year, investors are able to consistently earn these returns without any knowledge of fundamentals and their deviations from prices. In this case, there is neither a contradiction between market efficiency and the ability of HFT to consistently outperform the market nor is there necessarily anything about HFT that makes markets more efficient.
The fact that HFT represents a highly effective investment strategy that has little to do with market fundamentals raises several challenging questions. How is it possible for HFT to consistently earn high returns with limited risk in an efficient market? What happens as more and more people pursue these high returns with HFT? How does HFT on a large scale like this affect stock prices? These are difficult “general equilibrium” questions that have yet to be answered satisfactorily by financial economists. There is little doubt, however, that a deeper understanding of these issues is likely to yield important theoretical and practical insights about the true workings of financial markets.
Footnotes
1. For these portfolios, there is a tradeoff between the extent to which the portfolio will outperform the index, the length of time before this outperformance is to occur, the maximum possible underperformance during this period, and the upper bound on the relative capitalization of the largest stock in the index. See Fernholz, Karatzas, and Kardaras (2005) for details.
2. A discussion of some of these “equilibrium’’ issues is provided by Karatzas and Kardaras (2007).
3. In fact, this simple equal-weighted portfolio is, under slightly stronger assumptions, guaranteed to outperform the S&P 500 over a sufficiently long time period.
4. For example, high-frequency traders are often broker/dealers in order to reduce potentially significant trading costs.
5. In addition to combining long and short positions in this way, the portfolio also closes all of its outstanding positions at the end of each day. This helps to reduce the volatility of the portfolio’s return. For a more detailed description of how this portfolio works, see Fernholz and Maguire (2007).
References
Fernholz, R., I. Karatzas, and C. Kardaras (2005, January). Diversity and relative arbitrage in equity markets. Finance and Stochastics 9(1), 1-27.
Fernholz, R. and C. Maguire, Jr. (2007, September/October). The statistics of statistical arbitrage. Financial Analytics Journal 63(5), 46-52.
Karatzas, I. and C. Kardaras (2007, October). The numeraire portfolio in semimartingale financial models. Finance and Stochastics 11(4), 447-493
The Baseline Scenario 
Mr. Kwak says, “Although much attention has been given to the potentially destabilizing effects of HFT, the focus here instead is on the basic theory behind such strategies and their implications for the efficiency of markets”.
Kwak seems to be suggesting that efficient markets exist, or if they don’t exist, efficient markets are a policy objective. HIs assertion denies reality. Too Big To Fail banks, and the gov’t backstop which makes them possible, are the antithesis of an efficient market economy. For all intents and purposes, a market economy not longer exists, let alone efficient markets. But this doesn’t affect Kwak’s endless supply of meaningless commentary, for which there is no demand. This unlimited supply of meaningless babble coming from the intelligentsia, is yet another example of the absence of a market economy.
http://www.c-spanvideo.org/clip/3343308
http://www.c-spanvideo.org/clip/3343218
So the stock goes up, you sell. And HFT guarantees you lock in the higher price. But that is only because HFT is an elite technique not available to most investors, who could never rebalance on an hourly basis (let alone on a 90-second cycle). For those investors, even before accounting for “trading costs,” a small increase in a stock’s price cannot necessarily be locked in by entering a sell order, because of the time lag associated with non HFT accounts.
This time lag does not have to be caused by HFT for the HFTers to be, in effect, cheating.
HFT is a market technique that is in my view completely antithetical to the efficient markets hypothesis because it exploits gaps in knowledge flow that facilitate proper price discovery. The idea that price discovery processes converge within seconds of their initiation is ludicrous for trades of instruments backed by assets that have some sort of utility in the real (and slower-acting) world. Only the most sophisticated trading houses can engage in such activity, and perhaps shouldn’t, if they believe that their purpose is to properly distribute resources to human endeavors. Oh, wait a minute, they don’t.
Thanks for presenting this without the normal good/bad blather. Quite interesting and thought provoking.
Shouldn’t any “thought provoking” discussion of efficient markets begin with the Fed’s QE to infinity? KC Fed President Tom Hoenig posed the following question regarding capital markets. What resource has ever been distributed efficiently when the price is zero?
Oregano said, “Only the most sophisticated trading houses can engage in such activity, and perhaps shouldn’t, if they believe that their purpose is to properly distribute resources to human endeavors”.
Proper distribution of resources is not the purpose of trading houses. They’re part of larger markets whose purported objective is the efficient distribution of resources. Efficient markets are based on supply/demand fundamentals and regulations that force trading houses operate within moral and legal parameters.
Efficient markets breakdown when regulated institutions become large enough to achieve regulatory capture. Or as Simon Johnson said during a Book TV presentation, the six largest banks have “captured the State”. Edward Kane, senior research fellow at the FDIC, made similar comments during testimony before a Senate Banking Committee. Here are the links.
http://www.c-spanvideo.org/clip/3343331
http://www.booktv.org/Watch/11440/13+Bankers+The+Wall+Street+Takeover+and+the+Next+Financial+Meltdown.aspx
Terrific post by Mr. Fernholz.
It’s nice to see a fresh voice on BaselineScenario. The best book I have seen yet on HFT (High Frequency Trading) is that by Joe Saluzzi and Sal Arnuk. I recommend it to all investors who want to know how the majority of trades are executed now, and how the small individual investor largely gets screwed in the process:
The small individual investor can still win on his own, and I strongly encourage those willing to do their homework and diligence not to give up and fight to good earnest fight. Brokers, mutual funds, ETFs, etc…. in large part (>75%) are Charlatans who wish to take 2% or more of your money that you could make on your own for only commissions of trades (and lower commissions at that because when you trade your own account you probably have less turnover).
Empty skirts like Felix Salmon can attempt to scare off individual investors, as his check from Reuters is largely paid by those selling the magic broker/agent snake oil. Those with a working mind know better than to buy it.
Wanna see a JACK*SS insulting his readership and Main Street all in a single swoop??? Observe:
Hey Felix, (AKA jack*ss) don’t tell people, you don’t know, what they “don’t have” in snobbish tones. It makes you look like a patronizing PRICK, British accent or no British accent.
I should have added above that ZeroHedge blog often has terrific and informative posts on HFT. As does one of the Nanex sites. Nanex has some absolutely outstanding posts and graphs/charts on the HFT and the “funny business”, weird price movements and volume changes related to HFT and the algos.
Check these links out and the links inside those links,
http://www.zerohedge.com/news/2012-10-06/guest-post-nanex-investors-need-realize-machines-have-taken-over
http://www.zerohedge.com/news/what-happens-when-hft-algo-goes-totally-berserk-and-serves-knight-capital-bill
http://www.zerohedge.com/news/nanex-white-paper-high-frequency-trading-insatiable-its-hidden-costs
http://www.nanex.net/flashcrash/ongoingresearch.html
Moses, terriific post by Mr. Fernholz/Kwak? Doesn’t context matter? If “experts” continue to frame discussions of trading strategies as components of efficient markets, while ignoring the abrogation of market rules, they’re just putting out more misinformation.
If individual investors want to know what they’re up against, another good book is “The Big Short” by Michael Lewis. Listen to this 30 second clip of Michael Greenberger commenting on Lewis’s book and Credit Default Swaps.
Thanks for sharing this great piece of information. Do keep us update
with some more great information….. equity trading
Glenn, debt is not a resource, collecting interest is, perhaps that is where you got confused.
You make no mention of these HFT parasites locating for speed privileged computers ON THE TRADING FLOORS, paying kickbacks to the Exchanges, scalping fractions of a penny per millisecond. This article is nothing more than propaganda masked as “educational”. Smart readers aren’t buying your CRAP!
filbt, I’ve not commented on debt vs. interest as a resource. Perhaps that ‘s where you got confused.
Fernholz/Kwak are using this discussion of HFT as a vehicle to sell the myth that efficient markets exist. Following that thread, I believe the public/private institutions currently dominating “the market”, are ideologically opposed to a transparent, efficient, market economy. The evidence demonstrating this is incontrovertable. And yet, Fernholz/Kwak ingnore the information in “13 Bankers” that proves efficient markets are not a Wall Street objective. They could be poster boys for the Society of Dishonest Scholars, which is part of the Fed’s new Consumer Financial Protection Bureau.
http://www.booktv.org/Watch/11440/13+Bankers+The+Wall+Street+Takeover+and+the+Next+Financial+Meltdown.aspx
Phil Angelides was Chairman of the Financial Crisis Inquiry Commission. Regarding the crisis he said, “None of what happened was an act of God. The greatest tragedy coming out of this crisis would be to accept the idea that no one could’ve seen this crisis coming and thus, nothing could’ve been done. If we accept this notion, I guarantee you it will happen again”.
By promoting the myth of efficient markets, Fernholz/Kwak are taking the public’s eye off the ball. It’s a head fake designed to put us back to sleep so their Wall Street partners in crime can dump another crisis on taxpayers.
My apologies Glenn, you are correct, I was confused. Too many flo’s of information at once, weathered the circuits and a comment bypassed the board and was reprinted by mistake. I got my best people on it, so it ought not happen again. Cheers!
Oh yea, and Aetna sucks cock too.
http://www.huffingtonpost.com/2012/11/28/the-rise-of-the-sex-machines-roxxxy-sexbot_n_2207584.html
The line of women wanting to buy this holiday gift for their Rethug husbands stretched to infinity – the backlog for orders is 2 years! :-((
We’ve saved the economy with Main Street commerce once again.
Sexbots turn swords back into ploughshares, lower the cost of health care (less killer STDs like AIDS and other festering problems deposited ala winner warrior *punishment* sex), provide LIVING wage manufacturing jobs, double up the utility of high tech by implanting a spy camera/data collector in the sexbot – and what else…? What else in the plus column…?
Now women will be free to have more time to *volunteer* – you know – teach, modernize infrastructure, and do whatever else they want – such as look at the stars at night from nifty telescopes and binocs we bought with our LIVING wages.
Tada! Let the good times role!!
It’s all a *confidence* game, right?
I know what I forgot – where on the sexbot do you want to place your HFV trading finger control?
What a bunch of Crap. Real world doesn’t allow this kind of trading, every 90 seconds.
Come one baby, get real.
I don’t understand. Why do index-beating they funds tend to “place more weight on those stocks with small total market capitalizations and less weight on those stocks with large total market capitalizations?”
What is it about small market cap stocks that makes them outperform large market cap stocks?
Anond, I suspect it’s because HFT depends upon price volatility and transient gaps in information to extract profit. Small-cap stocks have less “drag” on these fluctuations (i.e., shareholders/traders watching them) than large-cap stocks. This is in a market that is not being driven by company or economic fundamentals. In a market that is fundamentals-driven, small-cap stocks will also often outperform large cap stocks if they are marketing disruptive products; i.e., you get a lot more relative growth from 0% of the market to 1% than you do from 99% to 100%. If you are a large-cap stock in a fundamentals-driven market, you’re there often because you’re a reliable generator of dividends and steady appreciation. It’s highly unusual for more than one or two Apples to appear in a generation – that couple high market capitalization with high appreciation.for years on end.
I am afraid I still don’t understand how this strategy can generate high returns in the long term. If it is really as simple and profitable as this, why would not many investors will pursue similar strategies, bidding up the relevant assets (even on the the high-frequency time scale) and wiping out the potential gains?
How do HFT strategies manage transaction costs? How does the IRS deal with the reporting requirements?
As long as there is information asymmetry in the trading system, HFT algorithms can make money off of it. They rely on transient differences between the asking and bid price and the ability to position one’s self in the physical transaction flow advantageously. Those traders that have been able to position themselves closest to the nexus of the trading and to purchase the highest-power computers will always maintain advantages, as the speed of light is a constant. If we were to banish the central exchanges and replace them with a peer-to-peer or regional clearing house system, new players would stand a better chance in this market because average transaction times would increase. But we would lose the transparency of a single-point clearing system. HFT is a video-gamer’s trading ploy and has relatively little to do with the absolute price of an asset – only time-and-place microdifferences in its price. The way that one kills the profitability in HFT is by taxing transactions, and imposing dwell periods between the buying and selling of an asset that allow information asymmetries to settle out.
@Oregano – never going to happen – taxing the transactions.
Tried to set up a payment plan with the IRS to pay off the $900 I owe that is the federal tax on unemployment insurance from Aug to Dec of one recent year, and I now have 4 letters from 4 different IRS locations – nationwide! – each claiming a different amount owed, a different monthly installment due them, and only the due date in common. Which just means that the check I sent is not the correct amount and that will trigger the automatic withdrawal of the full sum that ONLY the Massachusetts office claims I owe (which is different that the amount that was in my tax return for same year – BEFORE penalties, transaction fees and interest).
It’s a personal and HATE driven *economy*. People need to realize that no amount of *good* and *love* makes any difference when it comes to the injustice in the MATERIAL world – it’s not “existential”.
Just a heads up to all my *friends* here for 4 years about the chaos to ensue once the IRS gets to collect fines on a person who did NOT buy for-profit health insurance. The HFT spin cycle designers gave everyone the technology to auto-extract.
Who would write the software for IRS to track HFT transactions and charge a fee?
off topic, sorry
Question: When is President Obama going to quit allowing Timmy Geithner to give footjobs to his Citi cronies for SEC Chairman job??? I think Secretary Geithner has enough of his Citibank cronies’s cum on his feet now, he could use it to ski the western Nevada desert.
The latest joke shot up the flagpole by President Obama, as an insult to those who stood for HOURS in line to vote for him in Florida, because they thought he was finally going to get his head out of Geithner’s A.$.S on financial regulation.
http://blogs.reuters.com/muniland/2012/11/29/sallie-krawcheck-should-not-run-the-sec/
Let’s cut the CR*P on holding Presidential appointments hostage, because 3 Senate Republicans are ticked the psycho Mormon got his rear handed to him in the elections. Enough of filibusters bottlenecking legislation while the Republican who started the filibuster is skiing in Aspen, Colorado. Enough is enough
Those backwoods country yahoo filibusterers, provide their own company. The quality of company they keep is suspect, and tends to lean republician.
The list of what I rather be doing is HUGE, but I believe that there are some questions about the details of high frequency trading and what bank accounts those traders have access to and how many different players can lay claim to, in this case, IRS collections at certain points in time – but all the time, hence the application of the “to infinity and beyond” algorithm.
And to borrow the talking point – even if you get that $900 from me in one fell swoop, it’s not going to make a dent in the deficit….although in my case, as compared to “filths” with his trillions, it does make the difference between continuing to be a human being with every right to the fruits of YEARS of honest labor, and having a gang of psychos on a Urantia Book forum cheer at their victory – my gene pool causes their heads to do that exorcist 360 head spin….but I digress into a truth that will make me look nuts – so let’s get back to the latest rip-off planned for the Middle Class and the role HFT software will play in it.
Since once again I am being stalked in order to be the archetype for a software program to extract, in this case, the IRS fee that they can levy against an INDIVIDUAL for not having the money to buy for-profit health insurance, let’s continue to use me as the case study for the proof.
Easiest place to start is with the total number of people collecting unemployment today and include everyone back to the point in time in 2008 when 600K a month *lost* their connection to fiat $$. Even if this some people in this group are still buying for-profit insurance, they shouldn’t be. They’ll just get to poverty faster with no safety net when it comes to health care. But they will still be fair pickings for the IRS down the road, and the documentation I have of 4 letters from 4 different, nationwide locations indicates the program is DEFINITELY in the make-it-up as you go along to get to the magic date where the automatic extraction is programmed and will achieve the goal. The whole installment agreement program is just the means to get to the ends – it’s not real. A spin cycle of complexity to hide the fraud….
BUT, while the program is a you’ve been punk’d joke for the individual, during this paperwork spin time, 4 different *traders* nationwide are accessing that $900 as a collection that is a done deal – just like the heath care fees will be for the unemployed. So in my case, who are the *traders*? Do they know that they each claim a slightly different sum, none of which is the original amount from the tax return – meaning some traders are even using a sum based on compounded interest and each fee a different %? Do they know that they are all saying the 900 is theirs at the same time – meaning 900 is in 4 different accounts as 900 – not as one-fourth of 900? So in the final analysis, in order for everyone to actually get the 900, the INDIVIDUAL taxpayer will have to have the total 3600 (4 x 900) extracted. And the handover to the next assembly-line skinning station in the FIRE sector after the IRS starts when the 900 is a surprise extraction timed to occur when a greater sum (mortgage, rent, or entire months worth of bills) are “pending” so that they all bounce. Had that done already by the car insurance co. – they cleverly drew out twice the amount to create the bounce which was done without that double withdrawal being on the PERMANENT monthly statement – it was only caught by chance through an ATM read out….need I go on? I’m bored :-)
The solution to this is not “political”, nor is it better math….
A *government* is nothing other than the enabler of predators when they clear all civilized protections that the INDIVIDUAL has the god-given right to employ by writing *legislation* that gives the predators clear access to the extraction of it all – to infinity and beyond.
Who is nutz? And do we really need to get Jesus involved in such puerile, hooligan shenanigans? We need an end of the world to shut down the algorithm?
Well Tony the forest ranger may just show up and ruin the party one day!
http://www.upworthy.com/video-watch-the-big-powerful-bankers-get-schooled-by-a-wicked-smart-12-year-old?g=2&c=ufb1
Flash crashes, naked shorts, creative destruction, and black box manipulations to name just a few of the Wealth & Stealth Empires from the hedge funds and Private Equity agglomerations. The Market is supposed to support a healthy productive corporate world that serves their entrepreneur investors and the share holding risk takers that back them… Show us WHERE this brilliant “skim the cream off the top” has built up a healthy infrastructure. I guess Baseline Scenario just turne the distorted corner to “Bottom line Scenario”?
This is moral hazard at a social and cultural level…wasting people that don’t belong to them!!!!!
Fast Money…an invisible Cosa Nostra to say the least.
Pump and dump the suckers till their dry…and then scavenge & steal their assets that you manage to buy out…take over and rent back!
How do you sustain empires against time and successions?
Privatize everything that moves!