No one really knows if HFT is good or bad

Like every other human in the known Universe, I'm currently reading Michael Lewis' Flash Boys. But I've been thinking about HFT for quite a while, because Stony Brook has a lot of people who study it for a living (including some people who have done it for a living at Renaissance and other top firms). So even though I'm only halfway through the book - and I'll write a review when I finish it - I have some thoughts on the topic.

Basically, no one knows if HFT is good or bad. There are two reasons for this. The first is that no one really knows what HFT does to markets. The second is that no one really knows what is good or bad for markets.

Why don't we know what HFT does? Well, one reason is that there are a lot of different HFT strategies. Some HFTs could be pure market-makers, just gobbling up bid-ask spreads. Some could be "front-runners", looking for signals of large institutional trades and trading in between the chunks of these trades (this is the strategy Lewis spends a lot of time discussing). There are other common ones (like latency arbitrage, Twitter mining, high-frequency statistical arbitrage etc.). No one has the data to know how much of each of these is being done at any given time.

But a lot of HFTs simply don't know what their strategy really is. They hunt for patterns in prices or orders, find a pattern that seems to work, and trade on it until it stops making money. They don't have any idea why the pattern exists. Sometimes it only exists for a few seconds. In fact, if they stop to gather enough information about the pattern to figure out why it's there, it often disappears! Actually, there are deep mathematical (information-theoretical) reasons to suspect that lots of HFT opportunities can only be exploited by those who are willing to remain forever ignorant about the reason those opportunities exist. It's mind-bending (and incredibly interesting).

Needless to say, we have no idea what this latter type of HFT does to the market, because even its users don't know what it is! But even for the simpler strategies - market-making, front-running, etc. - we don't really know how they impact markets. This is because all of these strategies involve the phenomenon of asymmetric information. Most of our finance theories - the kind where you get nice clean results, like factor models or CAPM-type models - don't involve asymmetric information. Once you add it in, stuff gets complex and weird, fast. If you ever think you understand how financial markets work, go read Asset Pricing Under Asymmetric Information by Markus Brunnermeier, and you will be convinced otherwise.

Do market-makers increase or decrease liquidity? Do front-runners increase or decrease it? What about informational efficiency of prices? What about volatility and other forms of risk, at various time scales? What about total trading costs? Good luck answering any of these questions. Actually, Stony Brook people are working on some of these, as are researchers at a number of other universities, but they are huge questions, and our data sets are incredibly limited (data is expensive, and a lot of stuff, like identities of traders, just isn't recorded). And keep in mind, even if we did know how each of these strategies affected various market outcomes, that wouldn't necessarily tell us how the whole ecosystem of those strategies affects markets - after all, they interact with each other, and these interactions may change as the strategies themselves evolve, or as the number and wealth of the people using each strategy changes. 

Confused yet? OK, it gets worse. Because even if we did know how HFT affects markets, we don't really know if it's good or bad on balance. For example, HFT defenders often say HFT provides "liquidity". Is liquidity good for markets? How much is liquidity worth, are there different kinds of liquidity, and does it matter when the liquidity comes? If I have a bunch of totally random trading, that certainly makes markets liquid, but is that a good thing? Actually, maybe yes! In lots of models of markets, you need random, money-losing "liquidity traders" in order to overcome the adverse selection problem, thus inducing informed traders to trade, and getting them to reveal their information. But HFTs don't lose money, they make money - is their liquidity provision worth the cost?

To know that, even if we knew the impact of HFTs on informational quality of prices, we'd have to know the economic value of informational efficiency. Suppose the true worth of GE stock. according to the best information humanity has available, is $100. Suppose the price is $100.20. How bad is that? How much is it worth, in economic terms, to push the price from $100.20 to $100.00? Is it worth $0.20 per share? It depends on how GE's stock price affects the company's investment decisions. To know that, we need an economic model of corporate decision-making. We have many of these, but we don't have one over-arching one that we know works in all circumstances. Corporations are way more complicated than what you read in your intro corporate finance textbook!

(And this is all without thinking about weird things like behavioral effects of the humans who interact with HFTs...)

To sum up: There is very, very little that we know abouthttp://streetwiseprofessor.com/?p=8340 HFTs. We have incomplete data, incomplete understanding of the nature of HFT strategies, incomplete understanding about the interactions between ecosystems of strategies, incomplete theories of markets under asymmetric information, and incomplete understanding of the economic value of liquidity and informational efficiency.

So what the heck do we do?? Obviously we need a lot more research, but in the meantime, why not enlist the market itself to help give us some more info? If we legalize and facilitate a number of kinds of alternate kinds of stock exchanges where HFT has a much harder time - e.g. dark pools, batch-auction exchanges, etc. - we can learn a lot (though not nearly all we need to learn) about the overall impact of HFT (and other algorithmic trading!).

Like I said, I haven't finished Flash Boys (and I'll write a review when I do), but my early impression is that Lewis presents a very oversimplified picture of how HFT works - but in plumping for alternate types of exchanges, he gets the solution broadly right.


Updates
Here's a similar post by Craig Pirrong, who shares my uncertainty about the costs and benefits of liquidity and informational efficiency, but who is slightly more confident about the net effects of HFT on informational efficiency.

David Glasner has a great post on the topic as well. The biggest cost of HFT is probably the resources that are wasted on "information tournaments". I'm setting up a lab experiment about this very topic, in fact. This problem was pointed out in a famous 1971 paper by Jack Hirshleifer.