Tuesday, April 15, 2014

Book Review: Flash Boys

OK, I promised a review of Michael Lewis' Flash Boys, and here it is.

Flash Boys is an excellent book. It's fun and readable. It gives readers a window into how the stock market really works, and the kind of people who work there. It explains a lot about a very important issue that finance is facing.

For example, Lewis explains how a lot of the things that high-frequency traders are doing are just ways to game an SEC regulation, "Reg NMS", that came into effect in 2007. If this regulation were changed, a lot of the things HFT is doing would no longer make money. (Since gaming regulations is probably not a very socially valuable activity, a lot of financial economists have been calling for this regulation to be changed to create "batch auctions" every second or so; sadly, Lewis does not discuss this proposed solution.)

Also, Lewis gives a great explanation of how large banks use "dark pools", which were created to be a sort of refuge from HFTs, to rip off clients by selling them out to HFTs. Because the clients don't know the rules of the pools, and because they trust the big banks (yes, even after 2008), the clients lose money. In fact, I wish Lewis had shone more light on the dark pool issue in the first half of the book.

Finally, Lewis does a great job of exposing the injustice committed by Goldman Sachs against programmer Sergey Aleynikov. This actually had little to do with HFT, but was a classic tale of the evil that is callously committed by large bureaucracies where everyone is looking out for their own interests instead of trying to do the right thing.

So you should definitely read Flash Boys. But when you do, keep in mind two things that I think Lewis leaves out.

The first thing Lewis leaves out actually strengthens his case against HFT. Lewis, who tells a tale of bad guys ripping off good guys, glosses over the massive waste represented by the bad guys' own effort! At some point, it's clear that more speed is unnecessary for the efficient functioning of financial markets. There's no reason why prices need to adjust in a millisecond instead of a second. All the effort that HFTs spend on making this happen - all the math geniuses, and the supercomputers, and the hundreds of miles of fiber-optic cable - are valuable resources that could probably be put to much more productive use elsewhere in the economy. This has been pointed out by bloggers like David Glasner and Paul Krugman, and is based on a famous paper by Jack Hirshleifer, which you should read. A tale of social waste is not as exciting as a tale of plucky good guys rebelling against nefarious evil geniuses, but from an economist's perspective it's a big problem. The Death Star, if you think about it, was a massive waste of engineering talent.

The second thing Lewis leaves out is the other side of the story. All throughout Flash Boys, the victims of the HFTs - "investors" - are portrayed as good guys, just doing their jobs. The investors just want to trade, we're told, and the HFTs are unwanted middlemen who stand between the investors and stop them from trading. But who are these "investors"? Why are they doing all this "trading" in the first place?

Lewis' answer is that the investors are investing money on behalf of you and me - pension fund managers, for example, or mutual fund managers. But why should that require a lot of trading? Your pension manager is not Warren Buffett - he is not going to beat the market year after year. What he should do is to put your retirement money into a nice diversified basket of assets - ETFs, index funds, and the like - and just let it sit there for decades, maybe rebalancing it once a year or so. If he does that, the tiny amount that he gets front-run by HFTs at the beginning and end of those decades is going to make precisely zilch of a difference to your retirement account. But if instead, he trades and trades and trades enough to get substantially ripped off by HFTs, chances were that he was just playing the old sucker-and-be-suckered game that Michael Lewis wrote about in Liar's Poker.

In which case, he should be fired and replaced by a cat.

I'm actually not kidding. In 2012, the UK Observer held a stock-picking contest between professional investment managers - the good guys of Flash Boys - and a house cat named Orlando. The cat won. This result is, of course, supported by a long line of academic research into the unimpressive returns of professional money managers. As much as a quarter of your retirement account is being sucked away by the massive fees of money managers who, on average, are worse randomness. And this has been true since long before HFT existed.

In other words, HFT acts as a tax on trading, but much of that trading is hurting your mom and dad's retirement accounts in the first place - the "investors" who are the unambiguous good guys in Flash Boys are not so unambiguous in real life. In fact, if HFTs prompt pension funds to shift to passive management (or to "patient capital" investing that improves corporate governance), that could A) improve the efficiency of the financial sector, and B) boost returns for you and your mom and dad. In fact, there are signs that this is already happening! Led by California's massive CALPERS, big pension funds are shifting more and more money into ETFs, index funds, and the like.

Might we have HFT to thank for this sensible, rational shift? That's hard to say, but I think Michael Lewis should have at least mentioned the possibility in Flash Boys. Instead, I think he gives the "investors" and their "trading" a free pass.

So anyway, keep these omissions in mind when you read the book. But do read it. It's one of the most important popular books on modern finance, and - along with Scott Patterson's The Quants - is one of the first to introduce people to the brave, weird new world of electronic trading.


  1. "The Death Star, if you think about it, was a massive waste of engineering talent."

    Balancing opportunity cost of doing it against opportunity cost of not doing it, I say it wasn't a massive waste of engineering talent.

    1. "The Death Star, if you think about it, was a massive waste of engineering talent."

      Balancing the opportunity cost of doing it against the opportunity cost of building a new Chewbacca planet after theirs was destroyed, I say it was a huge waste of engineering talent. QED.

      RIP Chewbacca.

    2. Anonymous3:06 PM

      Considering how easily it was destroyed, was it really such a waste of engineering talent?

  2. Noah,

    Here is an article in Atlantic on HFT showing that there is inefficiency in the market at a few hundred milliseconds levels. Before computers, bid and ask spread was very large (market being inefficient) and broker commissions were very high, about 10 or 20 times what one pays today.

    Pension fund managers trade for one simple reason: they get commission based on asset under management and they can drive prices up at the end of the month, quarter or year that maximizes their commissions, or they are out for themselves, like cat on the prowl. To do this, they need to churn the market and HFT does that.

    Did this happen before HFT? Of course it did, but we did not have the visibility in their business.

    Is there a solution for it? Yes, just take the time advantage away from HFT traders by equalizing and randomizing the trade orders without telling them! (NYSE and others sell this knowledge to HFT traders). Effectively get HFT traders to put their skin in the game.

  3. Sorry, forgot to post the link!


  4. "The Death Star, if you think about it, was a massive waste of engineering talent."

    Can you explain why this is bad, but burying money in bottles to be dug out of the ground is good? Doesn't the Keynesian multiplier make this possibly beneficial?

    Unless we're running out of fiber, I'd say this is a deployment of otherwise idle resources. And we need more, rather than fewer jobs for our newly trained computer science majors.

    1. Well, the Keynesian theory says that although that seems like a waste, and usually is, occasionally it's actually not a waste!

    2. Also the Keynesians would point out that the same multiplier effects that sometimes come from wasteful investment could also be produced by smarter investment.

      If the primary impact of some investment is -5 utils, but it produces +10 utils of additional multiplier effect benefits, and it is compared to an alternative that produces the the same +10 multiplier effect with a +5 primary effect, then while both investments have a positive net utility, you should definitely choose the second.

      Why pay people to bury money in bottles when you could pay the same amount of people the same amount of money to plant trees?

    3. Anonymous4:14 AM

      Not a fan of HFT, but HFT was mostly financed by credit that would *not* have have otherwise been created, by private banks. Is there some shortage of networking gear in some other part of the economy that would have provided +10 utils? If not, then the marginal demand created for networking gear was additive to GDP.

      Why is HFT investment an example of resource opportunity cost, but government does the same thing it's "glorious expansionary Keynesian stimulus for motherland GDP"? When did Krugman suddenly adopt "the Treasury view", or does social opportunity cost only come into play when the private sector invests capital?

    4. Anonymous1:14 PM

      "HFT was mostly financed by credit that would *not* have have otherwise been created, by private banks."

      Please cite your evidence for that claim, because I would love to read it.

  5. "This result is, of course, supported by a long line of academic research into the unimpressive of professional money managers"

    Read that sentence again.

  6. Anonymous7:40 AM

    A couple of thoughts from an HFT trader

    1) HFT existed before Reg NMS. Reg NMS is a big pain for HFT because in order to be fully compliant, it is necessary to spend enormous resources on engineering, legal and compliance. I am not sure who it is good for, but HFT would be better off without it

    2) Social Waste - this is, I am sorry to say, bad economics.

    When private money is spent in the course of competing for profits in any industry, lots of positive externalities are generated. For example, HFT industry spends a lot of money on high performance hardware, thus effectively subsidizing its development. Or let's take Spread Networks, the line that was installed at a cost of hundreds of millions, between Chicago and NJ. You realize, this is just a fast, high performance telecom line? Right? it can be used for voice, video, whatever you like. The existence of this line, which was heavily subsidized by HFT, is a great thing !

    3) You mentioned this. Why do these investors trade with HFTs so much in the first place? We just gave a Nobel prize in economics to the guy who says they have no alpha whatsoever. They are the real social waste

    4) Maybe it is true that society would be in theory better off if these MIT grads worked on self driving cards instead of HFT. But, we do have a free society. People do want to work on whatever they want. What do most startups in California work on? Self drviing cards or photo apps ?

    1. Front-running is what HFT outfits do. It's completely unjustifiable on moral and legal grounds, despite the desperate spinning about its marginal utility by its practitioners and proponents.

    2. Anonymous9:22 AM

      Really? HFTs don't have customers typically. How can they front run customers they do not have? From Wikipedia


      Front running is the illegal practice of a stockbroker executing orders on a security for its own account while taking advantage of advance knowledge of pending orders from its customers. When orders previously submitted by its customers will predictably affect the price of the security, purchasing first for its own account gives the broker an unfair advantage, since it can expect to close out its position at a profit based on the new price level. The front running broker either buys for his own account (before filling customer buy orders that drive up the price), or sells (where the broker sells for its own account, before filling customer sell orders that drive down the price).

      Allegations of front running occasionally arise in stock and commodity exchanges, in scandals concerning floor brokers and exchange specialists.

    3. Anon:

      Your point #1 slightly contradicts your point #2, wouldn't you say? Also, a lot of other tough technical challenges generate positive spinoffs too.

    4. Anonymous9:21 PM

      Noah: not quite. Reg NMS is a set of regulatory rules by SEC, which are imposed on all market participants, HFT or not. All of them have to write software to comply with it. So this has nothing to do with HFT and is not in any way a benefit to HFT per se.

      As far as other technical challenges, well, sure, and we should leave alone all industries which are net externality positive, no?

  7. Do I see a contradiction here?
    1. "If this regulation were changed, a lot of the things HFT is doing would no longer make money."
    2. " a lot of financial economists have been calling for this regulation to be changed"

  8. The "cat vs Money Manager" thing fascinates me. Do the latter only survive because so many investors are ignorant, gullible, or foolish? Is it just the market taking forever to clear?

  9. In the Atlantic article you link to nice point on the math of compound interest. Just a 1% less return will eventually mean you have half as much money. In fact, after enough time it will mean you have 99% less money, and so on.

    How long? It depends one percent less than what. The historical average real, inflation-adjusted, return on a well-diversified stock portfolio is about 7%. The expense ratio on a good index fund is near zero, about 0.05%. So, let's compare a 7% return to a 6% return, which you can quickly do with an Excel spread sheet. It actually takes 63 years for an amount of money invested at 7% to end up twice as large as an amount invested at 6%, so a long time. But if you're investing for your children,…

    Still, after 40 years, not that long a time for a 401k, it's 47% more! But only your beginning money is 40 years old; next year's contributions are one year less old, have had one year less to grow, and so on. Compare, then, $400/month ($4,800/year) over 40 years at 7% to at 6%. It's $958k to $743k, so you end up with 29% more money! No small amount. And on top of this, a lot of these funds hype with things like trendy high tech stocks, that can be very risky. And, there are big incentives, often, for these money managers to really gamble, heads they win, tails you lose.

    Still, if everyone does index funds, who pushes prices towards their fundamental values? Not to worry; that's certainly not a job for laypeople; that would be left to the sophisticated Warren Buffet types, and good hedge fund managers, where it should be left.

  10. The cat vs moneymanager experiment primarily works because of smallcap bias which has outperformed the last 10 years. Something like 90% of all randomly generated equally weighted portfolios of 20 stocks outperformed the market-cap weighted S&P 500 index the last ten years. Nothing margical.