In defense of the EMH

The "Efficient Markets Hypothesis" is a popular target of anger and derision among lay critics of the econ profession. How can financial markets be "efficient" when they just crashed and took our economy down with them? And when sensible people like Bob Shiller, Nouriel Roubini, Bill McBride, et al. were screaming their heads off about a housing bubble years before the pop?

Of course I have some sympathy for these complaints. But the more I learn about and teach finance, the more I learn what an important and useful idea the "EMH" in fact is. I don't want to say that the EMH is unfairly maligned, but I do think that its vast usefulness is usually ignored in the press.

First of all, people should realize that the EMH is misnamed - it's not really a hypothesis, it's not about "efficiency" in the economic sense of the word, and it's not unique (so it shouldn't have a "the" in front of it). Some of this miswording was just semantic clumsiness on the part of the people who came up with the theory. Some was sloppy science.

The "efficient" part of "EMH" doesn't mean that financial markets lead to a Pareto-efficient outcome. You could have externalities - for example, every time you make a financial transaction, God might kill a kitten - and the market could still be "efficient" in the way that financial economists use the term. Similarly, a vastly "inefficient" financial market might be Pareto efficient, since it might only be possible to make profits by taking advantage of someone else's stupidity.

The "efficient" actually just refers to information-processing efficiency. What that basically means is that if there's some piece of information out there - some fact about a company's balance sheets, or some pattern in past prices, etc. - the market price should reflect that piece of information. That's what "efficient" means here.

But exactly how should prices reflect information? Here's the bigger problem with the term "EMH" (the "sloppy science" part) - it's not really a hypothesis. How prices reflect information will always depend on people's preferences. In finance, preferences include preferences about risk. So without a measure of risk, it's impossible to scientifically test whether or not prices incorporate information. To be a real hypothesis, the EMH needs to be paired with a specification of risk (or, more generally, a hypothesis about people's preferences with respect to uncertainty and time, and a hypothesis about the sources of risk). And since there are many possible such specifications, there isn't just one "EMH"...there are infinite.

To complicate things, "the EMH" says nothing about how long it takes for the market to process information. So even if an EMH happens to be true at one frequency (say, daily), it might not be true at the 1-second frequency.

OK, so is even one of these EMHs true, at some frequency? We can do statistical tests, but we'll never really know. First of all, our tests are all pretty weak. But more importantly, conditions may change! An EMH might be true for a while, and we might conclude it's true, and then things might change and for one or two years it might stop being true, and then we'd do some more statistical tests and say "Oh wait, I guess it's not true after all!", and then it might go right back to being true! We generally assume the laws of physics don't change from year to year, but it's easy to imagine that the "laws" of finance aren't as immutable. What if the market is "efficient" 99% of the time, and the rest of the time there's a catastrophic bubble? 

And to top it all off, theory says that the strong form of the EMH can't even be true.

So "the EMH" is very limited as a scientific hypothesis or physics-like law of nature. And I think that ever since many of these points were pointed out (I think by Andrew Lo, though someone else may have preceded him), financial economists have stopped talking about "the EMH" as such, except in a vague hand-wavey way during informal discussions. Sloppiness has been much reduced.

But I do seem to recall that the title of this post was "In defense of the EMH". So I had better get around to defending it! What I want to defend is the idea behind the EMH. Even if the data rejected every single EMH, the idea would still be incredibly useful for the average person. 

Let's call this idea the Random Markets Idea, or RMI.

The simplest form of the RMI was stated by Paul Samuelson in 1965: "Properly anticipated prices fluctuate randomly." Basically, if it was pretty easy to see where prices were headed, a lot of people would see it, and try to make free money by trading on it. Since people in the finance industry are doing a lot of work - watching the news like a hawk, doing constant analysis of changing numbers - chances are that the price change will happen so fast that you won't have time to get in on the action. So from the perspective of any of us who doesn't have a supercomputer in his head, prices movements must be unpredictable and surprising. They must seem random. 

That's it. That's the RMI. Note that this is very different than saying "On average, people don't beat the market average." This is more than that. This is saying that even if you manage to beat the market average for a year or two years or even ten years, you shouldn't expect to be able to repeat your performance next year. 

That may seem counterintuitive, or even silly. "Hey," you think, "I beat the market last year, so I must be one of the smart guys! And that means I should be able to repeat my performance...right?" Well, maybe. But the RMI says that that's actually very, very unlikely. It's far more likely that you just got lucky.

Now here we get to why the RMI is so useful to you and me and most people (and to the managers of our pension funds and mutual funds). It provides a check on our behavioral biases. Probably the most robust findings in the field of behavioral finance is that individual investors do badly. They are overconfident. They trade too much and take losses on trading costs. They suffer from biases like disposition effect, probability mis-weighting, recency bias, etc. And as a result they lose money, relative to the wise folks who just stick their money in a low-cost diversified portfolio and watch it grow. As for institutional investors - mutual fund managers and pension fund managers - we don't know as much about what they do, but we do know that very few of them manage to consistently beat the market, year after year (and most don't beat the market at all).

It's interesting to note that people usually think of behavioral finance as being an alternative to efficient-markets theory. And sometimes it is! But in the case of personal investing - i.e., the single most important way that you will probably participate financial markets - the two ideas support each other. The RMI says "You can't beat the market"; behavioral finance says "But you're probably going to lose your money trying." 

Of course, even the RMI isn't quite true. There are some people - a very few - who correctly guess price movements, and make money year after year after year (I work with a couple). But you're very unlikely to be one of those people. And your behavioral biases - your self-attribution bias, overconfidence, and optimism - are constantly trying to trick you into thinking you're one of the lucky few, even when you're not. 

The RMI is an antidote to this! Just remind yourself that market movements should be really, really tough to predict. Then, when you start to think "It's so so so obvious, why can't people see AAPL is headed for $900, I'm gonna trade and get rich!", you'll realize that no, it can't be that obvious. And you'll restrain your itchy trigger finger. And when you start to think "My money manager is awesome, he beat the market the last 5 years running, I'll pay his hefty fee and he'll make me rich!", you'll stop and realize that no, it was probably luck. And you'll think about putting your money in index funds, ETFs, and other low-cost products instead.

In general, the RMI focuses your brain on assessing risks instead of trying to outguess the market. This is important, because risk is a difficult thing to think about, while making bets and guesses about returns is relatively easy. But in the real world, most of your portfolio's return will be determined not by how well you make bets and guesses, but by the riskiness of the asset classes in which you choose to invest (stocks, bonds, etc.). Most of the return you get, in the long run, will come from taking risk. But because risk is a cost (imagine if you have an emergency and need to withdraw your money while the market is down!), you need to carefully balance your desire for return with your tolerance of risk. This is what the RMI helps you think about.

OK, let's step back a second. Why do we put our trust in any scientific theory? Well, because it's useful. We know Newton's laws aren't exactly right, but we know they're very useful for landing a rocket on the moon, so when we land rockets on the moon we don't worry about the slight wrongness. And as for our most advanced theories - relativity, quantum mechanics, etc. - well, even those might just be approximations of some more general theory that we just haven't figured out yet. But in the meantime, we use what we've got if it's a good baseline approximation.

The RMI - the general idea behind the various EMHs - is a good baseline for the personal investor (and probably for the pension fund manager too). It works pretty darn well. There are plenty of other areas in which market inefficiency/predictability may matter - financial regulation, corporate compensation, etc. - but you won't typically need to worry about those. You'll be better off treating the market as if it's more-or-less unpredictable and random.


Addendum: It would be unfair not to point out that the RMI is also an important baseline, or jumping-off point, for most financial research. First of all, it leads to the idea that most of the observable factors that explain stock returns should be things that move many stocks at once (and thus can't be diversified). Second, it helped motivate the "limits to arbitrage" literature - if predictable market movements are due to "the market staying inefficient longer than you can stay solvent" (as a famous hedge fund manager-turned-economist once put it), that tells us that when we see things like bubbles, we should look for reasons why "smart money" investors like hedge funds can't stay solvent. Third, the RMI focused financial economists themselves on explaining risk. That has led to the observation and investigation of interesting phenomena like fat-tailed returns, clustered volatility, tail dependence, etc. Fourth, the fact that it's hard to beat the market raises the important question of why so many people try (and why so many people trick themselves into thinking they did, when they didn't). That investigation has led to much of behavioral finance itself.

In other words, in science as in your personal investing life, the RMI serves as the fundamental baseline or jumping-off point. That doesn't mean it's the destination or the conclusion of financial economics. It isn't. But having a good baseline principle is extremely important in any science. You need to know where to start.


Update: I wasn't going to write about this, but John Aziz in the comments made me unable to resist. What I actually think about the RMI (or "the EMH") is this: At any given moment, there are infinitely many models that describe financial markets better than the RMI. And at any given moment, there are a finite number of available, known models that describe financial markets better than the RMI. But all non-RMI-type models will stop working well shortly after they are discovered. In other words, if you had to pick one model of financial markets and stick to that model for a very long time, the RMI is the best one you could pick. So in some sense, the RMI is the closest you can get in financial markets to an exploitable, stable "law of nature" like the models we use in physics. For people who don't have time or skill to constantly search for new models, the RMI is best. I plan to make this idea the subject of another post in the future.