Un-diversification: King of investor biases

(This post originally appeared at Bloomberg View.)

A lot of behavioral finance is basically about helping the rich and the upper-middle class. People in the middle and at the low end of the income scale don’t do a lot of individual investing -- most of their wealth, if they have any, is tied up in their house, pension or retirement account.

But that’s OK. When individual investors (i.e., you and I) make mistakes, the money we lose doesn’t flow to the poor and the unemployed -- it flows somewhere into the bowels of the financial industry. So behavioral finance researchers don’t tend to lose a lot of sleep over the fact that we’re giving the rich and the upper-middle class a hand.

Since the 1990s, a bunch of professors have found what brokers and financial planners doubtless already knew -- individual investors tend to underperform the market average. Actually, this is no surprise. Individuals have less information than the banks, hedge funds and money managers, and are usually not trained professional finance people. Of course, on average they’re going to lose, unless they all start buying index funds.

This isn't to say that all individual investors lose. We’ve all heard the stories of Uncle Bob who made a killing trading stocks. But studies repeatedly find that only 5 percent of individual investors can consistently do better than an index fund.

Why are you and I so bad at managing our investments? A lot of psychological explanations have been offered. There’s overconfidence -- the tendency of people trading stocks not to worry about why the person on the other end of the trade is so eager to take the opposite bet. There’s the disposition effect -- the tendency of people to sell winning stocks too early in order to lock in profits, or hold on to losing stocks too long in the desperate hope that they will recover. There’s the hot-hand fallacy, which is the tendency to mistake statistical blips for durable trends, and its twin brother the gambler’s fallacy, which is the mistaken notion that a run of bad luck has to be followed by a run of good luck. There’s attention bias, which draws people’s eyes to glamorous or familiar stocks and cause them to overlook more lucrative opportunities.

But according to a recent paper by a team of German economists from Goethe University in Frankfurt, there is one mistake above all others that hamstrings individual investors -- the failure to diversify.

Diversification is something we all hear about, but the logic of it has surprising trouble penetrating our heads. After all, how are you going to beat the market unless you make different bets than the market? The answer, of course, is that usually, you’re not going to beat the market -- it’s going to beat you. The German economists study an absolutely huge database of individual investors, and find that lack of diversification reduces the average investor’s performance by 4 percentage points a year!

To give you a rough ballpark idea of how much this matters consider this: if you saved $3,000 a month every month for 30 years and earned a return of 7 percent, you would end up with more than $3.6 million. But if you got a 3 percent return, you would end up with only $1.7 million, or less than half. That’s a pretty big deal.

The authors of the paper find that compared with under-diversification, all the other biases don’t matter much. That’s hardly surprising, because the more you diversify, the less room there is for any bad stock pick to affect your overall wealth.

How do you avoid the failure to diversify? Simple: Invest in index funds, or in exchange-traded funds that are similar to index funds. In other words, follow the example of a rapidly rising number of investors.

Now here’s a more unsettling question: What happens if too many people diversify too much? Markets aren’t just supposed to reduce risk -- they are also supposed to process information, and send capital to the companies that will use it for the most productive purposes. If everyone is just indexing, then the number of people dedicated to processing information -- to figuring out which companies actually deserve capital -- will go toward zero, and the markets will become just a casino.

So maybe there is reason for people to un-diversify their portfolios -- a little bit. If you have some real knowledge that other people might not have -- if you’ve done deep research on a company, or if you know an industry really well -- then maybe you should dedicate a bit (but only a bit!) of your portfolio to making a bet on that knowledge.

In general, though, the best advice that behavioral finance researchers can give you is to stay diversified.