Friday, August 01, 2014

Un-diversification: King of investor biases

(This post originally appeared at Bloomberg View.)
A lot of behavioral finance is basically about helping the rich and the upper-middle class. People in the middle and at the low end of the income scale don’t do a lot of individual investing -- most of their wealth, if they have any, is tied up in their house, pension or retirement account.
But that’s OK. When individual investors (i.e., you and I) make mistakes, the money we lose doesn’t flow to the poor and the unemployed -- it flows somewhere into the bowels of the financial industry. So behavioral finance researchers don’t tend to lose a lot of sleep over the fact that we’re giving the rich and the upper-middle class a hand.
Since the 1990s, a bunch of professors have found what brokers and financial planners doubtless already knew -- individual investors tend to underperform the market average. Actually, this is no surprise. Individuals have less information than the banks, hedge funds and money managers, and are usually not trained professional finance people. Of course, on average they’re going to lose, unless they all start buying index funds.
This isn't to say that all individual investors lose. We’ve all heard the stories of Uncle Bob who made a killing trading stocks. But studies repeatedly find that only 5 percent of individual investors can consistently do better than an index fund.
Why are you and I so bad at managing our investments? A lot of psychological explanations have been offered. There’s overconfidence -- the tendency of people trading stocks not to worry about why the person on the other end of the trade is so eager to take the opposite bet. There’s the disposition effect -- the tendency of people to sell winning stocks too early in order to lock in profits, or hold on to losing stocks too long in the desperate hope that they will recover. There’s the hot-hand fallacy, which is the tendency to mistake statistical blips for durable trends, and its twin brother the gambler’s fallacy, which is the mistaken notion that a run of bad luck has to be followed by a run of good luck. There’s attention bias, which draws people’s eyes to glamorous or familiar stocks and cause them to overlook more lucrative opportunities.
But according to a recent paper by a team of German economists from Goethe University in Frankfurt, there is one mistake above all others that hamstrings individual investors -- the failure to diversify.
Diversification is something we all hear about, but the logic of it has surprising trouble penetrating our heads. After all, how are you going to beat the market unless you make different bets than the market? The answer, of course, is that usually, you’re not going to beat the market -- it’s going to beat you. The German economists study an absolutely huge database of individual investors, and find that lack of diversification reduces the average investor’s performance by 4 percentage points a year!
To give you a rough ballpark idea of how much this matters consider this: if you saved $3,000 a month every month for 30 years and earned a return of 7 percent, you would end up with more than $3.6 million. But if you got a 3 percent return, you would end up with only $1.7 million, or less than half. That’s a pretty big deal.
The authors of the paper find that compared with under-diversification, all the other biases don’t matter much. That’s hardly surprising, because the more you diversify, the less room there is for any bad stock pick to affect your overall wealth.
How do you avoid the failure to diversify? Simple: Invest in index funds, or in exchange-traded funds that are similar to index funds. In other words, follow the example of a rapidly rising number of investors.
Now here’s a more unsettling question: What happens if too many people diversify too much? Markets aren’t just supposed to reduce risk -- they are also supposed to process information, and send capital to the companies that will use it for the most productive purposes. If everyone is just indexing, then the number of people dedicated to processing information -- to figuring out which companies actually deserve capital -- will go toward zero, and the markets will become just a casino.
So maybe there is reason for people to un-diversify their portfolios -- a little bit. If you have some real knowledge that other people might not have -- if you’ve done deep research on a company, or if you know an industry really well -- then maybe you should dedicate a bit (but only a bit!) of your portfolio to making a bet on that knowledge.
In general, though, the best advice that behavioral finance researchers can give you is to stay diversified.


  1. Anonymous2:28 AM

    Isn't the logic here somewhat circular? Roughly speaking index fund = diversification. So if you underperform an index fund then your underperformance must definitionaly be explained by your not being as diversified as an index fund. Or did I miss something?

    Following this through, if you do beat an index fund, then your overperformance must definitionaly be explained by your not being as diversified as an index fund. And so couldn't you equally conclude that the reason institutional investors can beat the market is explained by their lack of diversification.

    (Let's pretend for a moment that institutional investors always beat the market.)

    1. Anonymous2:35 AM

      Or should the Goethe Uni finding simply be understood as saying that the average individual investor underperforms the market by four percentage points?

      Do they have some way to identify/measure exactly how much of that underperformance comes from all of the other explanations (and diversification is then just the residual)?

  2. A few thoughts -

    What is diversification? Is it stocks, bonds, RE, gold, commodity? Or, it is a group of companies that may appear to be doing different things - agro, pharma, rocketman, killing fields etc?

    Almost all indexes and ETF baskets reflect an opinion of their creators; S&P 500 routinely add and subtract companies for the reasons that are not necessarily diversification but to reflect the trends and shifts in the economic activities...

    What do Indexes and ETF baskets really do? It dumb downs the volatility of individual equity in the baskets and indexes and keep the investor in a la la land of averages. They also eliminate tax-efficient investing...

    My personal experience is fairly negative in ETF, mutual funds and index investing. I have held these for ten+ years entered into them at varied times and have never been able to match the advertised performances for some mysterious reason like when you entered into the market etc.

    The best bet is to develop a sense of connectivity of activities - economical, political, and social - and invest where there is some return (dividends, distribution, gains and even the losses) that you control; and, assess your sense (pulse) of the connectivity every three months using your investments as a metric, and the only metric as it is where your skin is in the game; rest is noise and blah, blah, blah. Save your skin.

    Example: I never invested in energy until those Texans took over the country. I learned about the energy from finding to burning... I learned about investing in MLPs (did not know anything about that in 2000). I survived better than many through wars and market collapse. At the same time, financials took me to the abyss - though technical signs were many, it is very hard to give up 5% in div, and nearly 20% on initial investment... I learned how not to self delude... Take your profit once your objective is met...

    1. Anonymous9:33 PM

      It's like you didn't even read Noah's piece, or are intentionally a parody of what not to do.

  3. Anonymous10:35 AM

    Hey Noah,

    You write:

    "Now here’s a more unsettling question: What happens if too many people diversify too much? Markets aren’t just supposed to reduce risk -- they are also supposed to process information, and send capital to the companies that will use it for the most productive purposes."

    Do stock markets send capital to companies? If so, how? When I buy shares, my money goes to the previous owner of the shares, not the company.

    What about bond markets?


    1. Stock markets have some indirect effects on investment. Anticipation of future stock values affects the amount companies can raise in an IPO. Also, if your valuation gets too low you run the risk of being bought by Mitt Romney and then having to pay off the loans he took out to get the money to buy you, which makes it harder to invest, so it's better for your company if investors drive up your stock price.

  4. Without my having done the numbers myself I understand that the S&P 500 is heavily weighted towards finance and that global indexes are heavy on finance (e.g. Canada) and telephone companies.

    If the generally available indexes are top heavy in a few sectors, are there better ways to diversify?

  5. There's also the problem of not having all that much money to invest.

    You could buy an index fund, but odds are you will get socked for all sorts of fees, being a small investor. It's not as if you can choose who manages your 401k. Outside your 401k, you could try to roll your own diverse portfolio, but you'd get eaten by transaction costs, and it was worse in the good old days when odd lot trading was expensive. If you a retail investor, you probably have a lot fewer options.

    Then there is the problem of having a small buffer.

    Suppose you have managed to save a fair bit of money and have invested it wisely, then the market goes down 30%. If you have a lot of money, you just wait until it goes back up. If you don't, you are probably going to have to pull your money out of the market for a new roof, a new car, a college education, a medical emergency or to get by between jobs, as 30% down turns are rarely associated with robust job markets.

    I agree that most people are crappy investors, but if you gave me a billion dollars, I would have to do some serious screwing up to hot have enough to retire on, while if I have $50,000, I could lose it all quite easily.

  6. I've criticized some of your other posts, but I have to say that I pretty much agree with you this time.