Wednesday, July 03, 2013

Do beliefs contain useful information?



The "Do bets reveal beliefs?" discussion continues. This is very interesting to me, since in finance experiments, you often depend on bets accurately revealing beliefs.

But lost in the mayhem of the debate is a second question: Do beliefs contain useful information in the first place?

In an econ experiment, of course they do, because researchers want to understand how humans process information. But how about in the real world? Take my bet with Brad DeLong. Suppose that bet did reveal my true belief, i.e. that inflation is going to spike. Suppose I really really really believed that, very strongly. So what? The belief of Noah Smith, no matter how strong, tells you incredibly little about the future path of inflation that the market for TIPS didn't already tell you.

Now, maybe there are exceptions to this. Suppose you have a well-respected, widely trusted expert in monetary economics, such as Steve Williamson. Last March, Steve Williamson confidently predicted a near-term spike in inflation, despite low TIPS breakevens, claiming that his understanding of monetary economics gave him private information that the market did not possess. If you believed that expert prediction, and increased your inflation hedging accordingly, then you lost money. Perhaps you are mad at Steve for losing you money, and you sulkily suspect that maybe Steve didn't really believe his own prediction. You wish that Steve had been forced to somehow reveal that he really, truly believed that inflation would spike, and was not pulling your leg.

But even in the case of experts, I think you need to be very, very confident in the expert's record before you give special weight to that expert's opinion. For example, Michael Boskin is legendary for getting every major macroeconomic prediction wrong since the beginning of time (update: Scott Sumner dutifully informs us that some of Boskin's so-called "predictions" were actually just implausible and unverifiable explanations for things, not true predictions). Paul Krugman is somewhat ahead of the average of pundits, though it's a small sample. Robert Shiller has an impeccable record of bubble prediction, but that sample is even smaller.

So the only time beliefs reveal useful information about financial outcomes, such as inflation, is if you trust a very special expert. If that expert is pulling your leg, then you have a problem. But I contend that this situation is very very very rare, because reliably market-beating experts are very very very rare.

So we see that the situations in which (economics) bets can most easily be made - concrete, financially important outcomes - are not only the situations in which bets are least likely to reveal beliefs (because either hedging or making the same bet with better odds is always possible using public markets), but also the situations in which individual beliefs are the least likely to contain useful new information.

But I suspect that the bet advocates (Alex Tabarrok and Bryan Caplan) want to extract a different kind of information from bets. I suspect -  and if I'm wrong, please correct me - that they are concerned with the ulterior motives of people who advance economic theories. For example, perhaps they suspect that Keynesians don't really believe in Keynesian business cycle models, but want increased government spending for the sake of redistribution. Or perhaps they believe that inflation hawks don't really believe their dire warnings of inflation, but want higher real interest rates for the sake of redistribution.

In this case, forcing an economist to reveal his or her true beliefs might be very useful. If it was revealed that an economist didn't really believe in the quantitative predictions of a theory that (s)he had spent a great deal of time and effort promoting, then that might be a signal that the economist had promoted the theory because of some ulterior motive.

And knowing the ulterior motives of would-be experts can be very useful information in situations in which public financial markets can't give you an answer. For example, suppose Economist A says "The Fed should lower interest rates to boost the economy! I know this because of my New Keynesian model." And Economist B says "Lower interest rates will just lead to inflation without boosting the economy! I know this because of my RBC model." And suppose that the New Keynesian model also just happens to predict lower inflation over the next 6 months, while the RBC model also just happens to predict higher inflation. In that case, forcing Economists A and B to make a bet on upcoming inflation might - or so Tabarrok and Caplan seem to hope - be able to reveal whether one or both of these economists actually has little confidence in his own theory. That in turn would reveal that that economist very possibly had some ulterior motive in advocating for that theory in the first place, which in turn would tell you not to trust that economist on policy matters in which the ulterior motive might apply.

This is what I think Alex Tabarrok is hoping when he says that a bet is a "tax on bullshit". He wants not to extract useful information about financial markets, but to reveal the ulterior motives of disingenuous public intellectuals, and thus get a better idea of whom to trust in the Marketplace of Ideas.

This, I think, is an excellent goal. Some kinds of bullshit are useful and should not be taxed, but disingenuous theory-promotion by public intellectuals seems to produce a negative externality that should be taxed away. The question of whether public bets are the tool to accomplish this goal, however, seems to hings on a lot of the questions Tyler Cowen has raised, such as what a public intellectual really risks when making a bet.

31 comments:

  1. Argosy Jones12:35 AM

    I think Tabarrok's line of thinking is fatally flawed.

    Making bets might well become a means of making a profitable line of bullshit seem more convincing.

    Q: How big a bet would I have to place against Peter Schiff to get him to back off his bullshit on gold?

    A: The bet would have to be so large that I could never hope to cover it if I lost.

    ReplyDelete
    Replies
    1. Yeah but you don't need to.

      Peter Schiff most probably (99%) really believe his stuff on gold and Euro Pacific Capital is most probably positioned accordingly.

      IIRC, as the subprime crisis evolved, Schiff looked vindicated for a little while... until you realised that his bets (foreign stocks and foreign currencies) were going to get him buried despite him having been right about the US real estate... And you do see it in the poor performance of his mutual funds...

      Delete
  2. I really want to know what forms of bullshit are useful! Do you mean national borders, currency, things like that?

    ReplyDelete
    Replies
    1. Fwiw, belief in "god" is arguably the greatest proof that belief in bullshit can be extremely useful.

      Delete
    2. Yes, all of those are incredibly useful forms of bullshit.

      Delete
  3. I still don't get what is so complicated about Alex and Silver's initial point.

    Would Mitt Romney bet that tax cuts to the wealthy translates into higher government revenues through 'trickle-down'?

    Will Scott Sumner bet with me that the all-in Japanese QE policy (if they keep it) will not result in rises in RGDP per capita above the present baseline?

    ReplyDelete
    Replies
    1. That's nonsensical for a couple reasons, the first being that "trickle-down" is a strawman. But let's say Romney was willing to bet his entire fortune on the notion that total government spending significantly smaller than than today's (say 25% of GDP instead of today's 35%) creates significantly more economic growth over the next decade (a proposition for which there is some empirical support, but also considerable dissent from people Romney generally disagrees with). How does he accomplish this? Even if he does, how do you measure the counterfactual?

      Delete
  4. Anonymous5:51 AM

    It is hard to get any meaning out of this without a precise definition of "to believe something". I like the way Tetlock approach this better. He asks you to assign a probability (number) to the outcome. Anything else is just playing with words. What people mean with "I believe" or "x will likely happen" varies wildly.

    ReplyDelete
    Replies
    1. Phil Koop9:56 AM

      It is hard to get any meaning out of this comment without a precise definition of "probability". As it happens, the betistas are working with some version of de Finetti's: your probability is the odds at which you are willing to make a small bet. So they have your particular criticism well spiked.

      Noah's is a different story.

      Delete
    2. This is a really interesting philosophical question. Someday I'll do a post on this.

      He asks you to assign a probability (number) to the outcome. Anything else is just playing with words.

      I assign a probability of 478.2 to the outcome that inflation rises. So what? All that shows is that I'm an idiot. ;-)

      your probability is the odds at which you are willing to make a small bet

      Ah, but what if my bet can be hedged in other markets? ;-)

      Delete
    3. There are a couple of points here.
      (1) Useful information from beliefs. The post discusses useful information in the sense of (a) useful predictions of the future and (b) ulterior motives of individuals. Leave aside (a), which depends on whether experts are useful in predictions. There are more kinds of information that can be determined from beliefs than just people's motives. Someone who believes that something has a probability of 478.2 is indeed an idiot. That might be useful information about him. Equally, if you find out that someone believes in creationism, that the gold standard would be economically beneficial but that immigration would not, then that is useful information in trying to predict their likely voting behaviour. All sorts of information can be useful.
      (2) Prior question: what is belief in a statement? That's a philosophical question, but for social science purposes all you need is a tendency in a person to act in a manner which is overall more consistent with the statement being true than with its converse being true. "What if my bet can be hedged in other markets?" you ask. Well, if it can be hedged but you don't hedge it, or if it can't be hedged and you bet anyway then we can infer something about your beliefs. We need to know your overall net position to infer your beliefs, but I don't see any theoretical obstacle to doing so (practical obstacles set out below).

      Delete
  5. "disingenuous theory-promotion by public intellectuals seems to produce a negative externality that should be taxed away". Perhaps. But wouldn't you also be taxing away tentative theory-promotion and less confident public intellectuals, which are probably both things we need more of? Say you are 55% confident that your theory is correct. Even if you are wholly sincere, you might not want to put a career-risking bet on that, but that doesn't mean we shouldn't hear the theory.

    Also, most of these kinds of theories (what is going to happen to inflation?) are of the 'other things being equal then ...' kind. Should the bets be tightly defined by reference to what those other things are (and what counts as 'equal')? Or should bettors have to factor in to the odds they accept both their assessment of the risk of, e.g., a supply shock as well and their confidence in their theory?

    ReplyDelete
    Replies
    1. Two good points but bets, imho, are meant to reveal indeed the degree of confidence you have in your own theories. Sure, you could ask a number directly but few econo-pundits are going to tell you 'oh, well, actually, it's kind of complicated so I think my ideas are only 55% likely to be correct'. No, they'll go on and on... unless (maybe) they are forced to stake their careers on it.

      Of course, that's when the ceteris paribus condition comes in handy. You were widely wrong about hyperinflation breaking out?

      Two solutions: Say it's still early days. Or say something else happened to transform the picture.

      Again, depending on how the bet is phrased, it can help. Did not some Austrians bet on hyper/inflation within some years we've now exceeded? And they are thus now forced to say 'well, something else is happening'...

      It's not as good as them recognizing defeat but, again, it makes it easier for the viewing public to see them for what they are: Bullshitters trying to peddle bullshit theories.

      Delete
  6. NB: To address the actual point of this blog-post. 'Do beliefs matter?'

    Yes, they very much do.

    It's not that I am going to trust an expert even if he is very confident about his theories. Economic sciences being what it is, 'experts' are pretty useless at prediction.

    It's that the public discourse and the public policies do get affected by economic discourse and, in order to be able to choose between various theories, it's good to force those theories to be put to the test.

    Since controlled experiments are hard to come by in Macro, having economists make predictions is an alternative.

    As to whether some people have ulterior motives in promoting a theory, sure, it's true. But, here, I am less sure that bets help. I'd rather analyze actions... For example, I can tell that the Republicans don't care about deficits and debt by the way they were willing to run up deficits during GWB decade and that their real agenda when mentioning debt/deficit is entitlement 'reform'.

    I can't quite think of a bet that would force them to acknowledge that. Hell, R&R 90% being shown to be BS, the causality debt/low growth being shown to run the other way as well as the CBO revising its projections didn't change the discourse one iota.

    Still, hacking the intellectual support for these positions is important. Hopefully, it'll make sure they get small enough to be drown into the bathtub instead of the government... :)

    ReplyDelete
  7. "I can tell that the Republicans don't care about deficits and debt by the way they were willing to run up deficits during GWB decade and that their real agenda when mentioning debt/deficit is entitlement 'reform'." <<
    This is pretty big bullshit -- there were plenty of Reps, quite a large minority, who didn't like Bush's debt increases thru more spending.
    Why not look at the TARP bailout vote, for instance? More Reps opposed it than Dems.

    Far more Reps spoke out against Bush spending than Dems speaking out against Obamacare or the stimulus spending (send tax money to Dem donators! Like at Solyndra. Krugman thinks we needed more than the waste we got.)

    Debt increases thru tax cuts are different -- and historically lead to higher future growth.

    The really really big bet was Dr. Michael Bury (from The Big Short), who correctly predicted the drop in house prices and huge potential gain on shorting MBS based CDO stuff. Like Warren Buffett, the investment bets being looked for are those of undervalued companies. But these are, usually, micro based, not macro. (Michael knew that when he would win his bet, the US economy would tank. See his UCLA grad speech from 2012.)


    Inflation remains looking tame, Noah. Patrick's bet is looking bad; but the mid east could still have a few nukes going off before the 2 remaining years.

    ReplyDelete
    Replies
    1. Debt increases thru tax cuts are different -- and historically lead to higher future growth.

      Can you supply a citation for this?

      I think it is absurd for you to cite responses to the financial crisis as evidence for the general attitude of Republicans towards deficit spending. Did Ronald Reagan triple the national debt or not?

      Delete
    2. http://en.wikipedia.org/wiki/File:US_Debt_Trend.svg

      [bi-partisan trend comes to an end beginning 1980]

      Delete
    3. So, despite what it may seems, I wasn't looking to turn this into a political fight i.e. you may be right that Dems are even more relaxed than Reps about debt/deficit. You may even be right that "some" Reps have an ingrained dislike of debt/deficit and did criticise GWB for running the tab.

      Still, the tax cuts passed and the money was spent.

      As to your point about tax cuts and growth, this is exactly the stuff bets are good for.

      I'll bet you that tax cuts for the top 1% do NOT improve future growth. I would be willing to bet a meaningful chunk of my income on that and/or just get famous when I am proven right.

      Two issues: One is on the bet itself. How do we keep things 'ceteris paribus' so that the effect of tax cuts is undeniable and you're forced to recognise your error?

      The second issue is that I don't think winning this bet would make me famous. Too many people already agree with me and I am not already famous...

      Delete
    4. Also, the GOP tried to pass a Balanced Budget Amendment in the 1990s.

      "I'll bet you that tax cuts for the top 1% do NOT improve future growth"

      You've left that so open it's already wrong. Assume 100% tax rates -- does lowering that rate improve growth?

      Delete
  8. I see the main advantage of betting being accomplished without betting at all, just a clear decidable quantifiable time specific prediction, and I see the reputational wager as greater than most monetary ones, and also fear their risk adverseness may overcome their best estimate belief leading to an understatement of their confidence. Betting does force a clear prediction though, which is all too often omitted entirely. Now they may not pay a high price for being wrong, but I will forever disregard anything Kevin Hassett has to say for example.

    ReplyDelete
  9. But what is the "market" set by, but bets made on beliefs? In some areas, the aggregate of a number of beliefs may be more accurate, in others less so. A "correct" price rests more or less on a circular chain of belief - it's what what people pay based on their beliefs on what people pay. Physics less so - a poll, or even a market in where the planets are going to be is not going to tell you as much as an expert (but consider the sums paid for astrological advice).

    And bets are not good indicators - lots of people routinely bet much more than money (lives, careers, families) and get it very wrong indeed, often in great herds.

    ReplyDelete
    Replies
    1. It's not entirely circular, some beliefs do get falsified -- e.g. Solyndra isn't going to be an engine of economic growth and anyone who bet that way (e.g. taxpayers) lost their money by paying the wrong price. Markets correct prices over time so resources flow to correct bets and away from incorrect ones.

      I think a market would outperform an expert in astrological prediction, if there was any reason for such a thing to exist. I doubt prices would ever move much off their calculated values except in realms of genuine uncertainty and then I suspect they would outperform most experts.

      Delete
  10. isn't this all kelly Q and the strength of the signal? bets don't matter unless they are material, if you can't make the bet material enough to matter then that tells you the strength of the signal, or your relative implied price of your signal vs. the person you are betting against... which btw could all be complete bull or just emotionally / idealistically driven.
    I'll bet a dinner on a master's pool b/c it's fun and i don't care, i would not bet a car on the master's pool b/c the strength of my signal is flat to weaker than almost everyone i'm willing to bet against. However on the public scale i think the bet holds a lot more water b/c it's a measurement of your credibility. so what's your street cred worth?

    ReplyDelete
  11. I see someone already mentioned Tetlock 2005. The answer seems to be "not information with significant predictive power."

    ReplyDelete
  12. Anonymous11:59 PM

    Re: I still don't get what is so complicated about Alex and Silver's initial point.

    It's not complicated, it's just wrong. These sorts of "public bets" aren't actually useful and do not, for a wide variety of possible reasons, necessarily reveal that much about the beliefs of the people involved. It's really just wankery.

    Re: Would Mitt Romney bet that tax cuts to the wealthy translates into higher government revenues through 'trickle-down'?

    He very well might, if he perceived the value of creating the impression that he believed that to be greater than the cost of losing the bet.

    ReplyDelete
    Replies
    1. And, of course, such a bet would be trivial - remember his offer of a $10K bet with Perry?
      (in Romney's world, that was probably called 'standard Saturday morning golf bet')

      Delete
    2. You're both right inasmuch as Romney probably wouldn't care about making the bet, whether he thought it was likely to win or not.

      But say he made the bet and then was proven wrong (elections and re-elections long gone)...

      I think that someone could come back and, next time he talks about the tax income-generating tax cuts, cut him down to size.

      'sounds like a useful thing for improving the public debate.

      Delete
  13. Noah: "Now, maybe there are exceptions to this. Suppose you have a well-respected, widely trusted expert in monetary economics, such as Steve Williamson. "

    BWAHAHAHAHAHAHHAHAHAHAHAHAHAA!!!!!!!!!!!!!!!!!!!!!!!!!!11

    I've seen him here and (IIRC) on Brad's blog (or another econoblog); he's a blithe liar, pure and simple. He's a guy who's been calling for a spike in inflation for several years now, and when confronted with his previous predictions doesn't even try to spin them anymore.

    ReplyDelete
  14. This raises the obvious question of how you determine someone's beliefs. There is often a big difference between what someone believes and what someone espouses. How many anti-gay preachers have been caught with their pants down in a motel room with a teenaged boy? How many brokers arranged to sell AAA+ rated derivatives knowing, not just believing, that they were nearly worthless?

    A lot of people espouse things, because there is good money is espousing things. There is no need for them to actually believe them, or at least not to act as if they believe them. As long as they can convince third parties to believe what they espouse, they can make money.

    If you want to know what people truly believe, watch what they do. In this case, what people believe actually does matter, because if enough people believe in the bull market or web stocks or McMansions, their price will rise. If you just listen to what they espouse, all you know is that they believe that saying such things is to their advantage.

    ReplyDelete
  15. This is a topic highly related to real world trading.

    Good trades have at least this much known about them:

    1. Odds of win/loss
    2. loss size if loss
    3. payoff with win
    4. Frequency of bet (how frequently can the bet be made)
    5. Time between bets
    6. Counterparty reliability

    When Alex calls these a tax on bullshit, it seems to me that this is very, very difficult to actually implement because of different levels of wealth between counterparties. For example, lets say you (Noah Smith) have a bet with Greg Mankiw. Greg Mankiw is wealthy. I am assuming you are not. Additionally, Greg Mankiw has a huge reputation, and he's a Harvard prof. You have a good reputation, but it's still growing.

    What kind of bet could be a tax on bullshit for Greg Mankiw that would be relatively equal for you and Mankiw? Remember, he's already survived being part of the Bush administration. He's got a ton of money. He's got A-list reputation. My guess is there is no bet he would take that would seriously impact his wealth, and there is no bet that would impact his reputation if he can survive the Bush years.

    It's hard to make trading contracts which have the intended outcome.

    ReplyDelete
  16. I confess that every time I encounter someone in the midst of an intellectual discussion or debate resorting to "you wanna bet on it?" my reaction is that this is just asinine macho posturing and game playing. Sure, we should take seriously what people really do put money down on as indicating their beliefs, but most of these public bets are indeed just pathetic attempts at proving masculinity.

    ReplyDelete