Sunday, September 06, 2015

"The Case For Mindless Economics", 10 years on


Ten years ago, two economic theorists, Faruk Gul and Wolfgang Pesendorfer, wrote an essay called "The Case for Mindless Economics". The essay pushes back against the enthusiasm for neuroeconomics and behavioral economics. It's a very interesting read, both for people interested in philosophy of science, and anyone who wants to know how economists think about what they do. (Before you read this post, consider reading the whole essay, because there's lots in there that I gloss over.)

Gul and Pesendorfer don't discount the possibility that neurological and psychological research can be useful in economics. They write:
Neuroeconomics goes beyond the common practice of economists to use psychological insights as inspiration for economic modeling or to take into account experimental evidence that challenges behavioral assumptions of economic models. Neuroeconomics appeals directly to the neuroscience evidence to reject standard economic models or to question economic constructs.
In other words, GP aren't arguing against using neuroscience and psychology to inform economic model-making. What are they arguing against? Two things:

1. The use of neuro/psych findings to support or reject economic models, and

2. The use of neuro/psych to establish new welfare criteria, e.g. happiness.

GP's argument against using neuro/psych to test economic models can basically be summed up by these excerpts:
Standard economics focuses on revealed preference because economic data come in this form. Economic data can — at best — reveal what the agent wants (or has chosen) in a particular situation...The standard approach provides no methods for utilizing non-choice data to calibrate preference parameters. The individual’s coefficient of risk aversion, for example, cannot be identified through a physiological examination; it can only be revealed through choice behavior. If an economist proposes a new theory based on non-choice evidence then either the new theory leads to novel behavioral predictions, in which case it can be tested with revealed preference evidence, or it does not, in which case the modification is vacuous. In standard economics, the testable implications of a theory are its content; once they are identified, the non-choice evidence that motivated a novel theory becomes irrelevant.
I'm not sure I buy the logic of this argument. In general, preemptively throwing away any entire category of evidence seems dangerous to me. Why should economists only validate/reject their models based on choice data?

Here's a concrete example to help explain what I mean. In finance, there is a big and ongoing debate over the reason for Shiller's excess volatility finding - i.e., the finding that market returns are slightly predictable over the long run. Some people say it's due to time-varying risk aversion. Others say that it's due to non-rational expectations. As John Cochrane has pointed out, price data - i.e., choice data - can't distinguish between these explanations. The standard asset pricing equation is of the form p = E[mx], where p is price, m is related to utility, and x is related to expectations/beliefs. You'll never be able to use price data alone to know whether price movements are due to changes in m or changes in x. To do that, you need additional evidence - direct measures of either preferences, beliefs, or both.

That's the kind of evidence that psychology might - in principle - be able to provide. For example, suppose psychologists find that most human beings are incapable of forming the kind of expectations that time-varying utility models say they do. That would mean one of two things. It could mean that the economy as a whole behaves qualitatively differently than the individuals who make it up (in physics jargon, that would mean that the representative agent is "emergent"). Or it could mean that time-varying utility models must not be the reason for excess volatility.

So GP might respond something along the lines of: "So? Why do we care?" Of what use would be the knowledge that excess volatility is caused by psychological constraints rather than time-varying utility, if both ideas lead to the same predictions about prices? The answer is: They don't lead to the same predictions, if you expand the data set. For example, suppose you find that survey expectations can predict price movements. What once could be modeled only as randomness now becomes a partially predictable process. You can make some money with that knowledge! All you do is take a bunch of surveys, and place bets based on the results.

Are survey responses "economic data"? Are they choice data? That question is a bit academic. What if you could use brain scans to predict market movements?

In other words, I think there's not really any conceptual difference between what GP say psych can be used for ("tak[ing] into account experimental evidence that challenges behavioral assumptions of economic models") and what they say it can't be used for ("appeal[ing] directly to the neuroscience evidence to reject standard economic models or to question economic constructs"). It's all just the same thing - using evidence to create theories that help you predict stuff.

Anyway, GP's second point - that psych/neuro evidence can't provide new welfare criteria - also doesn't make sense to me, in principle. Here, in a nutshell, is their argument:
Welfare analysis for neuroeconomics is a form of social activism; it is a recommendation for someone to change his preferences or for someone in a position of authority to intervene on behalf of someone else. In contrast, welfare economics in the standard economic model is integrated with the model’s positive analysis; it takes agents’ preferences as given and evaluates the performance of economic institutions.
I don't see this distinction at all. To be blunt, all welfare criteria seem fairly arbitrary and made-up to me. Data on choices do not automatically give you a welfare measure - you have to decide how to aggregate those choices. Why simply add up people's utilities with equal weights to get welfare? Why not use the utility of the minimum-utility individual (a Rawlsian welfare function)? Or why not use a Nash welfare function? There seems no objective principle to select from the vast menu of welfare criteria already available. The selection of a welfare criterion thus seems like a matter of opinion - i.e., a normative question, or what GP call "social activism". So why not include happiness among the possible welfare criteria? Why restrict our set of possible welfare criteria to choice-based criteria? I don't see any reason, other than pure tradition and habit.

So personally, I find the logic of both of GP's main arguments unconvincing. In principle, it seems like psych/neuro data could help choose between models when choice data is insufficient to do so. And in principle, it seems like neuro-based or psych-based welfare criteria are no more arbitrary than choice-based welfare criteria (or any other welfare criteria, like "virtue").

But that's in principle. What about in practice? It's been 10 years since GP's essay, and many more since psychology and neuroscience entered the economist's toolbox. Psychology seems to have made real contributions to certain areas of economics, in particular finance. In general, those contributions have come in the form of generating hypotheses about constraints - for example, attention constraints - rather than by motivating new behavioral assumptions for standard models. In other words, psych ideas have occasionally given economists power to predict real data in ways that standard behavioral models didn't allow. These contributions have been modest overall, but real.

But I can't really think of examples where neuroscience has made much of a successful contribution to economics yet. That might be because neuroscience is still too rudimentary. It might be that it has, and I just haven't heard of it. Or it might be that it's just incredibly hard to map from neuro concepts to econ models. In fact, GP spend much of their essay showing how incredibly hard it is to map from neuro to econ. They are right about this. (And in fact, GP's essay should be required reading for economists, because the difficulty of mapping between disciplines really gets at the heart of what models are and what we can expect them to do.)

Also, in practice, no psychology-based welfare criterion, including happiness, has gained much popular traction as a replacement for traditional utilitarian welfare criteria based on choices. So while welfare is a matter of opinion, most opinion seems to have sided with GP.

All this doesn't mean I think neuroeconomics is doomed to be useless, just that it seems like it's in its very early days. There are a few hints that neuro might be used to select between competing economic models. And the topic of using happiness as a measure of economic success occasionally crops up in the media. But the task of using neuro (and psych) for economics has turned out to be much harder than wild-eyed optimists probably assumed when the fields of neuroeconomics and behavioral economics were conceived.

So I think that while Gul and Pesendorfer didn't make a watertight logical case, their warnings about the difficulty of using neuro evidence for econ have been borne out in practice - so far. Ten years might seem like a long time, but let's see what happens in forty years.

29 comments:

  1. Justin Wolfers makes the case there is not a wide gulf between using happiness surveys & an "easily" quantifiable statistic like unemployment. For example, UE relies on survey questions to determine whether you are in the labor force.
    My major problems with happiness surveys are the measure are ordinal. There is no reason to think the distance between 3 to 4 is similar to 1 to 2. Scales are arbitrary. The weights given to individual questions are often arbitrary as well. Another problem is that there is no reason to think the answers and interpretation of questions is independent of age. I doubt a 20 year old 80 year old think the same answering "Iam satisfied with my life."

    ReplyDelete
    Replies
    1. I'm not sure that ordinality, per se, is a big problem.Most econometric analyses of happiness data find that there is no substantive difference in the conclusions one reaches regardless of whether one treats the data as ordinal (with something like an ordered probit) or cardinal (and just uses least squares - Carbonell and Frijters, 2004). This suggests that the data behaves as though it approximated cardinal values in usage. More to the point, the statement "happiness data is ordinal" is more something economists say, rather than a conclusion with a strong empirical backing (since this is a hard thing to prove empirically). The issue of correlation between scale use and other criteria such as age or cultural background is, however, a potentially more serious issue that is harder to address.

      Delete
  2. Surveys have also been used for some time to differentiate changes in attempting to resolve the problem of time-varying risk premia versus irrational expectations, notably studies in the late 80s by Frankel and Froot for the especially problematic forex market data. Their findings were in fact pretty clear that risk-premia simply do not come close to sufficiently time varying enough to explain the volatility, but, of course, one can dismiss survey data on lots of grounds.

    I think PG overstate the lack of mapping from neuro to econ, although that mapping does not provide nice quantitative links. However, we are now reasonably certain that the time inconsistency involved in hyperbolic discounting can be explained by the apparent fact that different parts of the brain are involved in shorter versus longer time horizon calculations. There are some other such findings, but these links seem to be more qualitative than strongly quantitative in nature.

    Of course these last arguments not of interest as they involve something that is already "irrationally behavioral," but so it goes.

    Barkley Rosser

    Barkley Rosser

    ReplyDelete
  3. "The individual’s coefficient of risk aversion, for example, cannot be identified through a physiological examination; it can only be revealed through choice behavior."

    What a bad example. You in fact cannot in fact identify risk aversion through choice behaviour, unless you know how the subject sees the choice (i.e. choice mixes up risk aversion and risk perception). The arrogant assumption here is that risk is knowable and known (sounds familiar some how).

    ReplyDelete
  4. Soren9:19 AM

    Noah,

    Unrelated to this post, but since you write quite a bit about economics as a science (among my favorite blog posts of yours): I'd be interested in your take on how economists can derive almost diametrically opposite conclusions from the same set of observations. Here I am thinking about Krugman's recent post on "The Triumph of Backward-Looking Economics" vs. Cochrane's "Historical Fiction" (following up on a post by Stephen Williamson).

    It is curious how they can almost completely disagree about whether a particular simulation scenario, a "cautionary tale" in Tobin's words, was good or not. Krugman argues that it was good because Tobin got the "basic shape", the qualitative features, right. Cochrane says it is off because of the quantitative magnitudes. And so Krugman views it as a vindication of standard Keynesian macro, Cochrane quite the opposite.

    What should a casual follower of economics debates make of this? Presumably, the models of the economy to which Krugman and Cochrane respectively subscribe give rise to fairly different predictions about the effects of a policy. They also seem to agree that their models disagree in their predictions; but one claims that the facts support his model and not the other's and vice versa. It is most confusing.

    ReplyDelete
    Replies
    1. "What should a casual follower of economics debates make of this?"

      I'm a layman who's been reading macro blogs since 2010: Sumner, Rowe, Krugman, DeLong, Smith, Glasner, Cochrane, Koning, Nunes, Wren-Lewis, Williamson, Andolfatto, Roche, Cowen, Farmer and recently Romer (are my main ones) ... and occasionally Murphy, Keen,Giles or others.

      Here's what I make of it (to answer your question). Macro economics is clearly at the place in it's evolutionary history that medical science was in the 13th century: you've got your cuppers, your ear candlers, your blood letters, etc, and perhaps even a person or two who's actually on to something. They have centuries of struggle ahead of them to sort it all out. Unfortunately, the whole group of them gets called on every time the king/economy gets sick, because... well, what else are you going to do? These are the "experts" and however bad they suck, we're stuck with them.

      Delete
    2. ... but if it's truly a science (or on it's way to becoming a science), then reality will eventually sort it all out. Even if the answer converges to "we can't know."

      Delete
    3. ... or simply "we don't know."

      Delete
    4. Anonymous10:02 AM

      Better to ask us then crackpots like Tom Brown and that weirdo with the information transfer garbage.

      Delete
    5. Lol, my point is you can tell just be the massive level of disagreement in the science amongst experts (as opposed to mature sciences like physics, biology or meteorology), that macro appears to have several centuries ahead of it before it becomes more like a science. I have no idea if ITM is onto something, nor any other at this point. There's nowhere near a consensus.

      Delete
    6. M Davies12:23 PM

      Tom,

      I disagree somewhat with this idea that the macroeconomics is worse at making predictions or diagnoses than the sciences you mention because it less "mature". I'd be more prone to say that it is because people (and the systems comprised of them) are in some ways inherently more unpredictable - i.e. I doubt you could ever have laws of social science comparable to laws of nature, regardless of how much more macroeconomics develops.

      M

      Delete
    7. @M Davies, if that's the case, then rather than having various camps of economics who all seem very certain of themselves, perhaps they should be concluding that they don't deserve to be certain of anything. Instead we have the ear candlers yelling at the blood letters and vice versa as if either one of those groups can have more than 5% confidence in what they're talking about.

      Also, if what you say is true, ideally the search would go on forever, but confidence levels would only get lower and lower in any particular models, frameworks or theories.

      Delete
    8. ... but if what you say is NOT true, that indefinitely long search may *eventually* result in something useful. I don't see any reason it can't... it's not like the few pounds of deterministic meat we carry around in our skulls can really be much more complicated (if at all) than something like the Earth's weather system.

      Delete
    9. ... and perhaps reductionism is the wrong way to go (digging into psychology and choice and so forth).

      Delete
  5. Anonymous11:23 AM

    What I found interesting is GP's answer to Thaler and the like, if people's choices are mistakes. Eg compare GP's welfare criteria of their temptation preferences that is opposite to the criterias normally used for hyperbolic discounting. Hyperbolic discounting is seen as a mistake while temptation is part of the preference and rationalizes dumb decisions.

    I am not sure if it helpful that in econ you can always come around with a model that supports your maybe biased view about the correct welfare measures, because this way you have a bunch of literature about both theories that coexists with no sign of convergence. And I also think that GPs refusal of neuro data only leads to more polarization within theory (like you Noah, I am also confused about GP at this point).

    Still I like that they highlight the somewhat arbitrary choice of welfare criteria in Behavioral Econ even though I see no reason why they should be in any way more objective the others (this may also highlight the need for more discussion between philosophy and econ, because most of the time criteria are presented and accepted as being objective in some way).

    ReplyDelete
    Replies
    1. Anonymous10:05 AM

      If my preferences are in fact hyperbolic, why should I be making a mistake trying to maximize them? The idea that preferences must discount geometrically is absurd, preferences are whatever they are. Temptation is simply a tractable variant of hyperbolic discounting that does not require the solution of a game with literally a continuum of subgame perfect equilibria, in which I can find a solution that does practically anything I want. With temptation the welfare function is clear, with hyperbolic discounting you have to decide how to weight "future selves". And Noah again displays his lack of knowledge about economics, since he clearly doesn't understand Arrow's theorem or more modern versions like Gibbard-Satterthwaite. The world would be better off without this inane blog and its idiot non-savant writer.

      Delete
  6. Noah, what do you make of this study:

    "An Economic Framework of Microbial Trade"

    Here are the final sentences from the abstract:

    "We experimentally tested this tradeoff using a synthetic consortium of Escherichia coli cells and found the results match the predictions of the model. This framework provides a foundation to study natural and engineered microbial communities through a new lens based on economic theories developed over the past century."

    There are no minds (let alone any brain cells) involved at all.

    ReplyDelete
  7. Anonymous5:39 PM

    Your point about utilitarian versus Rawlsian welfare functions is not particularly germane: GP are proposing using revealed preference/utility function information to assess individual welfare, rather than supporting any particular aggregation method. That is, you seem to be confusing the information used to assess individual welfare with the choice of a social welfare function.

    ReplyDelete
    Replies
    1. I don't think you get my point. No matter what information a social welfare function is based on, it is still an arbitrary choice, and its choice still falls under what GP call "activism". Welfare function choice is normative, normative = morality, and morals are opinions.

      Delete
  8. Anonymous6:37 PM

    My point is that GP are not advocating or defending the choice of any particular SWF (beyond the Pareto criterion which they take to be noncontroversial). Rather, they are arguing that there is no case for using non-choice information to assess individual welfare in economics. In any event, there seems to be no reason to think that neuroeconomics can be of any help in choosing between alternative SWFs.

    ReplyDelete
    Replies
    1. Rather, they are arguing that there is no case for using non-choice information to assess individual welfare in economics.

      I know. But that assertion is just pure opinion. It's GP saying "We don't think this is a good welfare criterion." They present choice-based welfare criteria as "positive" and all other welfare criteria as "normative", which is wrong.

      Delete
    2. Anonymous6:59 PM

      All right, I think I agree with you on that point. Your discussion of aggregation or SWFs has nothing to do with it, however, so was just a red herring. That's all I was trying to point out.

      Delete
    3. I pointed that out just to show that SWF choice is always opinion. Which I think is an important point!

      Delete
    4. Anonymous7:15 PM

      Enough with the red herrings! Actually, I understood GP to be saying that revealed choice is an appropriate criterion for evaluating individual welfare changes in economic models & rejecting claims that different types of psychological information are needed or useful. I didn't understand them to be saying that it is a purely positive criterion. But perhaps I missed an important sentence somewhere.

      Delete
  9. If I understand you right, Noah, the heart of what you're getting at is that choice data doesn't reveal "utility" or "real value" or happiness or living standards or any of those sorts of concepts anyway, because it only tells what people choose not why or what good it does them, so rejecting the power of neuroscience to do it as well as choice data doesn't make sense. Agreed so far. I'm generally on the side of making fewer assumptions and getting more data, and I don't see why neuro data is up front invalid.


    But this paper seems to be largely fighting against a different proposal, that neuroscience can reveal what people ought to choose to make themselves happy even when it's different from what they actually choose. Of course we've all made purchases we've regretted so it's not a frivolous proposition. But I also understand being wary of it. I guess I'd have to reserve judgment till I saw an actually potentially influential neuroeconomics finding.

    ReplyDelete
  10. Great articles , Enjoyed


    Thanks
    Paytubs online
    http://paystubsonline.net/

    ReplyDelete
  11. Anonymous7:14 AM

    "So while welfare is a matter of opinion, most opinion seems to have sided with GP"

    This depends on what the goals of welfare analysis are, right? If you're interested in Kaldor-Hicks efficiency and potential Pareto improvements, then sticking with revealed preference and ordinal utility seems reasonable. I would think of this as a value-free exercise: if outcome B is a potential Pareto improvement on situation A, this is informative but doesn't mean we necessarily ought to do it - economics is a positive social science. I don't think anyone should jump directly from a cost-benefit analysis to a policy prescription.

    If you're instead interested in welfare in the sense of what constitutes the good life, and wants to specify some function actually capturing how well off people's lives are, then that's different. But it's a philosophical question. I'm not sure what you mean by "...welfare is a matter of opinion." - if you're saying there's no fact of the matter about what the good life is, then that could be something worth arguing explicitly. Maybe it's hard to figure out exactly the right criteria (see http://plato.stanford.edu/entries/well-being/) but that doesn't mean there's no right answer. Obviously happiness is going to play a large role in any good theory of well being, and suffering is not.

    ReplyDelete
  12. Paul Glimcher at NYU is at the verge of proposing new expected utility models that map neuro into a regular utility framework. You can see an intermediary step here: http://fbe.unimelb.edu.au/__data/assets/pdf_file/0007/954592/Ryan_Webb.pdf

    ReplyDelete