Back in September, Frances Woolley of Worthwhile Canadian Initiative wrote one of the most telling posts I've ever read on the culture of the econ profession:
Economists like to begin with a general model, work out its implications, and then go to the data to test the theory...facts in and of themselves are uninteresting unless they illuminate economists' view of the world, and can be explained in economic language.
Psychologists begin by observing people's behaviour, looking for generalizable patterns...Yes, psychologists have conceptual explanatory frameworks, such as prospect theory, to explain such phenomena,but these theories are not the elegant mathematical creations favoured by economists...
It's a dilemma for behavioural economists. Yes, it's possible to go into the lab or into the field, run experiments, and see how people actually behave...But what do you do next?...
[I]f people's demands are just a product of framing, salience, [etc.], how can we...make any inferences about the efficiency of different [policies]?
The economic approach to policy analysis is deductive - economists begin with assumptions about consumers' and firms' preferences and behaviour...Perhaps there is some way of including behavioural considerations into that framework...But that doesn't feel right to me - if a consumer's marginal benefit from consumption is something other than what is revealed by his or her demand I could just make up anything as a marginal benefit curve, and economics would become a completely ad hoc exercise.
Dabbling in economic psychology or behavioural economics is a little like taking the red pill - you go down the rabbit hole, and wake up realizing that the entire world is an illusion.
How can economists evaluate policy if they can't use people's preferences as revealed by their choices? We still need some way of figuring out which policies help people and which policies hurt people...
Once you fall down the rabbit hole, you just have to keep on going. If people's choices are not a reliable guide to their well-being, you have to turn to something else. Ask people how happy they are and measure well-being in terms of happiness. Evaluate health care spending by looking at objective measures of health, such as mortality, morbidity, or survival rates. Chuck out the entire elegant theoretical framework of welfare economics.
That idea has me, for one, reaching for the blue pill - after all, people aren't stupid, so standard economic analysis isn't a bad approximation of the real world, is it?
But I can't find one. I can't get all of that behavioural stuff outside of my head.This post is an absolutely wonderful window into the world of the economist's mind. I'll talk about what it means for the profession in a moment, but first let me do the philosophy-of-science bit.
Science proceeds through a reciprocal process of deduction and induction. You look at the world, you try to make sense of what's going on, then you look at the world some more to see if you really managed to make sense of it. And you just keep repeating that.
It doesn't really matter whether you start with deduction or induction. If you start with a theory that you just made up because it sounded reasonable, that's fine! You just have to go test it before you believe it. If you start with some observations, that's fine too...you just have to try to make a theory to make sense of them, or your observations will not allow you to make inferences from what you observed. And making inferences - i.e., increasing humanity's power to control our Universe - is what science is really all about.
If your science is only "deductive" or only "inductive," what it really means is that you are stuck. You can't get to the next step. Either you don't have theories that match the data, or you have observations you can't explain. That's fine. Getting stuck happens all the time; you just have to recognize it and keep trying, and then maybe someday you'll get unstuck, and that's called a scientific revolution.
What you don't want to do is accept being stuck. There is a temptation for a scientist to try to convince herself that her field doesn't need one of the two steps - to say "My field is an inductive science, we don't need theories," or "My field is a deductive science, we don't need to test our theories." If a field as a whole accepts one of these seductive errors, it can stagnate for long periods of time. Some people argue that psychology is stagnating by ignoring the need for theory. Others argue that particle physics is stagnating by ignoring the need to test string theory.
At least since Kahneman started doing economics experiments back in the 70s, people have been arguing that economics has stagnated by not paying enough attention to the real world - by valuing theories that seem intuitive, or tell a good story, so highly that we refuse to admit any evidence to the contrary. Behavioral economics is a response to that criticism, as is the increased focus on natural experiments.
This has been a good development. But the battle for more attention to evidence is still an uphill one, as Woolley's post demonstrates. The profession as a whole is very enamored with the elegance of certain foundational ideas - revealed preference, Von Neumann-Morganstern expected utility, the First and Second Welfare Theorems, Arrow-Debreu equilibrium, etc. And for a long time, economists were not held to the same standards as natural scientists - we didn't have to prove our theories. Our field grew out of a literary tradition, not an engineering tradition. Smith and Hume were brilliant people, but folks in those days didn't read their books and go "Oh yeah? Well, prove it!"
And so we got spoiled. With no one forcing us to move on from deduction to induction, we were able to shy away from the cold, bitter snowstorm of reality, slam our doors, and simply choose to keep believing in things that sounded good to us. You can see the remnant of this longing in Woolley's desire to "reach for the blue pill" - instinctively, she doesn't want behavioral results to be right, she feels a primal urge to huddle close to the warm fire of classical theory.
But in the end, she can't. As our society became more scientifically oriented, the literary tradition in economics died out - a process that was aided, ironically, by the mathematics that made the foundational results of classical theory so elegant and beautiful. When you make quantitative arguments, eventually people are going to ask where they can find those quantities in the real world. The door to reality has been wedged open, and snow is pouring in. That's why Woolley can't bring herself to "take the blue pill" - for better or worse, we're scientists now, not philosophers.
The only hope for our science - for any science - is to keep alternating between deduction and induction, no matter how long each switch may take. For microeconomics, the challenge is to get a more complete picture of both human behavior and the effects of markets, and to craft coherent theories that make sense of that picture. This effort precedes apace. (As just one example, check out this paper by Dan Benjamin, Matt Rabin, and Collin Raymond, which integrates a bunch of behavioral anomalies into one interesting theory.)
For macro, though, I fear that many of us are still telling ourselves that deduction is all we need - that we should keep putting "theory ahead of measurement," that we should continue to believe in theories after the data has rejected them. That needs to stop, or we'll never get anywhere. I like to think that the awarding of last year's Nobel prizes to two macro empiricists - one who created an alternative to DSGE, and one who rejected a ton of rational-expectations models - is a sign that other people have realized this as well. But it's going to be an even more uphill battle than in micro. Macro has much worse data, after all. It's almost certain that, unlike in micro - where additions like Prospect Theory often represent only moderate tweaks to existing frameworks - most or all of our existing macro theories will turn out to be not even close to right. But we have no choice but to reject those models, or we will eventually fade into noodling irrelevance.
Welcome, economist, to the desert of the real.
(It's, you know, a snowy desert. Like Antarctica.)
Of course, prospect theory as a minor tweak to utility theory is also not even close to right! Decision Field Theory or some other multi-dimensional tweak of a stochastic decision model from psychophysics looks like a better bet for explaining the patterns and dynamics of economic choice. These models are very difficult to work with even at the individual level and going beyond that... who knows.
ReplyDelete"If a consumer's marginal benefit from consumption is something other than what is revealed by his or her demand I could just make up anything as a marginal benefit curve..."
ReplyDeleteNow, see this is where so many economists get it so wrong. You can't just make up anything and have it pass muster. It not only has to fit the buying behavior, it has to fit so much other data and evidence we have on humans and their world, biology, psychology, sociology (have you read, "The Darwin Economy" from Robert Frank). It's only when you severely limit the data you're willing to admit that you have so little power to narrow down the possibilities. Here's one of my favorite quotes from the great growth economist Paul Romer of Stanford:
In evaluating different models of growth, I have found Lucas's (1988) observation, that people with human capital migrate from places where it is scarce to places where it is abundant, is as powerful a piece of evidence as all the cross-country growth regressions combined. But this kind of fact, like the fact about intra-industry trade or the fact that people make discoveries, does not come with an attached t-statistic. As a result, these kinds of facts tend to be neglected in discussions that focus too narrowly on testing and rejecting models.
Economists often complain that we do not have enough data to differentiate between the available theories, but what constitutes relevant data is itself endogenous. If we set our standards for what constitutes relevant evidence too high and pose our tests too narrowly, we will indeed end up with too little data...
– Journal of Economic Perspectives, Volume 8, Number 1, Winter 1994, Page 20.
And here's a little parable of mine:
A snobby empirical economist is hiking with a friend who eats a berry off a bush and immediately keels over and dies. The snobby economist thinks to himself, well it's just a sample of one berry; you can't draw any conclusions from a sample of one. It's just anecdotal evidence. And being hungry himself, he eats a handful, keels over, and dies.
Moral of the story: It's very inefficient, and perhaps very dangerous, to ignore abundant a priori information at your disposal, even if that information is not formal, but still logical – the logic chains are completely solid – and based on relatively realistic assumptions. You just currently don't have a formal version of it.
It wasn't hard at all to put together a completely solid logic chain showing those berries were poisonous using common, but informal, a priori knowledge about biology, human physiology, evolution, similarity across humans, and extremely realistic assumptions.
"It's, you know, a snowy desert. Like Antarctica." so let it snow :D
ReplyDeleteanyway, among the blue and red pills, I still prefer the violet one :)
ah, incidentally a question: why economists took the path of reducing economics to psychology instead of reducing it to thermodynamics?
ReplyDeleteOk this question arises from a kind of naive theory that has been bouncing in my (non economic) mind for some time...
I mean that, if I understood it well, energy is considered just a commodity in economics. But actually energy (or work as it is called in physics) is just the base of everything even our own matter... so why not thinking energy as the actual reality of the economy, the limit of energy circulating in the economy as the only possible constraint to the economy and the whole economical apparatus just as a function of energy?
Hey,
ReplyDeletepretty soon you will notice that values have time derivatives and you build dynamic models instead of comparitive static models.
From a non economist standpoint, many economists seems to have never read the basis of using mathematics & statistics in economics ( newton developed calculus to enable his investigations not the other way round). A lot of economists have used the works of "Maurice Félix Charles Allais" which was later widely disseminated in US with out the nuances he included ( especially kahneman & travesky who completely ignored the nuances) without ever properly attributing it to him
ReplyDeletemr violet: or what about mass? why not model the economy as the circulation of mass?
ReplyDeletemore generally, how has revealed preference managed to survive so long despite being a dormitive power? Arguing that observed behaviour reveals desire is one thing, but arguing that individual preferences are both purely exogenous *and* always defined as whatever is convenient would get you whipped like a black dog on a chain in any other field of study.
Mr. Violet: There is no need to reinvent the wheel. There is enough rubbish ‘econ’ (ala econophysics) theories as it is.
ReplyDeleteNoah: Typo second paragraph: "Frances", not "Francis".
ReplyDeleteRichard: and then 100 snobby economists, seeing what has happened, say that's still just a sample of one snobby economist, and all eat the berries and die.
ReplyDeleteNick:
ReplyDeleteThanks, fixed!
Excellent.
ReplyDeleteWe engineers have it a lot easier than economists. We can build our models, semi to full scale, and test them to failure and develop models that have predictive value. I don't like the idea of economists doing the same with real people.
ReplyDeleteBuildings, bridges, machinery and air craft still fail but not often.
It would be great if economics could reach that level of success.
"ah, incidentally a question: why economists took the path of reducing economics to psychology instead of reducing it to thermodynamics?"
ReplyDeleteIrving Fisher was a student of Willard Gibbs'. What goes around comes around.
One of the things I think is a current problem is the idea that, "Macroeconomics must be firmly grounded in microeconomic." Then the proponents create deductive models and refuse to check them against the data. I think the premise is correct, but I think they need to do as the physicists did with reconciling gravity with quantum theory -- admit that the need is great but they just can't do it yet, and keep working to achieve it. The approach we should be seeing is, "This aspect of macro seems to fit the data, why doesn't it quite fit the micro theory?" instead of, "We know beyond all questioning that free markets always reach equilibrium, without fail and in all circumstances, so macro must be wrong to imply that sometimes they do not, or that the equilibrium may not be Pareto optimal." Really, I'd like to see a lot more articles like Brad DeLong's handout to his students, "The second most important thing(s) to remember," a listing of a few of the ways markets fail.
ReplyDeleteOn induction and deduction.
ReplyDeleteIn his Postscript to the logic of scientific discovery, chapter 1 (induction) section 2 (Solution of the problem of induction) subsection 6, K.Popper reformulated the problem of induction as stated by Hume:
"(i) He indicated that there are countless (apparent) regularities in nature upon which everybody relies in practice, and many universal laws of nature, accepted by scientists, which are of the greatest theoretical importance. [...]
(ii) There can be no valid reasoning from singular observation statements to universal laws of nature, and thus to scientific theories.
This is the principle of the invalidity of induction.
(iii) We demand that our adoption and our rejection of scientific theories should depend upon the results of observation and experiment, and thus upon singular observation statements.
This is the principle of empiricism[...]"
and then Popper went on stating his main idea in order to solve the problem:
"(iv) We demand that our adoption and our rejection of scientific theories should depend upon our critical reasoning (combined with the results of observation and experiment, as demanded by (iii)).
This is the principle of critical rationalism."
In a footnote there is also something interesting:
"Hume's criticism of induction was unfortunately mixed up by Hume himself with his criticism of causality, of which it is, however, logically independent. With the most notable exception of Bertrand Russel [...] commentators as a rule have failed to disentangle these two points."
@Alex
ReplyDeleteabout 'why not mass'? I don't know.
about what follows: I am sorry, I didn't understand what you wrote, I am not kidding, really didn't understand!
@DavidN
I don't think there are rubbish theories. Any theory can become much useful when explained why is false.
The only problem could be with non-falsifiable theories, these of course cannot be falsified by data, so we get into metaphysics, but again metaphysical theories can become a cue toward good science at a certain point (see the metaphysical idea of the existence of atoms), so even metaphysical theories aren't trash, it's just that scientists cannot spend all their time arguing about them...
This reminds me of one of my favorite economics quotes, from Ronald Coase explaining the difference between good economics and bad economics:
ReplyDeleteQ: What separates the good from bad, the right from wrong?
RC: The bad or wrong economics is what I called the "blackboard economics". It does not study the real world economy. Instead, its efforts are on an imaginary world that exists only in the mind of economists, for example, the zero-transaction cost world.
Ideas and imaginations are terribly important in economic research or any pursuit of science. But the subject of study has to be real.
via Thoma: http://economistsview.typepad.com/economistsview/2011/01/coase-does-not-like-the-term-coasean-economics.html
Noah, the problems which you are describing for economics are really nothing special (except for the fact that the economics profession seems to be unusually corrupt). They've played with simple models, discarding any messy reality (unless it suited their politics), and are now confronted by the fact that reality is messier than their models, and that this level of messiness is now biting us in the *ss.
ReplyDelete@ "facts in and of themselves are uninteresting"
ReplyDeleteEspecially when you have access to precious few facts.
Make that unprecious few facts. Facts are worth little if there aren't enough to test models. y=f(x)?
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteEmpirically, we have little more than the law of diminishing returns, from which we "deduce" supply and demand curves. Except they are not curves, they are smears. These smears exhibit convexity and concavity in a rough kind of way, but all we have is a rough scatter gram with precious few points. Without the assumption of normality no algebraic equations exist.
ReplyDeleteWhat does that leave? Convexity and concavity in the most generalized sense. Geometrical deductions are possible from those shapes. Useful ones even. A general tendency towards equilibrium in a free market economy for instance.
In the hands of an experience policy maker with a good knowledge of history and as many contemporary facts as are available, someone like Greenspan for instance, an intuitive feel for good policy is the most you can hope for.
No symbols on the blackboard. No simultaneous equations. No equal signs whatever. No coefficients. Greater than, less than, roughly proportional to, roughly inversely proportional to, things better indicated in prose than arcane hieroglyphics tending towards pseudo-science.
Keynes and Adam Smith both had it right.
"In the hands of an experience policy maker with a good knowledge of history and as many contemporary facts as are available, someone like Greenspan for instance, an intuitive feel for good policy is the most you can hope for. "
ReplyDeleteWe certainly hoped that Greenspan would have good policy; of course we were disappointed.
Dilbert
ReplyDeleteEconomists have built theories and tested them to failure: Russia, Zimbabwe, India (until recently). China (under Mao) and North Korea spring to mind.
What astonishes me is: (1) the lack of appreciation of the consequences of complexity (the three body problem is a lot harder than the two body problem); and (2) the poor quality of the data - given the widespread use of electronic POS systems the government should know in real time how the economy is doing. If Walmart knows how many bananas they sold in the last five minutes there is no excuse for the government not knowing how much consumers spent yesterday.
Roger
ReplyDeleteSaying "Macroeconomics must be firmly grounded in microeconomic." is like saying "chemistry must be firmly grounded in quantum mechanics and particle theory."
However superficially appealing the statement may be it is simply not true. Much useful chemistry was and is done with no reference to quantum mechanics at all.
“You are all crazy.” That is the summation of Mr. Hicks, when modern economists began using his IS-LM model for macroeconomics.
ReplyDelete“You are deluded.” That is the summary response by Mr. Solow when economists use his DSGE model for macroeconomics.
Imagine a psychologist declares she has created the first successful mathematical theory of child development. She assumes that absent trauma, children have eidetic memories, respond to all stimuli with innate Socratic rationality and perform with the emotional stability of a saint. We would declare her incompetent and encourage her to move to another line of work.
As for macroeconomics:
The preferred model has a single representative consumer optimizing over infinite time with perfect foresight or rational expectations, in an environment that realizes the resulting plans more or less flawlessly through perfectly competitive forward-looking markets for goods and labor, and perfectly flexible prices and wages.
As the great Mr. Solow asks, "How could anyone expect a sensible short-to-medium-run macroeconomics to come out of that set-up?"
Face it. Macro is silly. Always was.
Mr. Violet "so why not thinking energy as the actual reality of the economy, the limit of energy circulating in the economy as the only possible constraint to the economy and the whole economical apparatus just as a function of energy?"
ReplyDeleteCool! We could use quantum electrodynamics to prove we have high unemployment!!!
Check out R Sugden's Credible Worlds theory for an alternative understanding of the role of models. You touched on it using the word literary to describe the economic tradition.
ReplyDelete