Friday, November 20, 2015

Unlearning economics


"Just then the floating disembodied head of Colonel Sanders started yelling
Everything you know is wrong"
- Weird Al Yankovic

The problem with discovering how the world works by intuition alone is that the model space is just enormous. Humans are really really smart, but very rarely are we smart enough to just sit down and think about how the world might work and actually get it right. Our most spectacularly successful leaps of theoretical insight - Newton's Principia, Einstein's relativity stuff, Mendel's theory of inheritance - were all very closely guided by data. The general pattern was that some new measurement technology would be invented - telescopes, plant hybridization experiments, etc. - that would provide some new unexplained data. Then some smart theorists would come up with a new theoretical framework (paradigm?) to explain it, and the new framework would then also explain a bunch of other stuff besides, and so people would switch to the new theory.

How about econ, though? Auction theory - a big empirical success - seems like it came out of pure intuition (though that could just be my ignorance of history). But if econ is like natural sciences, then we should expect that sort of success to be very rare. Usually, theories developed from armchair intuition - no matter how mathematical - will be wrong, just because wrong theories outnumber right theories by such a huge margin.

Right now we're in the middle of an empirical revolution in econ, and - unsurprisingly - a ton of standard, common theories are just not matching reality very well. For example:

1. If you slap some quick supply-and-demand graphs on the board, it looks like minimum wages should harm employment in the short term. But the data shows that they probably don't

2. If there's any sort of limits to mobility, then simple labor demand theory says that a big influx of immigrants should depress the wages of native-born workers of comparable skill. But the data shows that in many cases, especially in the U.S., the effect is very small

3. A simple theory of labor-leisure choice predicts that welfare should make recipients work less. But a raft of new studies shows that in countries around the world, welfare programs barely reduce observable work effort.

4. Most standard econ theory doesn't assume the existence of social norms. But experiments consistently show that social norms (or morals, broadly conceived) matter to people. 

Again and again, standard ideas - the stuff that most of the undergrad kiddos learn in their Econ 101 classes - are being smacked down by the heavy hand of new data. We're slowly unlearning economics.

Of course, much of the new empirical literature will eventually prove to be non-replicable or poorly designed, but much of it will prove to be solid and reliable; generations of empirical economists will carefully replicate findings and try to extend them to other contexts to see how well they hold up. And the good results will hold up, and theories will be killed as a result.

(Ultimately, that's good for econ theory, since the usual pattern is for better data to produce better theories. And it means more work for econ theorists, since simple theories will be killed first, and the better newer theories will take a lot of effort to make - so there will be jobs and money for theorists. Still, I expect some theorists to complain, because making simple theories is easy and making useful theories is hard.)

But anyway, what this means is that Econ 101 courses around the country probably need an overhaul. New data is rapidly producing a raft of new theoretical facts that students should be learning. Teachers should still teach the simple, classic theories that the new facts are beginning to kill...but mainly as a way to show how data can tell us when we're wrong.


Update: Here's a follow-up Bloomberg post. Maybe Angrist and Mankiw should team up to write the Econ 101 textbook of the future!

51 comments:

  1. Lol, as in all empirical sciences with small samples and bad incentives to play with noise, majority of the new revolutionary studies will prove to be garbage. Some will survive.

    ReplyDelete
    Replies
    1. What are "bad incentives to play with noise?" Do you mean the propensity of people to see patterns that aren't there... and the academic or other incentives for them to do so?

      I think you're probably correct. Just a few (at best) will survive.

      Delete
    2. The novelty incentive in academic publishing race that privileges noise. Additionally, given lack of decisive answers/tests, the sociology of the field requires the sorting of ability somehow. So people signal it by mathematical ability, since that can be relatively clearly measured. This, in turn, drives people to invent ever new complex mathematical structures to prove themselves. Unfortunately you can only really prove asymptotic results and these are, and I say that advisedly, basically useless.

      It used to be that this disease got expressed in ever more complex theories of the economy as such (DGSE and the like), now, the comedy shifted to empirical tests with ever more advanced methods.

      Delete
    3. @Krzys, what do you think of this take on math in econ.

      I liked the list at the end:

      --- start quote ---

      Not using mathematics for your model:

      Puts an upper bound on the complexity of your model (makes it easier to compete with smart people)
      Allows you to say what you're saying is novel even if it is equivalent to existing models (allows you to fool people into thinking you have new insight)
      Makes it impossible for anyone else to use your model (limits the use of your model to yourself)
      Allows your model to change in respose to new information without saying you are changing it in response to new information (so you can fudge any data)
      Allows you to get around stating clear predictions and precise conditions on those predictions (so you can fudge any prediction)
      Allows you to stay away from quantitative statements about the data (so you can fudge any data)
      Allows you to have effects of different magnitudes from causes of the same size (and vice versa) (so you can fudge any data)


      A lack of math is not depth of argument or intuitive understanding, it is flim flam.

      --- end quote ---

      Delete
  2. Long time commentator & blogger "Unlearning Economics" should be happy with this post, particularly the title.

    You make a point here that I try to make: if economists are REALLY scientists (puzzle solvers), then they should be overjoyed when their theories are falsified... they should be crying tears of pure unbridled joy, because puzzle solving should be their greatest pleasure in life, and having a simple candidate solution smacked down just gives them an opportunity... adding another puzzle to the inventory while simultaneously upping the challenge level. That sounds like a gift to a puzzle solver, not an offense. I've heard more than one physicist sound disappointed when a mystery seemed to be solved in physics... for that very reason! One less challenging puzzle to work on. )c:
    E.g. Lawrence Krauss expressed precisely this sentiment when it looked like the existence of the Higgs boson had been verified... he literally said he wished it hadn't, because if the Higgs boson didn't exist, then the mystery would be more complex and challenging and it would be a more interesting puzzle to solve. Can you imagine a economic theorist saying something like that? Hahahaha!...

    I like to ask macro bloggers this question: "What evidence would convince you that you're wrong about that?"... only a few (only one really) launched into a well thought out laundry list of examples that would demonstrate that their models (indeed their framework) were so false as to be unsalvageable and thus worthy only of abandonment. Other responses have been all over the map, but sad to say a few were essentially "I'm not wong: I'm as sure about this as anything" and a few others were more like "That's a strange question: I just demonstrated evidence I've been correct all along. I can't believe you're asking me that!" ... as if they NEVER considered what a world in which they were wrong would look like.

    ReplyDelete
    Replies
    1. You confuse, empirical studies with evidence.You also clearly do not realize that complex answers in small samples (i.e. all of them) means only one thing: noise.

      Yes, some simple theories are wrong and should be replaced. However, they cannot be replaced with more complex answers. Only with other simple ones.
      That's all the data allows.

      Delete
    2. I'm not saying that the answers themselves should be more complex. I'm completely open to simpler answers (i.e. simpler models). However solving the puzzle to find the simpler answer may be more of a conceptual challenge than first imagined by economists devising supposedly intuitive (yet complex) solutions from their armchairs.

      And I have had exactly one macro blogger tell me precisely the empirical evidence that would convince him that his framework (not just the models derived from that framework) was probably irreparably flawed. He went on to state that he's not interested in adding epicycles: he'd rather just write a post-mortem and abandon it. Also, his framework DOES result in simple models... with far fewer parameters than competing models (competing models such as those used by the NY Fed).

      Small number of model parameters (like 3 or 4, as compared to 40 to 50, which is more typical) helps avoid the problem of over-fitting historical data. Granted his models are more limited in scope, so the comparison in terms of number of parameters isn't entirely fair. So far his models have compared favorably to the available competition over the scope of macro parameters he limits his forecasts to. He's challenged other macro bloggers to put forward their own forecasting models but so far nobody has taken him up on that.

      I loved the idea of this challenge, so I've asked numerous other macro bloggers why they don't put forward their own forecasting models along with specific testable forecasts, and all I get are excuses generally. Being specific about forecasts means you're putting yourself in danger of learning something. The quickest thing (and most likely thing) you could potentially learn is that your models and/or framework are crap. Learning that they're not crap is going to take a lot more time and be more a lot more harrowing.

      I don't care who wins or what their politics are or if nobody wins because all the current models/frameworks turn out to be crap. But if theorists are really primarily interested in solving puzzles about reality, why aren't they eager to put their ideas on the line?

      Delete
    3. @Krzys, you're correct in your criticism of my original comment though: I shouldn't have used the word "simple" and "complex" the way I did. Instead of "complex" I should have said something like "a conceptual leap from current orthodoxy" or something like that. That doesn't necessarily mean complex or even counterintuitive. But it may require a big departure from what currently seems intuitive or at least based on intuition.

      Delete
    4. It's not as simple as that. You cannot jump from one conceptual framework to another as the data set changes. You take what (kinda) worked in one field and apply somewhere else: say you go from analyzing the market for widgets and then to labor, and hope for the best. If you develop totally new conceptual frameworks from one application to another, you are overfitting.

      There's either a couple of simple principles that apply to majority of cases (data sets), or there is nothing.

      There are severe limits on the kind of overall complexity you can have as allowed by the existing data sets.

      People, should really take Leamer's Tantalus metaphor to heart.

      Delete
    5. mkt428:29 PM

      I agree with Krzys. Experimental physicists have to work hard to get clean, uncontaminated data, locating neutrino detectors deep underground or building billion-dollar particle accelerators. But what they eventually get, if the apparatus and procedures work out, is clean data, often with millions of observations. Nature is friendly to natural scientists that way.

      Social science data are far messier, and no matter how hard the researchers work the observations will always be subject to the changing moods, social norms, animal spirits -- and changing laws too (nature's laws can be assumed to be unchanging; the exact nature of the economy is probably evolving).

      It's a simplistic and reductionist procedure to say "well I'll just look at the data and I'll see which theory is right and which ones are wrong". The data are too messy and complex to do that. The data provide evidence of course, but only weak evidence. It's like trying to do astronomy without access to telescopes -- and in a climate with perpetual fog.

      Economists, and indeed all researchers, need to be perpetually open-minded and to pay attention to the data. But empirical data are a mixture of signal and noise -- mostly noise in the case of social science data (more accurately, noise plus a large number of omitted variables that we would like to measure but cannot).

      Delete
    6. @mkt42,

      "Experimental physicists have to work hard to get clean, uncontaminated data"

      Evolutionary biologists historically did pretty good looking at messy "natural experiments." They could make falsifiable predictions (e.g. you won't find any rabbit fossils in a pre-Cambrian layer), and they could check them.

      I agree the data is probably messier ("uninformative" as Noah used to say), but that doesn't mean you give up on it. You don't really have other options.

      @Krzys, you write:

      "If you develop totally new conceptual frameworks from one application to another, you are overfitting."

      What if you use the same conceptual framework? Here's an evaluation of how some other purported "frameworks" stand up as frameworks. What's your opinion?

      Delete
  3. I'm glad at least some of it is finally showing up in Macro-Economics, since Macro has been where you get the greatest policy suggestions built around the thinnest of evidence.

    As for the minimum wage, it always struck me as more likely that the payers of increased statutory wages would try and deal with it through productivity gains or price increases (or both). Reducing their own returns is something they'd be very reluctant to do beyond short-term, and reducing employment may not be feasible. I'm curious as to whether there's any research on that front - whether minimum wage increases lead to price increases where they occur.

    ReplyDelete
    Replies
    1. Curiously, all this literature ignores the fundamental question: where's the money coming from?

      Delete
    2. It can come from a variety of sources, including the owners (if they take a smaller profit margin), the employees (if they faced reduced hours or fewer jobs), the consumers (if prices go up), and from none of the above if there is greater productivity.

      Delete
    3. Sure, it can go from anywhere. The question is where it actually comes from. The current literature is not asking the basic question and doing the appropriate welfare analysis.

      Delete
    4. Wages are flows. Revenues are flow. increasing wages also increases revenue because increased pay yields increased spending.

      Delete
    5. Wages are flows. Revenues are flow. increasing wages also increases revenue because increased pay yields increased spending.

      Delete
  4. Come on, Noah don't steal the name without the h/t to https://unlearningeconomics.wordpress.com/

    ReplyDelete
  5. This looks extremely cherry picked to me. I've never seen any contentious economic theory not have empirical support for *both* sides of the argument - this definitely includes minimum wage. I'm also certain I've seen you mention before that it's fairly easy to produce an econometric result that supports any theory - there's always variables not being controlled for.

    ReplyDelete
    Replies
    1. This looks extremely cherry picked to me. I've never seen any contentious economic theory not have empirical support for *both* sides of the argument - this definitely includes minimum wage.

      Oh, but there is a preponderance of evidence on one side in most well-studied cases. For minimum wage, all the argument is about long-term effects. Nobody finds much of a short-term effect.

      Delete
    2. There's also the size effect. Nobody really believes, there is no effect whatsoever. If you do, we should raise the minimum wage to $400/hour. It's high time the poor drove ferraris, too.

      Delete
    3. Yep, size matters.

      Delete
  6. Anonymous12:55 PM

    tfw you click a link expecting some kind of ungated lit review from an established journal, only to find a CEPR policy paper :-(

    ReplyDelete
    Replies
    1. It has a lot of good links. Do you know another good survey?

      Delete
  7. You say that "a ton" of standard theories are not consistent with the data, yet three of your four examples pertain to the labor market. Those three examples are really just one example: they suggest that the competitive model doesn't explain one particular market -- the labor market -- very well. This may well be true, and it should excite up-and-coming labor economists. But does it really mean that we have to "unlearn" economics?

    I mean, what would you expect to happen to the price of apples if a huge quantity of apples were dumped into your local market? What would you expect to happen to the quantity demanded of apples if the government fixed the price of apples above the laissez-faire price? How much do your labor market examples influence your beliefs about the answers to these questions?

    ReplyDelete
    Replies
    1. Actually, from what I know, basic price theory (supply-n-demand) works really damn well for most goods and services markets. Random utility models are incredibly successful at predicting consumer choice.

      It's not all of our theories we have to unlearn, just some of em. That's what's cool about the data revolution, it tells us the difference!

      Delete
    2. I'm rooting for your data revolution. Here's another example of how sorely it's needed: Do anti-price-gouging laws hinder getting relief supplies into areas hit by natural disasters? Economists think so.

      Agencies responsible for disaster response don't agree:

      "Cash is the most efficient method of donating – Cash offers voluntary agencies the most flexibility in obtaining the most-needed resources and pumps money into the local economy to help businesses recover." [FEMA website, my emphasis]

      "Financial contributions allow the Red Cross to purchase exactly what is needed for the disaster relief operation. Monetary donations also enable the Red Cross to purchase relief supplies close to the disaster site which avoids delays and transportation costs in getting basic necessities to disaster victims. [American Red Cross website, my emphasis]

      So the problem the economists were sure could be solved by an unrestrained price mechanism -- simply doesn't exist. Supplies are not lacking in or near disaster areas. But economists were willing to recommend a policy solution without even looking at the real world to see if there was a problem at all. Talk about the ivory tower...

      Would be good to have some data on this, of course, if someone could find a 'natural experiment' or quasi-experiment. But no economist suggested that, either.

      Delete
    3. "Actually, from what I know, basic price theory (supply-n-demand) works really damn well for most goods and services markets."

      Well, no. The model works well for most commodity markets (oil, standard plywood, iron ore, coffee...). It does not work well for most capital and intermediate goods, most basic consumer goods, most household-produced goods, most consumer services...In short, most goods. Because, in these markets there are either strong social factors (households), administrative pricing to capture profit across a basket rather than a single good (supermarkets, corporations), pricing that includes some longer-term rent stream (computer services, many capital goods) or some other factor.

      Douglass North explicitly acknowledged this decades ago, and then went on to explore why this is so (I don't buy all of his answer, but at least he didn't try to argue against the obvious).

      Delete
  8. Anonymous6:17 PM

    Standard theory serves the purpose it was designed to serve , further enriching the already rich. The fact that it's all wrong is completely beside the point. Smart economists are well aware of this , the rest - the true believers - are just useful idiots , analogous to those who chant " USA ! , USA ! " at rallies because they think we're all about spreading freedom and democracy , and defeating terrorists , and that the traitor Snowden should be hung.

    Big Lie Foreign Policy , Big Lie Intelligence , Big Lie Economics - take your pick - they're all from the same ancient playbook , of an unbeatable team.

    Marko

    ReplyDelete
    Replies
    1. Anonymous8:34 PM

      How do you even manage to tie your shoes in the morning, being so obtuse?

      Delete
    2. Ahh, One Last Anonymous Troll, I can always tell your comments by their unique combination of aggressiveness and lack of content...

      Delete
    3. Anonymous2:36 PM

      I like to stop by every once in a while.

      Delete
    4. Anonymous6:26 PM

      For empirical evidence consistent with Marko's substantive comment about economics, read Fred Lee's history of heterodox economics. One could cite much more.

      Opocher & Steedman (2015) have a theory consistent with empirical findings on labor "markets".

      - Robert

      Delete
    5. Anonymous11:52 AM

      Nice to see one of the true crackpots stopping by for a comment. Thanks Robert, we almost forgot you don't matter in the least.

      Delete
  9. I'm always surprised when Econ bloggers, ones employed as economists, that is, are so emphatic about the negative employment effects of increasing the minimum wage.

    I usually then point out a plain statement from my Econ 101 professor: if and only if the minimum wage is moved above the equilibrium wage, job losses will result.

    I then ask how they know what the equilibrium wage is, which normally ends the discussion because I don't get a reply.

    And it seems entirely reasonable to me that from a Political economics viewpoint, a govt mandated minimum wage will be below the equilibrium wage simply because power tends to reside on the capital side of the capital/labour divide.

    Therefore, the equilibrium minimum wage will always be above the mandated wage.

    ReplyDelete
    Replies
    1. labor force participation rates r low and not every restaurant delivers

      labor force participation rates r low and grannies go broke trying to pay for help

      Delete
  10. Anonymous6:06 AM

    Not a bad post - better than some of your recent ones if I may say so! Good news that a lot of the abstraction and ridiculous story telling that goes with it to get a model is being trashed, however, I would though that a lot of the empirical techniques used in this 'empirical revolution' are also questionable. So there is still a lot of unlearning to be done. In particular there is a lot of historical, psychological, anthropological and other work out there that contradicts what economics says about how social systems work. A lot of the causal links that empirical studies in economics asserts are questionable because the manner in which these are investigated are questionable. There needs to be acceptance that there is a limit to what numerical evidence, and quantitative methodology can achieve. For example if there overwhelming evidence in bank annual reports saying that 'we are restricting lending because of X" and borrowers "ovvewhelming say it is because of X", that is firm evidence for a liquidity trap, as good as you will get for X being the cause. But people who are doing this hands-on work are not economists. Habermas puts it well:

    https://larspsyll.wordpress.com/2015/11/09/some-unfounded-expectations-of-economic-theory/

    NK.

    ReplyDelete
  11. Anonymous7:01 AM

    Re 2. See: http://www.hks.harvard.edu/fs/gborjas/publications/working%20papers/Mariel2015.pdf

    "A reappraisal of the Mariel evidence,specifically examining the evolution of wages in the low-skill group most likely to be affected, quickly overturns the finding that Mariel did not affect Miami’s wage structure. The absolute wage of high school dropouts in Miami dropped dramatically, as did the wage of high school dropouts relative to that of either high school graduates or college graduates."

    ReplyDelete
  12. http://bilbo.economicoutlook.net/blog/?p=1623
    The money multiplier debunked and how banking really works (from April 2009.)

    ReplyDelete
  13. The secret is to have a measure of Effective Demand as Keynes and Kalecki were envisioning it. This measure still does not exist openly in economics. I have my measure which is working and explaining much...
    if someone like DeLong was to grasp the principles of effective demand, that one might be the most powerful economist on the planet... his mind would explode with insights... And effective demand is different from aggregate demand... economists equate the two...

    http://effectivedemand.typepad.com/ed/2015/11/keynes-vision-ed.html

    ReplyDelete
    Replies
    1. Anonymous11:54 AM

      This thread has really brought out the crackpots. All we need is that lunatic Information Transfer guy, Richard Serlin, and one of the goldbugs (Greg Ransom perhaps?) and we'll have a basketball team of the crazy and irrelevant.

      Delete
    2. Anonymous12:38 PM

      Maybe we could start a fantasy league.

      Delete
    3. Anonymous12:27 PM

      I draft this guy: http://www.amazon.com/MICHAEL-PATRICK-AMOS/e/B004LE3T2W

      Delete
  14. How's this for purely intuitive (non-math anyway) labor market theory: effective collective bargaining (effective might need centralized bargaining) AUTOMATICALLY keeps wages at SUBJECTIVELY acceptable levels because of the knowledge that wage levels were set in an equal power struggle with the other two negotiators, ownership and the ultimate consumer. Acceptable depending on relative rather than absolute results.

    Under what I call subsistence plus wage setting (monopsony) wages always leave labor dissatisfied (angry?) because they know that the final market (meaning the ultimate consumer) would have paid more -- no matter what the absolute wage level.

    Somebody correct me if I'm wrong, but, minimum wage graphs always seem to assume the buyer is ownership (management) -- not ownership AND the ultimate consumer together. Walmart, most obvious example, could double its wages and only raise prices 7%. Is that what is intuitively assumed with the price-and-demand graphs? I don't get that feeling.

    Fun time: if a minimum wage raise costs job loses at firm A as consumers pay more for less -- won't it also cause losses at firm B where the consumer used to buy more (before it started paying more at A)? Horrors :-0 as long as the newly flush employees burn their higher earnings in a show of defiance. Of course if they spend those dollars at other firms those other firms will have to put on more employees.

    In so far as the 65% of McDonald's consumers come through the driveway much of the change in dollar flow could be from the mid to the low income employees and low income firms. If so and if low income employees tend to patronize low income firms to a greater extent, then, the lost jobs (in the perpetual dollar flow) may actually be in mid paying firms.

    It's just intuition. :-)

    Hybrid redistribution suggestion: Redistribute from the top 1% who get 20+% of income -- up from 8% not too long ago -- to mid incomes by taxing income over $2 million at 90%, substantially lowering mid income tax burdens. The mid can then pay higher prices for the labor of the (formerly?) low income according to how collective bargaining shakes out the market.

    ReplyDelete
  15. You should h/t to UnlearningEconomics. Soon enough, you might even become a Post Keynesian!

    By the way, what do you say about some of the empirical evidence demonstrating that most prices do not conform to traditional price setting theory, but instead to administered mark-up "fixprices"?

    http://www.nber.org/chapters/c8331.pdf - Alan Blinder
    http://www.concertedaction.com/2012/01/17/post-keynesian-markup-pricing/
    http://www.jstor.org/stable/40719759?seq=1#page_scan_tab_contents
    https://www.vu.edu.au/sites/default/files/cses/pdfs/bloch-paper.pdf

    ReplyDelete
  16. Anonymous7:52 PM

    Just a minor correction - Einsteins relativity was as disconnected from experimental evidence as could be - apart from Mechaelson-Morley experiment which I am not even even sure that Einstein took any notice of, any experimental evidence for relativity came after his predictions. Mendelson probably fudged his experimental results also as they were too good.

    ReplyDelete
  17. Very Interesting Information That You Have Shared... Thanks For Sharing...
    Commodity Market Tips

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. Anonymous2:30 PM

    OK, so empirical work shows higher min wage need not result in lower employment. But it seems that theoretically econ has only one way of making sense of this, viz. model of imperfect competition or monopsony. That roughly gives you the possibility of firms responding to having to paying higher wages that eliminate rents with an increased volume of production. But empirically it is clear that higher minimum wages have not reduced employment in markets that are not monopsonistic.
    Of course with a surplus approach (Sraffian or Marxian) this would not be difficult to explain. Higher wages may simple come at the expense of some capitalist consumption without reducing competition-induced accumulation; and cost increases can't be passed on to consumers either due to competition.
    Rakesh Bhandari

    ReplyDelete