Saturday, November 28, 2015

Econ 101 and data (reply to David Henderson)


I wrote a post for Bloomberg View about how Econ 101 needs more empirics (a favorite hobbyhorse of mine). They titled it "Most of What You Learned in Econ 101 is Wrong", which was a catchy but inaccurate title, since the word "wrong" is often unhelpful in describing scientific theories. For example, in the post, when writing about minimum wages, I wrote:
That doesn't mean the [Econ 101] theory is wrong, of course. It probably only describes a small piece of what is really going on in the labor market.
Sometimes you test a theory by looking at policy experiments, and what you find is that the treatment effect is in the direction the theory predicts, but the fit is poor - the percent of the variance explained is small. Does that mean the theory is "right", because the treatment effect is in the expected direction? Or does it mean the theory is "wrong", because the fit is poor? 

The answer, of course, is that "right" and "wrong" are not very descriptive, helpful adjectives in this situation. 

Anyway, there's the question of what to teach kids. Personally, I think that you should teach kids empirics no matter how good your theories are. High school physics and chem classes teach simple theories that are amazingly good at explaining a lot of real phenomena, but they also make sure to include lab experiments, so that kids can see for themselves that the theories work. And those are high school kids; college kids will be even more capable. There's no reason a college econ student shouldn't learn how to run regressions in 101.

But I think this becomes even more important when Econ 101 theories have poor fit. People tend to confuse treatment effects with goodness of fit, so if you teach kids a bunch of theories with poor fit but which get the sign of the treatment effect right, the kids will leave class thinking that these theories explain a lot more of the world than they really do.

This distinction is what some critics of my Bloomberg View post don't seem to get. For example, David Henderson writes:
In other words, in most cases there is a small, presumably negative, effect on employment. And presumably in the other cases there is a large effect. How, exactly, does this contradict the claims that Mankiw makes and that many of us teach in our equivalents of Econ 101? It doesn't.
In this case, the theories that Econ 101 books (like Mankiw's) teach tend to get treatment effects of the right sign - minimum wage hikes of the typical size probably do have a (very small) negative effect on employment. When Henderson says that the evidence doesn't "contradict" the theory, he means that the theory gets the sign of the treatment effect right.

The problem is that by emphasizing theory so much, and by relegating evidence to some brief asides, Econ 101 textbooks (and classes) will tend to trick kids into thinking that the theories have better fit than they do. When you make people learn a theory in detail, I think they naturally tend to believe that the theory has strong empirical fit unless they see evidence to the contrary. 

My other example was welfare. Studies show that the effects of the implicit tax rates in welfare programs like the EITC are very small. That's consistent with the relatively small Frisch elasticity of labor supply found in most micro studies. Even when you get welfare programs with very large implicit tax rates (100%!), the effect is generally not as big as you might expect (with 100% implicit tax rates, you'd expect AFDC to leave income unchanged, when in fact it does boost it somewhat). That implies that welfare affects labor supply through channels other than implicit tax rates and lump-sum payments. For example, Moffitt's survey of theory and evidence on traditional (TANF and AFDC) welfare programs discusses the idea of "welfare stigma". That's an idea that is difficult to explain with an Econ 101 type theory. But it may be important in practice. Thus, it is an idea that can only really be presented by looking at the evidence. (Note that this is a reason welfare programs are probably worse than the labor supply effects would imply!)

Now Mankiw doesn't disregard evidence, and Henderson gives an example of the way that Mankiw presents it:
Although there is some debate about how much the minimum wage affects unemployment, the typical study finds that a 10 percent increase in the minimum wage depresses teenage employment [by] between 1 and 3 percent.
Evidence is given, but it is relegated to a brief aside. The kids don't see the data for themselves. They don't learn how to work with it. They don't learn how the studies tested what they tested. They don't learn how to go verify for themselves how useful econ theories are.

To these kids, econ theories must seem like received wisdom. Even evidence, when presented only as a brief aside with no understanding of methodology, must also seem like received wisdom. Again and again, I talk to econ students who complain that they are expected simply to swallow what they are taught - unlike in their science classes. College kids are smart, and many of them are skeptical. They grow up learning that "science is the belief in the ignorance of experts." That doesn't tend to sit well with the kind of "received wisdom" approach that almost every intro econ textbook takes. Nor should it.

So while I definitely don't want to pick on Mankiw - his books really are excellent, my personal favorites - I think that most intro econ textbooks use this "received wisdom" style of theory-centric education with only brief references to empirical findings. That tends to result in students who either A) believe too much in the power of the theories they learn, or B) disbelieve and distrust econ in general. I see a lot of examples of both (A) and (B). Both of those outcomes do a disservice to the world.

40 comments:

  1. Is your point just that these theoretical conclusions are (often) qualitatively correct, but quantitatively unimportant? That, e.g., raising the minimum wage does increase unemployment, but that changes in the minimum wage explain little of the variation in unemployment rates? Is it not possible that, while each effect discussed in Econ 101 is, by itself, not very important, that in combination the effects could be large? Also, couldn't these effects be much larger in the long run? And couldn't these effects be much more pronounced when taken outside of the range of experience (e.g., raising the minimum wage to $15/hour)? It seems a lot of Econ 101 might then be salvaged with: "these effects in isolation are small in the short run". Ideally, we would try to look at these effects empirically in combination, and in the long run, but absent that, finding the beginnings of these effects in isolation and in the short run seems promising evidence if anything.

    ReplyDelete
    Replies
    1. It certainly sounds like you are eager to fit the data to your preconceptions.

      Read http://www.cepr.net/documents/publications/min-wage-2013-02.pdf and then let us know if that does not address your concerns with long term effects or affects in isolation.

      Delete
    2. It certainly sounds like you are eager to fit the data to your preconceptions.

      Read http://www.cepr.net/documents/publications/min-wage-2013-02.pdf and then let us know if that does not address your concerns with long term effects or affects in isolation.

      Delete
    3. "It certainly sounds like you are eager to fit the data to your preconceptions."

      You say this as you link to a piece published by a left-wing think tank. (I've read this, by the way).

      If we're going to be throwing links around, here is one from me:

      http://ilr.sagepub.com/content/67/3_suppl/608.short

      If academic articles aren't your thing, this discussion with Neumark is useful:

      http://bloggingheads.tv/videos/36521

      Delete
    4. Anonymous5:45 PM

      Presumably, you will define "long run" to mean a time-scale long enough for some shitty linear regression to get a p-value of 0.05 on the relevant coefficient.

      Delete
  2. Anonymous5:25 PM

    Poor fit and lack of discussion of the evidence is one thing. Ignorance or uncritical acceptance of the underlying hypotheses (institutional, psychological etc...), or pertinence of these hypotheses and their validity domain is another. Maybe econ 101 is not "wrong", but in many cases the way it is taught is meaningless and useless.

    In math and science, the hypotheses and axioms that condition the validity of a statement are clearly stated.

    ReplyDelete
  3. Anonymous5:28 PM

    The thing is undergraduates are not just taught Econ 101, even in my first year as an undergraduate I was taught far more than that - including econometrics, which forced us to look at all sorts of data relating directly to an 'econ 101'. Secondly, I don't know how it works in the US, but in the UK we're not taught that 'minimum wages raises unemployment', or that welfare encourages laziness - literally not at all. The tutor provides all kinds of supplementary material for us to read rather than just teaching out of a textbook, and even in econ 101 the models are a bit more complicated than that. I honestly found it as a fairly offensive strawman, and your article could honestly directly damage (no matter how minutely) the job prospects of graduating students, do you honestly consider that your writings could almost be unjustly defamatory?

    ReplyDelete
    Replies
    1. Some links and titles to the referenced material taught in the UK on minimum wages and welfare would be appreciated.

      Delete
  4. Anonymous5:37 PM

    "The kids don't see the data for themselves. They don't learn how to work with it."

    Again this would only make sense as a criticism if undergraduate economics consisted solely of 'econ 101', but it doesn't. You're taught multiple modules, including empirical ones - it seems stupid to teach less theory in econ 101 to devote some time to empirics, when you can teach pure empirics in an entirely separate econometrics module, ideally with a professor with more expertise in that area. To leave out this extremely crucial detail is again highly defamatory to the kids you're apparently trying to help (rather than embarrass).

    ReplyDelete
  5. But most students taking Econ 101 in the US are not economics majors. So, Noah's criticisms are much more relevant there than they are in the UK.

    ReplyDelete
  6. If only the minimum wage in Seattle was $7 an hour instead of $11, then the unemployment rate would be -3% instead of 3%!

    ReplyDelete
    Replies
    1. Be fair Gilberto. On Mankiw's lower range empiric it might only fall to -1%.

      Delete
    2. Be fair Gilberto. On Mankiw's lower range empiric it might only fall to -1%.

      Delete
  7. I did a combined degree in Australia, Law and Economics. Whereas my Law lecturers would usually say there Is usually no one right answer and it is the quality of your arguments that count, in Economics (at least at the undergraduate level) there is (especially in microeconomics) just one answer (like math) and you either get it right or not at all. No matter how original your arguments might be on a particular issue, it is either right or wrong. No points for originality. In fairness, in any discipline it is important to understand what the received wisdom from accumulated learning is. The received wisdom is easy to understand in Law but much harder in Economics.

    ReplyDelete
  8. I took a vow to never mention my textbooks again in public, and I've mostly kept to it, but I will break it here. I began the project with the same thought as Noah's; my prospectus from the beginning said the "empirical turn" in economics was one of the most important developments in the discipline and largely unrepresented in the principles texts. The micro book has a lot of empirical stuff in it, and an early chapter walks students through OLS regression. But it was in the macro book, which I wrote later, that empirics are really emphasized; there's a long section, for instance, on curve-fitting using the simple linear consumption function as an example. Another example is the chapter on business cycles which develops three typical varieties (investment cycles, policy cycles, financial cycles) and then spends lots of space discussing what the distinguishing markers would be and looking for them in the data.

    I never had much hope that the books would get adopted much, and now I have no hope at all. But the purpose of writing them was to put these pedagogical approaches out there so people could react to them and go further. I wanted to get the ball rolling. But here it is, over a year later, and I'm not seeing any rolling anywhere. (And that includes the INET package, which in my opinion doesn't deal with pedagogy at all. And hardly any empirics.)

    I apologize for this rant -- shall return to keeping my mouth shut on this.

    ReplyDelete
    Replies
    1. Martin3:30 AM

      Peter,

      You should mention your textbooks more often. I had no idea that it existed and I would definitely have a look.

      You could consider regularly contrasting your approach with that of the other textbooks out there and why you think that your approach is better. It would be an interesting discussion on pedagogy as well.

      Perhaps, you and Noah can discuss your textbooks and go chapter by chapter or topic by topic over your book and other books for their approach to pedagogy and have a bit of a discussion on it. I would be very interested to learn more starting out a career in teaching.

      Delete
  9. Jerry Brown2:59 AM

    I liked the post and I liked the title. Most of it is wrong- Sometimes.

    ReplyDelete
  10. The problem here is not how economics is taught. The problem is the social belief that having sat through one introductory class makes you an expert in the subject. Surely in any reasonable society policy makers would read at least a couple of peer reviewed papers on a subject without relying on their memory of a college course.

    ReplyDelete
  11. Anonymous7:35 AM

    Well, shouldn’t there be evidence for the most fundamental neoclassical economic proposition? Where’s the empirical evidence that equilibrium prices are determined by supply and demand curves?

    Please point me to the empirical evidence that equilibrium prices are determined by supply and demand. And, if there’s not any, why do you subscribe to this theory? In what other science is such a foundational proposition accepted without supporting empirical evidence?

    ReplyDelete
    Replies
    1. I’ll use your comment as an opportunity to discuss how I currently teach S&D in micro, and how I would revise my micro text if there were to be a second edition. (Which sounds like an exercise in modal logic.)

      S&D curves are heuristic devices that help us work through the logic of how markets work. In general they are not constructed out of empirical evidence. First, except perhaps for finance, these curves are not deterministic; the so-called law of one price typically fails. Second, we generally don’t have an empirical basis for predicting buyer or seller behavior at prices or quantities well removed from what we see today.

      What we do construct empirically are elasticities, which come with a range of error. If you want, you can extrapolate them to full-fledged curves, but the error is so great that you are really back in the realm of thought experiments.

      But it’s even worse than this, because S and D curves can logically fold or have discontinuities if the underlying relationships are sufficiently complex. The habit of drawing monotonic curves gets in the way of realizing this, so in some ways you could say that they are counter-heuristic.

      My bottom line is that it can be helpful to trot out S&D from time to time, but in applied contexts the focus should be on elasticities, as well as the consideration of factors that affect the logical underpinnings of how markets should be expected to work.

      Regarding this final point, one of the sections in my text that I think is successful is the discussion of how goods become homogeneous in the chapter on markets; also I think I did an OK job in transitioning students away from simple S&D toward search theory in labor markets. I struggled with conveying interactive nonconvexity and multiple equilibrium in the chapter on GET, but I did what I could. I would be happy if someone out there could improve on it.

      Delete
    2. Anonymous9:07 PM

      "S&D curves are heuristic devices that help us work through the logic of how markets work..." -P. Dorman

      This, of course, is nonsense. Neoclassical economics claims that equilibrium prices are determined by supply and demand curves. This fundamental proposition determines the other questions asked about capitals, the concepts (e.g., utility, social welfare and so on) used to analyze capitalism and the conclusions reached about capitalism. The whole theoretical edifice of neoclassical economics was invented to arrive at this result.

      Therefore, my question stands. Rather than answer it you went off on some sort of navel gazing rant. Please answer the question. Where is the empirical evidence that equilibrium prices are determined by supply and demand curves.
      Economics is supposed to be a science. Neoclassical economics is supposed to be a theory that explains the mechanism behind the observed phenomenon. Moreover, a theory is a tested and confirmed hypothesis.

      So, where's the empirical evidence that equilibrium prices are determined by supply and demand curves?

      I'm waiting.

      Delete
    3. Anonymous10:22 PM

      http://lmgtfy.com/?q=supply+demand+prices+empirical+evidence

      Delete
    4. Anonymous really does need to read Economist Rules by Dani Rodrik. Peter's reply was quite good but you dismiss it? Dani does a masterful job of sorting through the issue of models, theories, and evidence. It is an excellent read.

      Delete
    5. Empirical evidence abounds. If you were watching the Solar PV industry circa 2004, interesting stuff happened.

      Demand for silicon usable in solar PV wafers was increasing about 40% per year. In 2004, the PV industry started consuming the entire supply of semiconductor industry waste silicon. Suddenly that price of that supply jumped. The PV industry was scrambling to get contracts for delivery of silicon, and the producers were able to demand nice profit margins to supply the silicon.

      But the PV industry doesn't actually need semiconductor grade silicon. They're happy to take it when it's otherwise going unused, but once they used all the available waste supply, manufacturers started creating PV grade silicon. In a couple-few years manufacturers were able to produce high volumes of PV-grade silicon that was inherently cheaper to manufacture and prices dropped.


      You can certainly make some argument that some of the time the price of silicon is the cost to produce it plus a reasonable profit margin. But when push comes to shove, and demand was way out of whack with supply, the price spiked in ways that could not be explained by cost plus profit margin.

      Delete
    6. Anonymous1:16 PM

      Now that's just ignorance. I asked for empirical evidence that equilibrium prices are determined by supply and demand curves and you offer descriptive instances where the forces of supply and demand influence market price. That is, you missed the question.

      Delete
  12. Anonymous10:02 AM

    http://noahpinionblog.blogspot.com/2012/09/econotrolls-illustrated-bestiary.html

    ReplyDelete
    Replies
    1. Anonymous12:04 PM

      Geez, I hope this isn't a non-substantive reply to Anonymous at 7:35 AM. Ridicule isn't a substitute for substantive responses.

      Either there is or isn't empirical evidence that equilibrium prices and quantities are determined by supply and demand curves.

      Please direct me to the evidence if it exists. Is that an unreasonable request?

      Delete
    2. Please stop. This has become troll behavior. I'm asking folks to read the latest from Dani. You really need to as you are confused as to the role of models.

      Delete
    3. Anonymous1:22 PM

      You're trying to bully and intimidate someone into not asking a very important question.

      Just please answer the question. If you can't, then say so, but don't try and intimidate me with name calling. The question stands.

      Most economists are neoclassical. Neoclassical economics has been around for over 100 years. There's certainly been time to gather empirical evidence that equilibrium prices are determined by supply and demand curves. Where is it? If you want to stop what you describe as troll behavior, then just provide the links to the scholarly articles including empirical evidence that equilibrium prices are determined by supply and demand curves.

      If not, you can keep your keystrokes to yourself.

      Delete
  13. Just a couple of stray thoughts:
    A minimum wage hike (hopefully) shifts some purchasing from higher to lower wage firms -- at least some of which would have been spent at mid and high wage producers. Purchasing power shifts to a different level tends to stay at that level. So maybe you should look for the minimum wage hike job loss among mid level jobs.
    * * * * * *
    For so-called (not exactly) "laziness": look at American born (would have been) workers who wont work for $400 a week -- on jobs that could (either have in the past -- or could with collective bargaining) paid $800 a week.

    E.g., Supermarket contracts gone two-tier to Walmart undercutting -- now undertaken mostly by early stage, not yet family raising applicants.
    E.g., Taxi driver jobs that now pay $500 (for 60 hrs) -- now undertaken by new arrivals who are sacrificing their live to make better lives for their children; think Issac Asimov's parents "buried alive in the candy store" (definitely not by me anymore).
    E.g., Retail clerk jobs that are only undertaken by the most dedicated, suffering employees.

    Think Americans who wont go to work in a labor market where collective bargaining cannot extract the max from the CONSUMER -- in a toothless political milieu that can't move progressive legislation into reality. Think 100,000 out of 200,000 Chicago gang age males in drug dealing street gangs.

    Just a couple of different angles to lot at (to look for).

    ReplyDelete
    Replies
    1. Yeah, you can make shit up. Or you could look for a peer reviewed paper and look at the evidence.

      Delete
    2. Yeah, you can make shit up. Or you could look for a peer reviewed paper and look at the evidence.

      Delete
  14. Again... economics is a dynamic system. You may see a rise in teenage unemployment after a minimum wage rise, but that may just be a short-term effect. Wait a bit and Violá... the teenage unemployment rate begins to fall as more income works its way through to teenagers and the teenagers serving them, for instance in clothes, sports and other retail goods where teenage salespeople do best to sell to other teenagers.

    ReplyDelete
  15. Ezra's point seems to be that Econ 101 teaching lacks depth in a number of ways, and ends up leaving the student lacking in useful understanding.

    I will have to think about that a bit...

    But one thing that immediately caught my eye is how, in the quoted passage from Mankiw's book, he chose to focus on teenage unemployment to discuss the sensitivity of employment to minimum wage shifts.

    Maybe I'm just cynical, but that stinks of a deliberate move to make the response seem larger.

    ReplyDelete
  16. College kids are smart, and many of them are skeptical. They grow up learning that "science is the belief in the ignorance of experts." That doesn't tend to sit well with the kind of "received wisdom" approach that almost every intro econ textbook takes. Nor should it.

    If kids don't like how econ is taught they can always learn it somewhere else on thier own time

    ReplyDelete
  17. Anonymous5:44 PM

    Yes. I couldn't agree more. Now that I am retired, I finally have time to read an econ textbook. I actually have several, including Mankiw's Macroeconomics, and I find the stories they tell are not only unscientific but profoundly antiscientific, in that they create the impression that if you tell a plausible story, it must be true.

    ReplyDelete
  18. I recently worked at a digital marketing startup for about 18 months. In the beginning we had a lot of great office parties, promotions and prizes were commonplace, we had a lot of downtime to take coffee breaks and surf facebook - life was good. But as the company failed to meet its growth targets, all those little perks started to go away. It seems to me that while my wage didn't change, and my employment status didn't change, there was lots of room for price adjustment in ways that aren't likely to be captured in the empirical research.

    ReplyDelete
  19. Nice reply! What you are saying is right in line with what Dani Rodrik wrote in Economist Rules:

    http://www.amazon.com/Economics-Rules-Rights-Wrongs-Science-ebook/dp/B00TIZFO90

    If you have not read this - please do so as you will certainly enjoy it!

    ReplyDelete
  20. Anonymous6:15 PM

    Dear Noah,

    Pardon my ignorance, but I have seen you write that you hold a Physics undergraduate degree. So I would like to ask, with no disrespect at all; Have you ever taken a Econ 101 class intended for Economics students? Not that this would negate any of your arguments, I just wonder if you might have a straw man impression of how Econ 101 classes are.

    Best Regards

    ReplyDelete
    Replies
    1. Why do you think Noah is referring to Econ 101 for Econ majors? I'm certain he isn't. Econ majors will get plenty of nuance in classes beyond Econ 101, it's Econ 101 for non-Econ majors that is the worry. It gives people a false and blunt idea of Econ.

      Delete