Theory vs. Data in economics

OK, I promised a more pompous/wanky followup to my last post about "theory vs. data", so here it is. What's really going on in econ? Here are my guesses.

First of all, there's a difference between empirics and empiricism. Empirics is just the practice of analyzing data. Empiricism is a philosophy - it's about how much you believe theories in the absence of data. You can be a pure theorist and still subscribe to empiricism - you just don't believe your theories (or anyone else's theories) until they've been successfully tested against data. Of course, empiricism isn't a binary, yes-or-no-thing, nor can it be quantitatively measured. It's just a general idea. Empiricism can encompass things like having diffuse priors, incorporating model uncertainty into decision-making, heavily penalizing Type 1 errors, etc.

Traditionally, econ doesn't seem to have been very empiricist. Economists had strong priors. They tended to believe their theories in the absence of evidence to the contrary - and since evidence of any kind was sparse before the IT Revolution, that meant that people believed a lot of untested theories. It was an age of great theoryderp.

That created a scientific culture that valued theory very highly. Valuable skills included the ability to make theories (math skill), and the ability to argue for your theories (rhetorical skill). Econ courses taught math skill, while econ seminars taught rhetorical skill.

Then came the IT Revolution, which dramatically reduced the costs of gathering data, transmitting data, and analyzing data. It became much much easier to do both high-quality empirical econ and low-quality empirical econ.

But at the same time, doing mediocre theory became easier and easier. The DSGE revolution established a paradigm - an example plus a framework - that made it really easy to do mediocre theory. Just make some assumptions, plug them into an RBC-type model, and see what pops out. With tools like Dynare, doing this kind of plug-and-chug theory became almost as easy as running regressions.

But Dynare and RBC didn't make it any easier to do really good theory. Really good theory requires either incorporating new math techniques, or coming up with new intuition. Computers still can't do that for us, and the supply of humans who can do that can't be easily increased.

So the supply of both good and mediocre empirics has increased, but only the supply of mediocre theory has increased. And demand for good papers - in the form of top-journal publications - is basically constant. The natural result is that empirical papers are crowding out theory papers.

But - and here comes some vigorous hand-waving - it takes some time for culture to adjust. Econ departments were slow to realize that these supply shifts would be as dramatic and swift as they were. So they focused too much on teaching people how to do (mediocre) theory, and not enough on teaching them how to do empirics. Plus you have all the old folks who learned to rely on theory in a theory-driven age. That probably left a lot of economists with skill mismatch, and those people are going to be mad.

At the same time (more hand-waving) the abruptness of the shift probably creates the fear that older economists - who review papers, grant tenure, etc. - won't be able to tell good empirical econ from mediocre. Hence, even empirical economists are quick to police the overuse of sloppy empirical methods, to separate the wheat from the chaff.

Now add two more factors - 1) philosophy, and 2) politics.

People have a deep-seated need to think we know how the world works. We have a very hard time living with uncertainty - most of us are not like Feynman. When all we have is theory, we believe it. We hate Popperianism - we recoil against the idea that we can only falsify theories, but never confirm them.

But when we have both facts and theory, and the two come into a local conflict, we tend to go with the facts over the theory. The stronger the facts (i.e. the more plausible the identification strategy seems), the more this is true.

The data revolution, especially the "credibility revolution" (natural experiments), means that more and more econ theories are getting locally falsified. But unlike in the lab sciences, where experiments allow you to test theories much more globally, these new facts are killing a lot of econ theories but not confirming many others. It's a Popperian nightmare. Local evidence is telling us a lot about what doesn't work, but not a lot about what does.

In physics it's easy to be a philosophical empiricist. As a physics theorist, you don't need to be afraid that the data will leave you adrift in the waters of existential uncertainty for very long. Physics is very non-Popperian - experimental evidence kills the bad theories, but it also confirms the good ones. In the early 20th century, a bunch of experimental results poked holes in classical theories, but quickly confirmed that relativity and quantum mechanics were good replacements. Crisis averted.

But that doesn't work in econ. A natural experiment can tell you that raising the minimum wage from $4.25 to $5.05 in New Jersey in 1992 didn't cause big drops in employment. But it doesn't tell you why. Since you can't easily repeat that natural experiment for other regions, other wage levels, and other time periods, you don't get a general understanding of how employment responds to minimum wages, or how labor markets work in general. Crisis not averted.

So philosophical empiricism is far more frightening for economists than for natural scientists. Living in a world of theoryderp is easy and comforting. Moving from that world into a Popperian void of uncertainty and frustration is a daunting prospect. But that is exactly what the credibility revolution demands.

So that's probably going to cause some instinctive pushback to the empirical revolution.

The final factor is politics. Theoretical priors tend to be influenced to some degree by politics (in sociology, that's usually left-wing politics, while in econ it tends to be more libertarian politics, though some left-wing politics is also out there). A long age of theoryderp created a certain mix of political opinions in the econ profession. New empirical results are certain to contradict those political biases in many cases. That's going to add to the pushback against empirics.

So there are a lot of reasons that the econ profession will tend to push back against the empirical tide: skill mismatch, the limitations of natural experiments, and the existing mix of political ideology.

Of course, all this is just my hand-waving guess as to what's going on in the profession. My guess is that econ will be dragged kicking and screaming into the empiricist fold, but will get there in the end.