Three ways of understanding the world

P-E Gobry has a narrow view of what science is:

[L]et me explain what science actually is. Science is the process through which we derive reliable predictive rules through controlled experimentation. That's the science that gives us airplanes and flu vaccines and the Internet... 

Countless academic disciplines have been wrecked by professors' urges to look "more scientific" by, like a cargo cult, adopting the externals of Baconian science (math, impenetrable jargon, peer-reviewed journals) without the substance and hoping it will produce better knowledge.

Predictably, one thing this leads to is the conclusion that economics isn't a science:

Since most people think math and lab coats equal science, people call economics a science, even though almost nothing in economics is actually derived from controlled experiments. Then people get angry at economists when they don't predict impending financial crises, as if having tenure at a university endowed you with magical powers.

If you want a rebuttal to the "econ isn't a science" thing, Adam Ozimek has one here.

But I want to make a different point. I think Gobry is right that there's something special about controlled experiments, whether or not you want to restrict the word "science" to mean only that. But there are other ways of systematically understanding the world. In fact, I think there are 3 big ones:

Method 1: History

One way of systematically understanding the world is just to watch it and write down what happens. "Today I saw this bird eat this fish." "This year the harvest was destroyed by frost." "The Mongols conquered the Sung Dynasty." And so on. All you really need for this is the ability to write things down.

This may sound like a weak, inadequate way of understanding the world, but actually it's incredibly important and powerful, since it allows you to establish precedents. What happened once can happen again. Maybe you don't know why, or how likely it is, but you know a bad harvest or a Mongol invasion isn't out of the realm of possibility, and that is valuable knowledge.

Method 2: Empirics

A second way of systematically understanding the world is repeated observation. This is where you try to make a large number of observations that are in some way similar or the same, and then use statistics to identify relationships between them. This is most (though not all) of how economists understand the world.

The first big limitation of empirics is omitted variable bias. You can never be sure you haven't left out something important. The second is the fact that you're always measuring correlation, but without a natural experiment, you can't isolate causation.

Still, correlation is an incredibly powerful and important thing to know.

Method 3: Experiments

Experiments are just like empirics, except you try to control the observational environment in order to eliminate omitted variables and isolate causality. You don't always succeed, of course. And even when you do succeed, you may lose external validity - in other words, your experiment might find a causal mechanism that always works in the lab, but is just not that important in the real world. This is a big big problem for psychology, including prospect theory. 

Experiments give you information about mechanisms. When these mechanisms have external validity - for example, when the same process that moved balls down ramps on Galileo's desk happened to be the one that moves the planets in their orbits - then experimental science (what Gobry just calls "science") is incredibly powerful, more powerful than either of the other techniques. But it doesn't always work.

You may be thinking: Where does theory fit in with all this? My answer is that theory is part of all three of these. Theory is needed to understand causal mechanisms found in experiments, to explain correlations found with empirics, or to isolate the important features of a historical event. Sometimes the theory comes before the observations, sometimes afterward.

You may also be thinking: Where do natural experiments fit in with all of this? Well, they're kind of Method 2.5. The boundaries between these methods aren't always perfectly clear, in any case.

So what we've got here is a sort of hierarchy of ways of understanding the world. There's a tradeoff between general applicability and the amount of knowledge you get. Experiments rarely work, but when they do you get a lot of understanding. History works any time, but you rarely understand why things happen. Empirics is (are?) in the middle.

But what I see is a lot of people dissing empirics as somehow inferior to experiments. That's what's really behind the "econ isn't a science" trope. Why does this happen? Don't people get that empirics, though less powerful than experiments, can be applied in a much wider range of situations? 

My guess is that it's all because empirics came out of order. History is cheap, and experiments are also (sometimes) very cheap - think of Mendel growing peas in his garden. But empirics usually requires Big Data, which is expensive. And even the simplest empirics requires statistics. So while we got written history over 8000 years ago, and experiments almost 1000 years ago, we didn't get modern statistical empirical methods until maybe 100 or 200 years ago. And only recently, with the rise in information technology, have empirics really exploded.

To a lot of people, the empirics revolution must seem like a step backward. We look back to the huge successes of chemistry and physics and medicine in the last few centuries, and the rock-solid theories they generated, and we compare it to the regressions economists are running nowadays, and we say "Ugh, this isn't science!" We look at the progression from history to experiment, and we think that new methods (if they exist) should go the same way - i.e., they should lead us to deeper understanding. But empirics, instead, goes in the direction of wider applicability with less-deep understanding, and that rankles some people.

I don't think they should be rankled. Empirics is an innovation that allows us to know some things about big phenomena that previously we could only understand through written history. It's not a substitute for experiments, it's a complement. It's a valuable addition to humanity's toolkit, whether you want to call it "science" or not.