Russ Roberts and the new empirical world

My last post, about Russ Roberts' EconTalk interview with David Autor, was requested by Russ as part of a general symposium on the topic. But it also reminded me that I had wanted to blog about two earlier EconTalk episodes - Russ' interviews with Joshua Angrist and Adam Ozimek.

These interviews, along with my own and Autor's, are basically part of a series. In this series, a bunch of people basically try to convince Russ that empirical economics matters a lot. Russ plays the part of the econ public - the average older academic, plus the econ-literate community of policy wonks, blog readers, and the like. He starts off basically thinking that econ is about theory (as it really was back when he was learning his chops). Then, through a successive series of conversations, he becomes convinced that not only have things changed, but that this is as it should be.

First, he talks to Angrist. Angrist is the chief academic evangelist for the "credibility revolution," the methodology at the heart of the new empirical revolution. Angrist tells him about a whole raft of exciting results that have come out of this methodological shift. Russ doesn't really know these methods very well, since only recently have they been codified, systematized, and popularized. So he doesn't quite know how to respond to Angrist's assertion that a revolution is underway. He keeps asking things like "But your new methods aren't convincing everybody...right?" And Angrist gets a little frustrated, basically responding that it's impossible to convince someone with evidence if they don't take a careful look at said evidence. If Russ were more up on this debate, he could have given Angrist a stiffer challenge, bringing up external validity concerns, or the fact that the new methods don't really work in macroeconomics. But remember, the whole point of this series of interviews is to introduce Russ into the new world! There will be time for pointed criticism later.

Second, Russ talks to me. I am a snarky, combative blogger who has trumpeted econ's empirical turn as something to be celebrated - something that will make the whole field more scientific. Russ and I have a great discussion and debate. Unlike Angrist, who focuses on research methodology and doesn't really bother going around asking people if they've changed their minds, I can offer a few concrete examples of economists who have shifted their views in response to empirical results, both of the "credibility revolution" type and other types like structural models and time-series macroeconometrics. I also rant about my favorite hobbyhorse, i.e. the value of empiricism. Russ is more convinced, but my combative nature probably leaves him uneasy with simply accepting my assertions.

Next, he talks to Adam. Adam makes basically the same points I do, but more eloquently and in a gentler fashion. Adam has been involved in an extensive debate with Russ over the value of convictions vs. evidence in economics. In his interview, Adam declares that although empirical results should be interpreted with humility, "stories and our own narratives about the world need even more humility," and we need more skepticism of our basic lenses [i.e. worldviews and ideologies] than we do of empiricism." Those are beautiful lines. Adam also cites empirical literatures that have convinced large numbers of economists to change their minds (to varying degrees) about big questions. Some are the same examples I cite, but he adds the example of trade, and cites Autor's work.

Russ is no longer nearly as dismissive of empirical econ, but he is still firmly in the role of the skeptic. He consistently refers to "sophisticated" econometric techniques, as opposed to "casual" evidence that he finds generally more convincing (he is not alone). The first few dozen times I heard Russ use the term "sophisticated" to refer to things like credibility-revolution econ, I was flummoxed, since many of these methods are simple comparisons of means. Suppose I tell my dog to bite your leg,* and I compare the number of bite marks after the attack to the number before - how sophisticated is that? But eventually I realize that by "sophisticated", Russ just means "systematic". He's saying that he trusts the evidence of his own eyes more than the opaque systematic data collection and analysis methods of empirical economists.

If I had realized that during my own interview, I would have pointed out that this is how science always works. Most people don't know how scientists know the distance from the Earth to Mars, or the mass of the electron, or that bacteria cause the plague. The scientific consensus on these topics was created by insiders - people with knowledge of both the theories and the empirical methods - convincing other insiders. Educated outsiders don't understand the theories and the methods, but they trust the scientists to get things right (and once in a while they even trust the scientists too much). Uneducated outsiders don't even trust the scientists - they believe vaccines cause autism, or that the world is flat, or that diseases are caused by demon possession, or whatever.

Russ isn't an uneducated outsider - he's an educated outsider in a field where the scientists (or whatever you want to call them, empirical researchers) haven't yet built up the trust and credibility to have their results gain instant acceptance by educated outsiders. Scientists cured the plague and put humans on the moon, but economists have a much harder job applying most of their results, since the results are usually about policy, and policy is ponderous, slow, clumsy, and subject to lots of random shocks. So Russ has not yet been shown the money, so to speak, and he's understandably wary.

But like most economists, Russ is smart, open-minded, and intellectually honest. In his interview with Adam, you can see that he is warming to the idea of the empirical revolution. Russ has been deeply disturbed by the ideological tone of the macroeconomic debates since 2008, and he recognizes that people might interpret casual evidence in ways that confirm their own worldview. He talks about psychological rigidity and people's tendency to defend their previous statements and positions. He worries that sloppy empirical work might be used by tendentious researchers as a front for their preferred policy positions (and this is a real worry), but he seems to realize that casual or anecdotal evidence is far more subject to this sort of misuse. Adam's contention that we need to be even more skeptical of our ideologies than of our evidence hits home.

So Russ is getting more into the empirical world here. In his interview with Autor, he doesn't question the validity of empiricism as a research method, but focuses on understanding the results in the context of theory. He does not ask many pointed questions about the research methodologies Autor uses, but he is not dismissive of the idea that empirical econ could poke holes in a hallowed policy consensus.

This is great to see. Though Russ is considerably more libertarian than the average economist, I think of EconTalk as a sort of a barometer for the wider econ world. The things Russ thinks out loud on the air must be things that thousands of older econ profs are thinking silently in their seminars, and that thousands of policy wonks are thinking silently while reading blog debates. The older generation came of age in a time where theory was king, and it's going to take some time to adjust to the new world. But that adjustment is happening.

So who should Russ have on EconTalk next? How should this series continue? I'd like to see some more top empirical researchers get a chance to strut their stuff. For example, he could get John Haltiwanger on to talk about declining dynamism in the American economy. He could get Raj Chetty to talk about mobility, education, and inequality. He could get Susan Athey to talk about machine learning techniques. Guests like this will continue to convince him that empirical econ is making lots of useful and reliable discoveries, as well as publicizing these discoveries to his many listeners!


Updates

Looks like I missed a couple of "episodes" in this "series". Here's Russ talking to Ed Leamer way back in 2010, and to James Heckman more recently.