Tuesday, September 13, 2011

Sean Carroll on social science vs. natural science


I generally don't like to repost other people's blog posts in full on my blog, because it deprives the writers of pageviews. And of course the fact that I say that reliably indicates that I'm about to make an exception. Sean Carroll, who writes for the physics blog Cosmic Variance over at Discover, has written an absolute must-read post on social science vs. natural science. So here it is in full, with emphasis added by me:
Over on the Google+, Robin Hanson asks a leading question: 
Explain why people shouldn’t try to form their own physics opinions, but instead accept the judgements of expert physicists, but they should try to form their own opinions on economic policy, and not just accept expert opinion there. 
(I suspect the thing he wants me to explain is not something he thinks is actually true.) 
There are two aspects to this question, the hard part and the much-harder part. The hard part is the literal reading, comparing the levels of trust accorded to economists (and presumably also political scientists or sociologists) to the level accorded to physicists (and presumably also chemists or biologists). Why do we — or should we — accept the judgements of natural scientists more readily than those of social scientists? 
Although that’s not an easy question, the basic point is not difficult to figure out: in the public imagination, natural scientists have figured out a lot more reliable and non-obvious things about the world, compared to what non-experts would guess, than social scientists have. The insights of quantum mechanics and relativity are not things that most of us can even think sensibly about without quite a bit of background study. Social scientists, meanwhile, talk about things most people are relatively familiar with. The ratio of “things that have been discovered by this discipline” to “things I could have figured out for myself” just seems much larger in natural science than in social science
Then we stir in the matter of consensus. On the very basics of their fields (the Big Bang model, electromagnetism, natural selection), almost all natural scientists are in agreement. Social scientists seem to have trouble agreeing on the very foundations of their fields. If we cut taxes, will revenue go up or down? Does the death penalty deter crime or not? For many people, a lack of consensus gives them license to trust their own judgment as much as that of the experts. To put it another way: if we talked more about the bedrock principles of the field on which all experts agreed, and less about the contentious applications of detailed models to the real world, the public would likely be more ready to accept experts’ opinions. 
None of which is to say that social scientists are less capable or knowledgable about their fields than natural scientists. Their fields are much harder! Where “hard” characterizes the difficulty of coming up with models that accurately capture important features of reality. Physics is the easiest subject of all, which is why we know enormously more about it than any other science. The social sciences deal with fantastically more complicated subjects, about which it’s very naturally more difficult to make definitive statements, especially statements that represent counterintuitive discoveries. The esoteric knowledge that social scientists undoubtedly possess, therefore, doesn’t translate directly into actionable understanding of the world, in the same way that physicists are able to help get a spacecraft to the moon
There is a final point that is much trickier: political inclinations and other non-epistemic factors color our social-scientific judgments, for experts as well as for novices. On a liberal/conservative axis, most sociologists are to the left of most economists. (Training as an economist allegedly makes people more selfish, but there are complicated questions of causation there.) Or more basically, social scientists will often approach real-world problems from the point of view of their specific discipline, in contrast with a broader view that the non-expert might find more relevant. (Let’s say the death penalty does deter crime; is it still permissible on moral grounds?) Natural scientists are blissfully free from this source of bias, at least most of the time. Evolution would be the obvious counterexample. 
The more difficult question is much more interesting: when should, in completely general terms, a non-expert simply place trust in the judgment of an expert? I don’t have a very good answer to that one. 
I am a strong believer that good reasons, arguments, and evidence are what matter, not credentials. So the short answer to “when should we trust an expert simply because they are an expert?” is “never.” We should always ask for reasons before we place trust. Hannes Alfvén was a respected Nobel-prizewinning physicist; but his ideas about cosmology were completely loopy, and there was no reason for anyone to trust them. An interested outsider might verify that essentially no working cosmologists bought into his model. 
But a “good reason” might reasonably take the form “look, this is very complicated and would take pages of math to make explicit, but you see that I’ve been doing this for a long time and have the respect of my peer group, which has a long track record of being right about these issues, so I’m asking you to go along this time.” In the real world we don’t have anything like the time and resources to become experts in every interesting field, so some degree of trust is simply necessary. When deciding where to place that trust, we rely on a number of factors, mostly involving the track record of the group to which the purported expert belongs, if not the individual experts themselves. 
So my advice to economists who want more respect from the outside world would be: make it much more clear to the non-expert public that you have a reliable, agreed-upon set of non-obvious discoveries that your field has made about the world. People have tried to lay out such discoveries, of course — but upon closer inspection they don’t quite measure up to Newton’s Laws in terms of reliability and usefulness. 
Social scientists are just as smart and knowledgable as natural scientists, and certainly have a tougher job. But trust among non-experts isn’t demanded, and shouldn’t be based on credentials; it is given on the basis of a long track record of very visible success. Everyone would be in favor of that.
As far as I am concerned, every single thing written in this post is spot-on. I just have a few additional points:

1. What Carroll is saying is far more true of macroeconomics than microeconomics. Microeconomists are pretty good at predicting how bidders will bid at an auction, or how many people will use a new planned subway extension. (An old prof of mine once successfully managed to predict the relative amounts of denominations of money that counterfeiters would fake!) People don't often hear about these quiet successes, caught up as they are in the failures of macro. But they are real.

2. Although the lack of consensus among (macro)economists should give laypeople pause, the converse does not hold; the fact of consensus on a particular issue doesn't mean economists are right. The "Great Moderation" idea, for example, was as near to a business-cycle theory consensus as one could find, but collapsed in the wake of the recent crisis. As another example, a common defense of the DSGE modeling paradigm is that "everybody's doing it." Sorry, but no.

3. Carroll doesn't go into this, but the microfoundations of a discipline are also a factor in how much its experts should be trusted. Climatology has too long a horizon to make repeated verifiable predictions, but it's microfounded by physics that everyone agrees on. Macroeconomics, on the other hand, is microfounded on the parts of microeconomics that we don't really understand yet - consumption-saving behavior, firm investment decisions, technological change, expectation formation, risk aversion, and the like. Once micro people get these nailed down (and I think they will), we'll get some much more credible macro models.

Anyway, go check out the original link, which has some interesting links of its own.


Update: Robin Hanson responds to Carroll. The key argument is this:
Yes, economists were only a supporting influence in creating all those great economic institutions...But similarly, physicists were only a supporting influence in creating all those great physical devices...There were useful steam engines long before thermodynamics helped explain them, and it took far more than basic physics to get men to the moon, just as it took far more than basic economics to create modern industries. It is actually quite hard to say which discipline has been more useful overall to the world. 
This just seems wrong to me. You need to know Newton's Laws to hit the moon with a rocket. You don't need to know supply and demand curves to build a company. Physics theory has been absolutely essential to the creation of a number of industries, economics only to a couple, macroeconomics to basically none. And it should also be noted that before the mid-1900s, economics was non-quantitative, basically a type of philosophy. Which means that back when things like the corporation and the factory were being invented, economists' role was mainly as cheerleaders for free markets. Which was a very important role, but was a political role, not a scientific one.

14 comments:

  1. Noah, there is another factor to consider in this trust question: you should trust experts if, or to the extent that, they employ the standards of debate among each other that are conducive to the progressive approximation of "truth". These include recourse to evidence, a regard for external validity (respecting the findings of other scientists) and a severe penalty for commission of Type I error.

    Alas, on these grounds most social sciences tend to come up short. The problem is especially acute in economics, and micro does not escape this judgment. I am not saying that there aren't illuminating and useful articles being published all the time (especially in applied micro), but that the discipline as a whole discourages trust insofar as it fails to adhere to the principles that make science trustworthy. For instance, in the applied areas I work in, microeconomists sprinkle their writings with material that can only be called theological (showing how everything must be derived from utility maximization, assuming that wages are always equal to the value of marginal product, etc.). They serve no meaningful purpose other than genuflection, and they are demonstrably, empirically false in the contexts I most care about. This means I absorb the valuable parts of their work and respect the skill that goes into them -- but trust? Not really.

    ReplyDelete
  2. Anonymous3:27 AM

    You have a very obviously false understanding of science -- built on a dramatic poverty of exemplars & a what appears to be a complete vacuum in the domain of the study of the science of science ... i.e philosophy & history of science.

    ReplyDelete
  3. The social science/natural science split doesn't make much sense to me. You can consider many examples of professional groups enjoying high prestige and trust (e.g. doctors) long before they developed any useful body of knowledge.

    The blunt fact is that economists are not widely trusted because of their social role. Although not all economists make it their full-time business to provide ideological cover to political interest groups, that is the part of the business which most non-economists see most of the time.

    ReplyDelete
  4. Economics is a very new discipline; perhaps in the state of astronomy just after Kepler or Newton. The basics are still being worked out. It's early yet for macro to make substantial intellectual claims.

    ReplyDelete
  5. Economics might be a new discipline but the problem is - at least from what I can see, a lot of economists are just so damned arrogant. If the discipline is still on shaky grounds have some humility, a lot of economists seem to think they know all the answers to every social issue.

    ReplyDelete
  6. This is a very insightful post from Sean Carroll, and an interesting discussion.

    Though Anonymous (comment number 2) was rude, and didn't offer any examples, he or she does have a point, which is that work in the "hard sciences" has long overlapped with social issues and politics. In the mid-20th century and beyond, some (and perhaps many) physicists' views on the behavior of nuclei (notably fusion) were linked in complex ways to those physicists' feelings about the world geopolitical situation. This was not a simple one-way connection from physical insight to political opinion.

    Physicists are also not immune to concluding what they expect (or hope) to conclude. For example, for twenty years from 1974 to 1994, theorists kept expecting a missing particle (the top quark) to be just around the corner: measurable with the next accelerator, just slightly too high in mass to have been seen in past experiments. One eminent experimental group even claimed (and then retracted) a "discovery" of this particle long before it was actually found. It will be very interesting to see how this works out with the Higgs particle, which everyone expects to see in the next year or so at the LHC. I don't mean to pick on high-energy physicists. One could find related examples, and in one memorable case outright fraud, in my own area of condensed matter physics. But I did a master's in history of science, and one of my research focuses was the history of the search for the top quark, so I have thought more deeply about this particular case.

    With all that said, I do feel the discourse in physics (and probably in other hard sciences about which I am less expert) usually meets the criterion in Peter Dorman's comment: it is "conducive to the progressive approximation of 'truth'", though we surely could have a lively conversation about the definition of "truth".

    ReplyDelete
  7. Anonymous12:12 PM

    David,

    The problem is that if the physics don't find that particle, they will quit that model and try find a better model that fits better the reality. They done it before, that is the reason they are building greater collisors, for try find or not that particles, and if they not find that particle just show they need quit the model and try develop a better one (that problably will need more complex experiments, and that is something that makes the things easier to physics, they can make "experiments" upon their objects).

    It is a lot more effective method than have "zombie economics"...

    Maybe economists can just start to quit models that show don't work at real life again and again, but are forever "undead".

    João Carlos

    ReplyDelete
  8. Andrew is spot on. We don't trust economists as readily as physicists because of the enormous financial influences that want them to declare particular results.

    Robin Hanson, who raised the question, is a case in point. The George Mason University Economics department which he works for is well documented to have been created by Koch brother financing.

    ReplyDelete
  9. Anonymous10:57 AM

    Carol first writes:

    "good reasons, arguments, and evidence are what matter, not credentials. So the short answer to “when should we trust an expert simply because they are an expert?” is “never.”"

    He then point out that this admirable procedure is impossible to carry out:

    " But a “good reason” might reasonably take the form “look, this is very complicated and would take pages of math to make explicit..."

    and hence useless. Next up: reading your own x-rays.

    The real question is, whom does one trust. The case I've been brooding on is climate change. Here is a case where there really are experts who have a knowledge base that is of a different order than that in economics and who are in agreement. Yet they have achieved little public acceptance. As for taking action based on their findings, this looks like it won't happen until the world is doomed. What is going on?

    I see some things at work. First, people decide to accept as experts those whose findings they find convenient, ideologically acceptable, profitable, etc. It's possible that "whom do you trust" and "who is truly an expert" are the same question. I haven't made up my mind on that.

    Secondly, people choose to consider themselves experts. Even I, who rarely visit climate-related blogs and am far from expert, have encountered obviously specious arguments made with great passion by people with training or knowledge. Why are those arguments even made, let alone accepted?

    Third, those who stand to gain, at least in the short term, have been able to spread disinformation on a large scale. The well known availability heuristic then influences the ninety plus percent of the public
    who have no real information.

    Does all this add up to an explanation? If so, how is the addition done? I wish I knew; any illumination offered will be welcome.

    In the case of economics there is the additional factor that some scholars in the field, even some eminent scholars, seem quite willing to allow political and ideological commitments to override everything else. Greg Mankiw is a case in point; he's made public statements that contradict his own textbook. Some go further and decide what theories to accept based on their political implications. I won't bother with examples because no one here needs them.

    Another consideration applying to economics is that people see recommendations being applied and failing. This stimulus is an example. That there was no net stimulus, and that what was done was much better than nothing, are too subtle to affect public discourse.

    All in all, confusion. Anyone out there want to try to clear it up?

    ReplyDelete
  10. Do people really think economics has got to the level of Kepler or Newton? Would that not imply that economics has a lot more predictive power than it seems to show currently. It might be viewed less charitably that economics is in the land of astronomical epicycles (grandly baroque mathematical models that were wrong) that in few ways accurately reflect reality.

    ReplyDelete
  11. Clearly there is a significant difference between the type of science where one can use a laboratory experiment to test a mathematically derived hypothesis based upon theoretically "known" laws and the type of research that one encounters in social science. For one thing, one is not able to conduct true experiments with a control in economics (although some things are purported to come close, like lottery-derived selection mechanisms for participation in programs). Those cases where it is possible to do something like a randomized experiment, it's microeconomics, not macroeconomics, that is being tested.

    Further, generally such forces as occur in physics are constant. Factors that affect economic measures are variable and rely upon mass behavior and potentially the formation of rational expectations, etc. Thus in economics, there tend to be assumptions with limited validity. Nevertheless, those who try to understand it plug along.

    ReplyDelete
  12. Our discussion has gone way too far from the real topic. In a situation will differing expert opinions how does a non expert decision maker decide? The answer is simple, he should not make the technical decision but personnel decision. And of course be willing to fire if results do not meet expectations. President Lincoln did not make war strategy decisions, he did choose the Generals.

    ReplyDelete
  13. Anonymous6:49 AM

    The students performing social science essay writing tasks should also take note that the basic steps or procedures for writing social science essays could also be applicable for other essays on the sciences such as applied, natural, physical, cognitive, and formal science essay writing tasks.

    ReplyDelete