Monday, December 08, 2014

Economists just don't look very ideological



A while ago I blogged about a paper by Roger Gordon and Gordon Dahl that showed that economists aren't ideological - or, at least, that their ideology doesn't line up much with the American left-right political axis. Now a trio of economists, Zubin Jelveh, Bruce Kogut, and Suresh Naidu, have come out with a very rigorous empirical analysis on the subject. They comb through a bazillion papers and do word analysis on the text! This paper is really impressive - it's very rigorous and obviously took a ton of work and careful thought.

The authors discuss their paper at 538.com, and they emphasize that they do find evidence of ideology in the way economists write their papers. But Kevin Drum, blogging about the paper, points out that the results, while statistically significant, don't look that economically significant - i.e., not that big:

ideology
What I see is a nearly flat regression line with a ton of variance. Those blue dots are all over the place. If the authors say their results are statistically significant, I believe them, but it sure looks to me as if (a) the real-world error bars are pretty big here, and (b) economists as a whole are remarkable unbiased. I mean, look at that chart again. I would have expected a much steeper line. Instead, what we see is just the barest possibility that ideology has a very slight effect on economists' findings. 
If these results are actually true, then congratulations economists! You guys are pretty damn evenhanded.
I agree with Drum. That's just not a lot of ideology. Although it's hard to interpret the meaning of the slope of the regression line - it depends on the economic impact of the policy recommendations that the y-axis represents - the fit of the regression line is obviously very low. This is true for all of the various ideology measures the authors use:


So political ideology, as far as we can tell, just doesn't explain much of the variance in academic economists' policy recommendations. Keep in mind that absence of evidence is not evidence of absence - the authors simply might not have measured ideology accurately. 

But given the fact that economists deal every day with topics that are inherently political, I'm surprised we don't find more bias in paper-writing. 

And there's also the possibility that the sample is heavily influenced by a few outliers. There are a couple of departments out there that may be influenced by big donations from political activists. That might inject a few politicized folks into the academic mix. (Update: Matthew Martin makes this point graphically.)

So overall, I think my thesis that most of econ doesn't line up along left-right political lines seems like it's borne out by the data. Still, that leaves the possibility that a few policy issues, or a few top economists, might be strongly motivated by political ideology. I'm sure we can all think of one or two anecdotes.

48 comments:

  1. I would say this about academia in general - the incentives push heavily toward results that increase reputation/prestige. This is either through eye-catching results (p-value mining, etc.) or by being very sober, rigorous, and non-ideologicallu "going where the theory/data lead and damn the consequences" (or giving the appearance of such). That's why you see prominent very right- or left-wing people in top departments - the publication in AER or Science is primal

    ReplyDelete
    Replies
    1. Anonymous4:07 PM

      And conversely while Noah's wily crusade against right wingers when he was just a nobody graduate student and then a pivot towards the SV, and particularly Anderseen, is so ingenious. On one hand he makes a name for himself -- especially among certain kind of technocratic liberals -- for attacking the obvious weak, semi-retarded points of tired old men like Cowen or Mankiw who can just muster forth the same semi-recycled crap. And of course as a bonus pile onto Ferguson the Feeble who was beaten so badly he had to flee all the way to China to peddle his crap. On the other hand with those bonifides established and a secure job for life perch secure he appeals to the SV types for more immigration and free trade that capitalist hoarders love more than anything else except tax cuts and hunting brown people for sport.

      In another 2-3 years when he becomes a partner at Anderseen Horowitz, grows a beard and starts ordering asian hookers by the busload his genius will be fully recognized but today, I salute you Noah Smith, you wily genius. And to think, you could be a depressed physicists somewhere doing meaningless analysis of some phenomenon no one would actually read and hope some old man finally keels over and dies so you can get a tenure-track position in something that might actually advance human knowledge. Bravo.

      Delete
    2. Dammit, Anonymous, your skills are just too damn good to be an anonymous commenter. You need a blog, dude. I'm comparing your diss of me to Frederik DeBoer's diss of TNR (http://fredrikdeboer.com/2014/12/05/dear-very-serious-journalists/), which is pretty darn good, and yours is just so much better it's unfair. Time to get a blog.

      Delete
    3. Anonymous4:35 PM

      That's Marc.

      -morgan

      Delete
    4. To Marc,

      There is an advantage of having a blog! Like Noah, you can pick and choose what you want to say - totally out and out bs with context from other blogs and columns.

      Noah,

      Have fun deleting my comments. Pretty thin skinned you are!

      Delete
    5. Anonymous5:34 PM

      KV,

      How is he thin skinned? He took a semi-serious 'bash' of him and used it to promote more clicks to an article he recognized wasnt going to be really read. By linking it to his prolific and entertaining twitter stream filled with all his very serious technocratic liberal fans literally half a dozen more eyes will now see this incredibly mundane but sort of worthwhile in an honest broker sort of way post. Some of them even came unneeded to his defense, in an effete liberal way that made it all worthwhile.

      As an aside, kudos to his 'honest broker' channeling De Long approach on this. ("ECONOMISTS ARE DIRTY TOOLS OF REPUBLICAN BANKSTERS" = lots of clicks, Yves Smith style when she isnt posting outright Russian propaganda to get those dirty RT.com re-clicks, hey an old lady got bills yo.
      "Economists sometimes are affected by politics but actually very rarely because the regression line is flat" will not. Even if it has that subtle and sweet " fuck p < .05 and anyone who still doesnt understand its hogwash, probably those dirty liberal arts graduate students who if they just knew the perfect amount of statistics -- not too much or of course they'd be poor, like, you know, statisticians -- would be rich like economists" meta Noah line in there

      If you are going to hate at least take the time to understand him -- being a pretty sweet guy and kind of naive American he actually drips a lot of his person into his writing. And if you werent so riddled with autism -- in the Noah Smith/internet kids these days definition -- youd probably pick up enough to at least body him. Of course doing so would just highlight that you are a piece of cock sucking quasi anonymous shit that populates marginal revolution or some other right wing echo chamber. Which your actual post tried to do I suppose. Except you know, it would have actually worked.

      Die under a stack of Ayn Rand books, you beige nobody.

      -- Actual Anonymous.

      Delete
    6. I am literally LOLing in the middle of Whole Foods.

      Delete
    7. Anonymous6:20 PM

      I rest my case.

      -morgan

      Delete
    8. Anonymous6:35 PM

      If only. To be one of the most powerful tech-tyrants of the new millennium, have Carl Icahn know you by your name. Adorned with the skull shape to act in the reboot of Coneheads without a prosthetic and the power to lure Noah away from DeLong to me? That is truly to be a god among men.

      And by men I mean really men, not chinless paper pushers fantasizing about being princes and lords just as soon as the evil government standing in my way with its evil 40% tax rate is gutted and the clock reset to the 9th century where men were men and women, gays and brown people were slaves.

      Delete
    9. Anonymous4:15 PM

      You slay me, Marc.

      Delete
    10. Noah, I'd say that the Krugman War proved that economists are in fact ideologically biased. The right was totally willing to run right-wing BS in the face of facts, for years. And the rest of the economics professoriate didn't call them on it.

      Heck, can't Prescott's calibration stuff be considered as a good data point?
      Is there a corresponding case on the liberal side where the liberal economist basically said that if the data disagree with a desired model, then the data shouldn't be used?

      Delete
  2. Wasn't there a poll that asked scientists to ask for their ideological leanings and it turned out that only a very small minority of all scientists was conservative?
    This one: http://www.people-press.org/2009/07/09/section-4-scientists-politics-and-religion/
    I think economists are still an outlier among scientists, though as a group, they probably reflect society best. A very charming explanation for this might be that the impact of economics on the public discourse is far greater than suspected. But there are other, less charming explanations as well...

    ReplyDelete
  3. Taking this seriously, I think one issue with the study is that it looks at economists across all fields, whereas the usual complaints about ideological split (e.g. "freshwater" vs "saltwater") are specific to macroeconomics - who are of course the economists most frequently in the news these days.

    ReplyDelete
    Replies
    1. That's true, and also, I know how easy it is to conceal highly politicized policy recommendations in macro papers. Even this word analysis would be unlikely to find it.

      Delete
  4. You honestly think the main body of economists' findings aren't ideological? I'm not buying it, not even that you really believe that?

    ReplyDelete
    Replies
    1. In terms of American left-right political ideology? Yes, I think most econ papers are not very ideological. Of course, you can say "All econ is ideological because it's a tool of the capitalist system" or various other things like that, but in the sense of breaking down along U.S. left-right political lines, I think most doesn't.

      Delete
    2. Phil Koop7:56 AM

      The claim was not that economist's findings have no ideological content or interpretation, but rather that very little of the variation in the ideological content of economic findings can be explained by the ideology of the author.

      Delete
    3. Phil Koop8:00 AM

      In particular, nothing stated here precludes a systematic ideological bias in conclusions if that bias is not related to the ideology of authors. That could be the case if there is publication bias, for example.

      Delete
    4. Phil, this is a great point.

      Delete
    5. Yes, and since to my mind the charge is more that economics overall tends to have a right-wing bias, that's important.

      Delete
  5. I can tell you as someone that's known Noah pretty much my entire life, I'm almost positive he couldn't grow a beard.

    ReplyDelete
    Replies
    1. Sorry man. Didn't wan't anonymous above to start planting rumors that you could grow a beard and bury yourself in Asian strange.

      Delete
    2. Real Anonymous, you have yet another fan.

      Delete
    3. Anonymous1:19 PM

      Lesson learned: never google "asian strange" and definitely never view images.

      Delete
    4. That's a lesson Chris taught me many years ago...

      Delete
  6. Some things are so self-evident that one need not subscribe to a particular ideology to believe it. For example, you cannot grow an economy by spreading the wealth of the most productive and useful members of society to the least. They tried that in Russia and it failed.

    ReplyDelete
    Replies
    1. Anonymous11:24 PM

      You are describing the Tsarist nobility as 'the most productive' in Russian society? I'd ask you to do us a favor and throttle back on the auto erotic asphyxiation but it seems like all the brain cells necessary for even rudimentary critical thought have died out. Why are you even posting here, did all the men's rights neck beards have to clean their basements or their moms wouldnt take them to next years Comicon and you have no one to talk to?

      Delete
  7. Noah,

    Suppose a professor says I'm 100% objective and unbiased in my grading: The tallest student will get the highest grade, the next tallest, the next highest, and so on. A researcher studies this professor for his impartiality, and finds he's perfectly impartial; he stuck 100% to his objective formula.

    Would you say he's 100% impartial and unbiased? Or is he severely biased, just biased in his selection of "objective" criteria, in this case against short people?

    The bias in economics comes in what kinds of models the gatekeepers allow to be published in top journals, for example ones which make assumptions that make right wing policies look better, and erase their undesirable effects (undesirable at least to the majority of people). It comes in the criteria they establish for work that will be rewarded, and how it must be done. The bias is not in language like we see in political writing, but in the kinds of assumptions which those in power permit in the models, for one.

    ReplyDelete
    Replies
    1. For two, and I could go on, as Simon Wren Lewis notes; another example of severe, and profoundly consequential, bias is that the only optimization criterion usually permitted is Pareto -- the libertarian, only with unanimous consent, criterion -- as opposed to the usually never permitted, utilitarian, total societal utilis, optimization criterion; you're not even permitted (or are strongly discouraged) to give both, and let the reader decide and utilize this information. See:

      http://mainlymacro.blogspot.com/2014/05/pareto-inequality-and-government-debt.html

      Delete
    2. That's possible. But how could we test that idea, quantitatively? Is it possible?

      Delete
    3. Well, as I'm in dire need of a dissertation, I find that intriguing (my recent hope is for a more generalized Wallace model, along with some other insights and explanation). But actually, it would be very useful to use some kind of computer search of pubs in top journals, looking for the optimization technique. Then we could quantify, among other things, the percentage of pubs that let the reader know what the total societal utils optimum is, what it looks like, and/or some kind of sensitivity or other analysis of it. Also, any pubs that just give both to the reader, the Pareto and the utilitarian, in an objective way and leave it to the reader to make her normative conclusions.

      There's a lot of this type of data you could gather, run regressions on, and do other interesting things with.

      Delete
    4. I really find it an interesting idea. With data on the type of optimization, you could run regressions to see what types of papers are more or less likely to give a total societal utility optimization, what types of authors, and their schools, changes over time, and other things having to do with the research design. And also, if the prestige of the journal is related to providing the readers the total societal utility optimum, or related information.

      Delete
    5. Daniel Davies pointed out that a society in which one person owned everything, and everybody else owned nothing, is Pareto-optimal.

      Delete
  8. I'd like to see a comparison of the results for economists who are Democrats vs Republicans vs non-affiliated.

    ReplyDelete
    Replies
    1. "I'd like to see a comparison of the results for economists who are Democrats vs Republicans vs non-affiliated."

      Considering the insanity and hatred of science by the right..........

      Delete
  9. Anonymous11:27 AM

    I am ideologically opposed to the s2color background. Does that count?

    ReplyDelete
  10. Maybe the spectrum being sampled is too narrow to show bias. If one took all economic thought on this planet, the vast majority of American academics would comprise a fairly tightly clustered group. The Judean Peoples' Front disagrees loudly with the Peoples' Front of Judea...

    ReplyDelete
  11. Am I missing something or does Drum make a big mistake by not inspecting the axes carefully? It seems the scale on the x-axis is about 1/10 that of the y axis, which wouldn't seem like a big deal if we were talking about differently-scaled variables, but it seems like they are be measured on the same units scale. I would say that this seems like a big deal. Soo... anyone check out the actual marginal effects?

    ReplyDelete
    Replies
    1. My point is, we don't even know what the y-axis represents. Even if the line were 1/10th as steep or 10 times as steep as it is now, how "much" ideology would that really represent?

      What we can see is that the regression line doesn't explain much of the variance in the ideology measure.

      Delete
  12. Anonymous4:59 PM

    Ultimately, nobody wants to confront the deepest basest fact: Normative reality is Hegemonic reality and the hegemons are not the .1% we're the 73% that spend part of our earning lifetime in top 20%... and as such most of us (online, in the workplace, in the voting booth) identify as the top half for most of if not all of our lifetime.

    And if the frame, if the bias does not assume our reality as a rule, and only as an exception, make allowances for the bottom half...

    We will throw a revolution and kill everyone trying to keep us from having our way. That's literally the definition of revolution when the top half, says NO DAMN WAY.

    Economists are not "biased" towards us, neither are businessmen, or scientists, or politicians, they are afraid, afraid of ourselves, and what we will do, if anyone actually does more than sell Che t shirts like Bruenig, and Yglesias before him.

    We aren't going to create a game that doesn't reward good parents with more opportunities for their children, NO MATTER WHAT.

    So to truly be a wonk, accept it and work within the normative reality.

    -morgan

    ReplyDelete
  13. The correlation is weak (.40) even after using a binary function to cluster all economists into strict left and right-wing buckets, based on an dubious set of n-grams (e.g. "SEC" = 'right wing').

    There is rich irony in economists declaring themselves unbiased by constructing an ineffective model and proving that it has no predictive value when it comes to their biases.

    ReplyDelete
  14. Anonymous7:52 AM

    My feeling:
    How is defined an "economist" in this study?
    What "political options" are considered?
    How are they measured?

    ReplyDelete
  15. Anonymous10:10 AM

    MMMMM? Question the assumptions.
    Maybe the whole damn field of economics is biased.

    ReplyDelete
  16. People forget that Economics isn't Math! Economics is the social science! I like style of Shiller and his approach.

    ReplyDelete
  17. Anonymous12:53 PM

    the 538 chart has been updated. Seems to have been an error going from paper to blogpost. Relationship appears stronger now.

    ReplyDelete
  18. Anonymous3:07 PM

    The line in the scatterplot does not look like it is influenced by outliers.

    FWIW, my prior is that economists as a whole are not very biased, but given the flaws here (as mentioned in several comments above), this does seem somewhat damning. Consider that every one of seven steps in the process has measurement error, which gets compounded. Remember that measurement error attenuates (makes closer to 0 than the true value) correlations. Thus, even though the resulting correlation is only ~.1, the true level of bias could be quite meaningfully large (and even that misses the most fundamental biases, and collapses over all economists / subdisciplines).

    ReplyDelete