If mathiness were used infrequently,...it would do localized, temporary damage. Unfortunately...as the quantity increases, mathiness could do permanent damage because it takes costly effort to distinguish mathiness from mathematical theory.
The market for mathematical theory can survive a few...articles filled with mathiness. Readers will put a small discount on any article with mathematical symbols, but will still find it worth their while to work through and verify that the formal arguments are correct, that the connection between the symbols and the words is tight, and that the theoretical concepts have implications for measurement and observation. But after readers have been disappointed too often by mathiness that wastes their time, they will stop taking seriously any paper that contains mathematical symbols. In response, authors will stop doing the hard work that it takes to supply real mathematical theory. If no one is putting in the work to distinguish between mathiness and mathematical theory, why not cut a few corners and take advantage of the slippage that mathiness allows? The market for mathematical theory will collapse. Only mathiness will be left. It will be worth little, but cheap to produce, so it might survive as entertainment.
[I]n the new equilibrium: empirical work is science; theory is entertainment. Presenting a model is like doing a card trick. Everybody knows that there will be some sleight of hand. There is no intent to deceive because no one takes it seriously. Perhaps our norms will soon be like those in professional magic; it will be impolite, perhaps even an ethical breach, to reveal how someone’s trick works.
When I learned mathematical economics, a different equilibrium prevailed. Not universally, but much more so than today, when economic theorists used math to explore abstractions, it was a point of pride to do so with clarity, precision, and rigor...If we have already reached the lemons market equilibrium where only mathiness is on offer, future generations of economists will suffer.
Romer has now joined the chorus of old famous guys - Krugman, Solow, Stiglitz, Farmer - who are very vocally mad about the way mainstream economics theory is done.
Romer is not afraid to name names. Interestingly, although he's talking only about growth theory and not about business cycle theory, most of the people he's mad at are the same guys that the Keynesians are mad at - Robert Lucas, Ed Prescott, and David K. Levine. He also calls out Thomas Piketty.
Romer gives specific examples of what he calls mathiness (links are to working-paper versions):
1. Prescott and McGrattan (2010): Romer says that this paper includes a term that the authors label "location," but that doesn't correspond to any real measure of location.
2. Boldrin and Levine (2008): Romer criticizes this paper for assuming that a monopolist would also be a price-taker, and for making various hand-wavey arguments.
3. Lucas (2009): Romer criticizes this paper for making a hand-wavey argument to dismiss the idea that investment in embodied technology (books, blueprints, etc.) can be a source of sustained growth, when there are well-known models in which it can. Romer also points out a random math error in the paper, and uses this to argue that reviewers don't pay close attention to math.
4. Lucas and Moll (2014): Romer criticizes this paper especially harshly. Lucas and Moll claim that their model, in which there is no creation of new knowledge, is "observationally equivalent" to models in which new knowledge arrives very slowly. Romer shows that the truth of this claim depends on which order you use when taking a double limit. He reveals that he told the authors about the problem, but that they ignored him and left it in the paper.
5. Piketty and Zucman (2014): Romer points out the by now well-known "gross vs. net" problem in Piketty and Zucman's definition of savings.
All in all, this seems like a pretty loose collection of criticisms. Hand-wavey arguments, dubious definitions, bad assumptions, and math errors are all very different things. So this essay at first can seem like a grab-bag of gripes that Romer has with individual rivals' papers.
But I think Romer is on to something about the culture of econ theory, at least in the "macro"-ish realms of growth, business cycle, macro-labor, macro-trade, and macro-tax theory (I don't know nearly as much about the culture of the "micro"-ish fields like game theory, decision theory, I/O, etc.; and I know that finance theory has a very different culture). In these "macro"-ish fields, people seem to view math more as a tool for stylized description of ideas than as a tool for quantitative prediction of observables.
Romer's examples of "mathiness" are all very recent examples. But going back to earlier models, I don't really see much more tight connection of variables to observables. Yes, in a Solow model you can tie capital K to observable things like structures and machines and vehicles. But you'll be left with a big residual, A. Then you can break A down and extract another term for human capital, H. Can you really measure human capital? Human capital can't be bought and sold on a market; you have to bundle it with other goods. So it's very difficult to get a clean measurement of the value of the existing stock of human capital, the way you could get a clean measurement of the existing stock of delivery trucks. Romer cites human capital as a good example of non-"mathiness", but I don't really see a huge difference between that and the "location" used by Prescott and McGrattan (2010). Maybe a minor difference, but not a huge one. As for the remaining A, there's not really any quantitative way to measure the stock of ideas except as a residual. And as for goofy assumptions, well, any growth model is going to have at least one or two assumptions that would make a newcomer to the econ field throw up her hands in disbelief.
Mathiness isn't anything new, it's just the way these econ fields work. The math is there as a storytelling aid (and possibly as a signal of intellectual ability). I think Karthik Athreya said it best:
My view is that a part of what we do is "organized storytelling, in which we use extremely systematic tools of data analysis and reasoning, sometimes along with more extra-economic means, to persuade others of the usefulness of our assumptions and, hence, of our conclusions...This is perhaps not how one might describe "hard sciences".
Do the guys Romer calls out play a little faster and looser with definitions and rely more on hand-wavey arguments? Oh, I'm sure they do - but that's because they're famous old guys. Writing down hand-wavey stuff is a privilege afforded to famous old guys in every academic discipline I know of. In econ, it gets politely published in top journals, but all the hotshot young people just sort of shake their heads anyway, and the only net effect is to pad out the length of the journals. Are the guys Romer calls out more political than the average economist? Maybe.
But in general, the whole discipline of macro theory - in the general sense, including growth and parts of labor, trade, and tax theory - is chock full of mathiness. Even most of the best models ("best" being a highly relative term, of course). The original Solow model seems to me like a rare exception, not a typical example of the Good Old Stuff.
But in any case, I highly recommend the Romer piece, which is a master class in catching errors in models, as well as a fascinating window into the Byzantine world of academic politics.
Update: Brad DeLong has a follow-up post explaining his view of some of the history behind the argument in the field of growth economics. Basically, the idea is that George Stigler didn't like people using models with imperfect competition, since this might open up a window for government intervention. DeLong thinks that Lucas and other "freshwater" types inherited this anti-imperfect-competition bias, causing them to be too down on Romer's models. This is interesting history that I didn't really know about before.
Update 2: Paul Romer has a response on his blog. Excerpts:
It appears that Prescott, Lucas, Levine, and others of the unofficial "freshwater" club have annoyed more high-level colleagues than just Paul Krugman. Only a few months ago, Roger Farmer took to his blog to unleash an anti-Prescott blast. Romer and Farmer are not politically-minded media-engaged types like Krugman and DeLong, but their aggravation with the Lucas/Prescott school is, if anything, even more intense.
And yes, Romer is right that I'm jaded. Is it so obvious? *takes swig from hip flask, rubs beard stubble*
But in any case, I highly recommend the Romer piece, which is a master class in catching errors in models, as well as a fascinating window into the Byzantine world of academic politics.
Update: Brad DeLong has a follow-up post explaining his view of some of the history behind the argument in the field of growth economics. Basically, the idea is that George Stigler didn't like people using models with imperfect competition, since this might open up a window for government intervention. DeLong thinks that Lucas and other "freshwater" types inherited this anti-imperfect-competition bias, causing them to be too down on Romer's models. This is interesting history that I didn't really know about before.
Update 2: Paul Romer has a response on his blog. Excerpts:
Noah Smith asks for more evidence that the theory in the McGrattan-Prescott paper that I cite is any worse than the theory I compare it to by Robert Solow and Gary Becker...
There is no such thing as the perfect map. This does not mean that the incoherent scribbling of McGrattan and Prescott are on a par with the coherent, low-resolution Solow map that is so simple that all economists have memorized it. Nor with the Becker map that has become part of the everyday mental model of people inside and outside of economics...
Noah’s jaded question–Is the theory of McGrattan-Prescott really any worse than the theory of Solow and Becker?–may be indicative of what many economists feel after years of being bullied by bad theory. And as I note in the paper, this resignation may be why empirically minded economists like Piketty and Zucman stay as far away from theory as possible...
For specific purposes, some maps are better than others. Sometimes a subway map is better than a topographical map. Sometimes it is the other way around. Starting with any good map, we can always increase the resolution and add detail.
No map is perfect, but this does not mean that all maps are equal. It certainly does not mean that an internally consistent map that with so little detail that you can memorize it is on a par with incoherent scribbling.In the rest of the post, he goes into depth about why he thinks the McGrattan and Prescott paper constitutes "incoherent scribbling." But he also notes that the other papers he goes after in his "mathiness" piece should not be let off the hook:
Noah also notes that I go into more detail about the problems in the Lucas and Moll (2014) paper. Just to be clear, this is not because it is worse than the papers by McGrattan and Prescott or Boldrin and Levine. Honestly, I’d be hard pressed to say which is the worst. They all display the sloppy mixture of words and symbols that I’m calling mathiness. Each is awful in its own special way.He also has another short post about a Lucas paper.
It appears that Prescott, Lucas, Levine, and others of the unofficial "freshwater" club have annoyed more high-level colleagues than just Paul Krugman. Only a few months ago, Roger Farmer took to his blog to unleash an anti-Prescott blast. Romer and Farmer are not politically-minded media-engaged types like Krugman and DeLong, but their aggravation with the Lucas/Prescott school is, if anything, even more intense.
And yes, Romer is right that I'm jaded. Is it so obvious? *takes swig from hip flask, rubs beard stubble*
Good points. Deirdre McCloskey had already covered some of this stuff years ago.
ReplyDelete"Writing down hand-wavey stuff is a privilege afforded to famous old guys in every academic discipline I know of."
ReplyDeleteIt's true! I'd point to Leonard Susskind as a good example in physics:
http://arxiv.org/abs/1402.5674
This isn't to say Susskind doesn't know what he's talking about though (I think he's great) ... I'd liken it to famous artists in their later periods heading towards abstract expressionism.
Noah,
DeleteActually in looking more closely at it I'm not sure Romer is right -- at least about Lucas and Moll (2014).
http://informationtransfereconomics.blogspot.com/2015/05/the-irony-of-paul-romers-mathiness.html
One order of the limits of the double limit does not make physical sense. Sure it makes mathematical sense, but not in a real economy. Ironically it means Romer is guilty of mathiness in pursuit of Lucas!
Step one : We admitted we were powerless over mathiness—that our lying had become unmanageable.
ReplyDeleteWelcome to Economoholics Anonymous , Paul , and congratulations on taking the first step on your road to recovery.
Marko
I found the paper difficult to follow. Regardless, the asymmetric information point is compelling. An author understands her argument better than her audience does. Moreover, it is less costly to produce an unsound argument that appears sound than a truly sound argument. Consequently, it is in the interests of the author to produce unsound arguments that appear sound, and knowing this, the audience won't buy what the author is selling. The solution is to introduce peer review, wherein disinterested editors evaluate whether the arguments are sound. In turn, however, this introduces a principal-agent problem, since the audience cannot perfectly monitor the performance of the editors.
ReplyDeleteThe solution to a principal-agent problem is to align the incentives of the editors with the interests of the audience. My question is, what is the misalignment which leads to approval of invalid or irrelevant mathematical arguments? How do editors benefit from this, besides not having to carefully evaluate all of the math in front of them?
Old guys are powerful. If the editor wants to be incurred to the fancy meetings and get his own paper published when peer reviewed by old guys, he has to publish old guy mathiness. Plus old guys ranting about stuff with our without maths bring audience and prestige to the journal,
DeleteWouldn't this be solved by anonymous editors?
DeleteI read Romer (2015) as being motivated by what he thinks are two problems in mathematical economics:
ReplyDelete1. The abandonment of operationalism in growth theory.
2. The continued publication of models in which firms are price-takers in the economic growth literature. Romer believes that this cannot be justified on the basis of both macro and micro data.
Romer believes that if operationalism was adopted or (at least) more closely adherred to, then problem #2 would vanish. Operationalist models, by being falsifiable in principle, allow one to evade the spectre of academic politics. Since the price-taking assumption is the wrong way to go in Romer's mind, it is natural that price-taking models either:
A. Don't fit the facts of economic growth. (5. in your list, although this one is the most out of place I admit)
B. Fit the facts of economic growth by:
i. introducing a self-contradictory assumption. (3. and 4. in your list)
ii. introducing an obviously false (IRL) assumption (usually involving non-observables). (1. and 5. in your list)
iii. not actually being a price-taking models, but masquerading as price-taking models for what Romer thinks are political reasons. (2. in your list)
In general, I read Romer as saying there are three ways mathematical economists evade operationalism:
I. Outright evasion: no connection with real data is possible. The results are driven by assumptions on non-observational variables. Since the assumptions of economics are typically quite heroic and referees/editors are no good at evaluating models' plausibility and conclusions on the basis of assumptions alone, models without measurables are especially deleterious to the advancement of growth economics as a science. This is why I think 1 and 3 in your list are included.
II. Sophistry and linguistic obfuscation: the language used in the models to describe the mathematics does not correspond to either the actual assumptions or the quantities one might need to measure IRL to assess the theory. This why Romer makes what many have called "personal attacks" on authors in the mathiness paper. The Boldrin and Levine paper, to Romer, is obviously written in bad faith in the sense that the authors obtain some neat conclusions using the monopolistic competition model and then use language to make it seem that their model provides evidence in favor of price-taker economies. This is a purely nominal problem can be "solved" by editors simply enforcing an honesty norm on published papers. I think this is why 1, 2 and 3 are included.
III. Mathematical: any connection with real data results in immediate contradiction with the theory. In order to get the paper into a top journal, this failing must be obscured through the use of rigorous or less rigorous mathematics. This is why 1, 4 and 5 are included.
Tactics I. II. and III. might be called "mathiness" in the sense that they rely on loose or ambiguous connections between observables IRL, equations, and the description of those equations in an academic article.
If you've not before perceived the tendency of freshwater types to denigrate things that 'open up a window for government intervention' you gotta open your eyes.
ReplyDeleteIn a way, the abstract use of mathematics in economics constitutes freshwater economics, especially the self-correcting nature of the neoclassical economy and the equilibrium. To come to a point where X reliably becomes Y, is to have found an equilibrium. But with every financial crash, with every cartel, with every monopoly and with every default, the equations cannot be solved anymore. An equilibrium has yielded to a cascade. So an extra variable, the government, becomes necessary. To accept the government is sort of acknowledging the imperfections of mathematics to perfectly describe the economy. It is an ideological conflict, but also a conflict between academic traditions.
ReplyDelete"To accept the government is sort of acknowledging the imperfections of mathematics to perfectly describe the economy."
DeleteNo its not. The problem is not with mathematics as such but with the models. If that wasn't the case Romer wouldn't have anything to complain about.
Mathiness is Next to Growthiness
ReplyDeletehttp://econospeak.blogspot.com/2015/05/mathiness-and-growthiness.html
It is bracing to see the intense (dare I call it petulant?) indignation expressed by Paul Romer toward papers by McGrattan and Prescott, Lucas and Moll, and Boldrin and Levine. He goes so far as to confess "embarrassment" that his suggestions as discussant were acknowledged by McGrattan and Prescott in an earlier version of their paper. He complains of "a lemons equilibrium in the market for mathematical theory" and laments "years of being bullied by bad theory."
Superficially, Romer's diatribe against mathiness may recall Nicolas Georgescu-Roegen's principled objection to unhinged "arithmomorphism." But any perceived resemblance is purely coincidental.
Georgescu-Roegen was described in a critical note as "the methodological conscience of the profession for over a decade" whose mathematical renown rendered "his closely argued objections to the domination by mathematical methods... all the more welcome."
Nice article. This quote in particular stood out:
Delete"By this account, then, the value of excessive mathiness was that it enabled mediocre junior faculty survive and get promotion [...]"
Ahem Williamson ahem.
Maybe Romer should be more concerned about the fact that his endogenous growth models are inconsistent with the growth facts and less about other people's work.
ReplyDeleteThat's interesting. I actually don't follow the empirical growth literature at all. Can you send me one or two links to papers about this?
Delete> empirical growth literature
Delete> growth literature
> empirical
I don't know where Romer criticizes Piketty for 'mathiness'
ReplyDeleteHere he actually identifies him as an empirical economist with every right to ignore theory if the mathiness keeps up
"The resignation is why I conjectured that we are stuck in a lemons equilibrium in the market for mathematical theory. Noah’s jaded question–Is the theory of McGrattan-Prescott really any worse than the theory of Solow and Becker?–may be indicative of what many economists feel after years of being bullied by bad theory. And as I note in the paper, this resignation may be why empirically minded economists like Piketty and Zucman stay as far away from theory as possible."
http://paulromer.net/ed-prescott-is-no-robert-solow-no-gary-becker/
I
Ok, I see where. I missed that part above. Sometimes I skip around!
ReplyDeleteThe Solow model seems to me like a rare exception [...]
ReplyDeleteNot sure I follow: the Solow model is just as bullshitty and empirically unsubstantiated as the rest of the literature. But then early on you yourself point out how problematic K is, so maybe I'm reading you wrong...
Indeed. Romer criticizes Joan Robinson for academic politics when, in fact, she was only making the same argument of mathiness regarding the use of capital that triggered the Cambridge Capital Controversy.
DeleteIn Paul Romer's paper on mathiness, he gives mobile phones as an example where increasing returns to scale are very important. But to prove his point, he presents a mathematical model where there are no economies of scale! That is, in his model "surplus scales linearly in [market size]", but he still implies that his model demonstrates strong economies of scale.
ReplyDeletePerhaps I'm missing something, but Isn't this exactly the mathiness that he's denouncing?
I am wondering if Jean Baudrillard is becoming more prescient by the day:
ReplyDeletehttp://en.wikipedia.org/wiki/Jean_Baudrillard
------------------------
...His pictures of society portray societies always searching for a sense of meaning—or a "total" understanding of the world—that remains consistently elusive. In contrast to poststructuralists such as Michel Foucault, for whom the formations of knowledge emerge only as the result of relations of power, Baudrillard developed theories in which the excessive, fruitless search for total knowledge lead almost inevitably to a kind of delusion. In Baudrillard's view, the (human) subject may try to understand the (non-human) object, but because the object can only be understood according to what it signifies (and because the process of signification immediately involves a web of other signs from which it is distinguished) this never produces the desired results. The subject, rather, becomes seduced (in the original Latin sense, seducere, to lead away) by the object. He therefore argued that, in the last analysis, a complete understanding of the minutiae of human life is impossible, and when people are seduced into thinking otherwise they become drawn toward a "simulated" version of reality, or, to use one of his neologisms, a state of "hyperreality". This is not to say that the world becomes unreal, but rather that the faster and more comprehensively societies begin to bring reality together into one supposedly coherent picture, the more insecure and unstable it looks and the more fearful societies become.[14] Reality, in this sense, "dies out".[15]
-------------------------
I would like to say something about an old post -- on micro-foundations. I think to insist on micro-foundations is like saying if we don't fully understand quarks and string theory, we should throw Newtonian physics out of the window. The fresh water types insist theirs is the truly scientific approach, while in reality, they're anti-science cultists.
ReplyDeleteMy 2 cents.
No matter how much fancy math the fresh water types use in their papers, it's basically political biases wrapped in technicality appearance. That's why the complete disregard of reality and evidence. You can put as much lipsticks on a pig as possible, but it's not gonna turn into a beautiful woman.
ReplyDeleteAnd repeatedly posting nonsense on the Internet will not turn it into sense. Your 2 cents aren't even worth 1.
DeleteAnonymous,
DeleteYou did not address the bias, and disregard of evidence issues.
To elaborate: IS-LM type of model is sort of like Newtonian physics, which RBC types think is worthless and should be thrown out, while in reality RBC theory is illusion in people's mind but pretending to be "real science".
ReplyDeleteOnce again, I find myself at a loss trying to reconcile the internet depiction of Lucas as a rigid ideologue with what I actually find when I read him.
ReplyDeleteFor example: DeLong says he was anti-imperfect competition because of the Stigler influence. Check out Lucas's 1985 review of Krugman's book (need JSTOR access).
http://links.jstor.org/sici?sici=0022-3808%28199006%2998%3A3%3C664%3ATPAMS%3E2.0.CO%3B2-2
I wish I had more time to weigh in on this, before it's five minute blog expiration, but I'll say this:
ReplyDelete1) Romer is criticizing mathiness in economics, not math, which he certainly sees as valuable if applied well.
2) With a model, all that is essentially said is, if this stuff is true, then this other stuff is true, or will happen. The model doesn't, in of itself, say this is what will happen in the real world, where the initial if's, or assumptions, are never true completely, and may be extremely untrue.
Thus, as I always say, even though I have no name, a model is only as good as its interpretation, to reality. The big problem with Prescott and Lucas and gang, is that they prove something is true in the model and then they assert that it's also true in the real world, where the fantastical assumptions they make are usually comically materially untrue – but hey it makes their libertarian/plutocratic ideology look better, and that's what counts.
A model can teach important lessons about the real world, but if it's a good model, and if it's interpreted to reality intelligently and using what we (often painfully obviously) know about the real world, not ignoring any other knowledge we have.
One model in particular, which I've actually taken apart and completely understand, is Wallace's 1981 AER. It's often used as "proof" that quantitative easing can't have any effect. To even most economists, it's an impenetrable wall of math, so they can't tell. But, I'm about to have my big post explaining the intuition, which basically shows that the result of "irrelevance" depends on ridiculous assumptions that clearly when they hold as little as they do in the real world mean that quantitative easing, especially large and unconventional, can have a big effect. And this is what the empirical evidence shows. But if you understand the model, you can see right away that it's not going to hold in the real world. It just depends too much on fantastical assumptions that the evidence is comically obvious come nowhere close to holding.
Ooh, Richard Serlin explains Neil Wallace! You really don't have a clue how irrelevant you are.
DeleteNoah, can you explain the meaning of the image?
ReplyDeleteHow about:
ReplyDeleteBenjo water economists?
Sewage water economists?
Run off water economists?
Waste water economics?
ReplyDeleteVader appears to be trying to figure something out, or in any case do something, without a whole lot of dexterous rigor. I think that way of proceeding is what the image is meant to illustrate.
ReplyDelete