(This post originally appeared at Bloomberg View. It's sort of a mass-market version of this earlier post.)
If you care at all about what academic macroeconomists are cooking up (or if you do any macro investing), you might want to check out the latest economics blog discussion about the big change that happened in the late '70s and early '80s. Here’s a post by the University of Chicago economist John Cochrane, and here’s one by Oxford’s Simon Wren-Lewis that includes links to most of the other contributions.
In case you don’t know the background, here’s the short version: Around 1980, macroeconomists abandoned the models they had been using and switched to something very different. The old kind of model was called structural econometric modeling (SEM), based on equations for economic aggregates -- investment in office buildings, consumption of cars, etc. These models were also called “Keynesian,” because they usually included some assumptions that were loosely based on the writings of economist John Maynard Keynes. The new type of model was called dynamic stochastic general equilibrium (DSGE), and it tried to account for the individual decisions of consumers and producers. Everyone, and I mean everyone in academia, abandoned SEMs in a very short period of time, and many switched over to DSGEs.
Why did DSGE models take over? Two reasons. The first was the stagflation of the 1970s. The Keynesian SEMs predicted that when the Federal Reserve lowered interest rates, it should have given the economy a boost; instead, all it did was create useless, harmful inflation. This made a big impression on economists. About a year ago I asked a group of economists whether the Fed should temporarily adopt a higher inflation target. Robert Lucas, who probably has more claim than anyone to being the father of modern macroeconomics, thundered: “We tried (stupid) inflation! It didn’t (dang) work!”
Now, it was possible to tweak the old Keynesian SEM models to explain why inflation didn’t boost the economy. But at the same time, the aforementioned Lucas and some other heavyweights such as Tom Sargent were revealing that there was a very deep reason those SEM’sshouldn’t work. It boils down to the famous saying that “correlation doesn’t equal causation.” Suppose economists noticed that businesses where people wear Star Trek T-shirts are more productive than others. Simple -- just have everyone wear a Star Trek T-shirt, and you’ll boost national productivity, right? Wrong.
In the same way, Lucas showed that trying to boost gross domestic product by raising inflation might be like the tail trying to wag the dog. To avoid that kind of mistake, he and his compatriots declared, macroeconomists needed to base their models on things that wouldn’t change when government policy changed -- things like technology, or consumer preferences. And so DSGE was born. (DSGE also gave macroeconomists a chance to use a lot of cool new math tricks, which probably increased its appeal.)
OK, history lesson over. So why is this important now?
Well, for one thing, the finance industry has ignored DSGE models. That could be a big mistake!
Suppose you’re a macro investor. If all you want to do is makeunconditional forecasts -- say, GDP next quarter – then you can go ahead and use an old-style SEM model, because you only care about correlation, not causation. But suppose you want to make a forecast of the effect of a government policy change -- for example, suppose you want to know how the Fed’s taper will affect growth. In that case, you need to understand causation -- you need to know whether quantitative easing is actually changing people’s behavior in a predictable way, and how.
This is what DSGE models are supposed to do. This is why academic macroeconomists use these models. So why doesn’t anyone in the finance industry use them? Maybe industry is just slow to catch on. But with so many billions upon billions of dollars on the line, and so many DSGE models to choose from, you would think someone at some big bank or macro hedge fund somewhere would be running a DSGE model. And yet after asking around pretty extensively, I can’t find anybody who is.
One unsettling possibility is that the academic macroeconomists of the '70s and '80s simply bit off more than they could chew. Modeling a big thing (like the economy) as the outcome of a bunch of little things (like the decisions of consumers and companies) is a difficult task. Maybe no DSGE is going to do the job. And maybe finance industry people simply realize this.
There are signs that some academic macroeconomists are starting to come to a similar conclusion. In another post, Cochrane talks about attending a recent macroeconomics conference, and how researchers are abandoning big, all-encompassing theories in favor of simpler, more targeted models designed to explain specific ideas rather than predict the whole economy. And at the Fed, academically trained economists have flat-out refused to abandon their old-style SEMs. That seems like a strong sign that the Fed hasn’t found any DSGE model that convincingly explains the business cycle.
So it seems to me that industry people and academics need to have more of a conversation than they’re having. If industry simply missed out on the big intellectual revolution of the '80s, academics need to help them get on board. On the other hand, if academics have set themselves an impossible task, they need to think hard about what to do instead.
Updates
This James Heckman quote from 2010 is sort of the converse of my post. He asks: If old-style "Keynesian" SEM's are so bankrupt, why is Wall Street still using them exclusively?
What struck me was that we knew Keynesian theory was still alive in the banks and on Wall Street. Economists in those areas relied on Keynesian models to make short-run forecasts. It seemed strange to me that they would continue to do this if it had been theoretically proven that these models didn’t work.The standard answer to this is that the "Keynesian" models are OK at unconditional forecasting, but not at policy-conditional forecasting. But since DSGE models should, in theory, be the best models for policy-conditional forecasting, Heckman's question leads naturally to mine...
Pretty simple answer: the finance industry only uses a model that has been proven to work.
ReplyDeleteHar har. You obviously don't work in finance.
DeleteI'd say the following: if a model has a lot of parameters that are hard to calibrate - there is a very high chance it will fail out of sample. So, practitioners will want to use simpler models that lend themselves to easier validation procedures.
ReplyDeleteSome DSGE models are very simple with very few parameters. The problem is more that the simple ones don't work in any situation, and the complicated ones only work for the particular situation the model's inventor was dealing with in the paper in which it was published.
DeleteIf the last 7 years have taught us anything it's that fiancial models work really well...
ReplyDeleteAn alternative explanation is that financial analysts and macroeconomists view modeling trade-offs differently. Financial analysts are primarily interested in making good forecasts, while macroeconomists are often more interested in deriving optimal policies. It may be that, beyond a certain point, better forecasts come at the expense of analytical tractability. The mid-century models had a pretty hard time saying anything about the desirability of one policy versus another, unless one made silly assumptions like MUST HAVE MOAR OUTPUT. The DSGE models have an easier time with this, but do worse in forecasting.
ReplyDeleteAn important question to ask is whether we really want the best and the brightest solving for optimal policies in models with poor forecasting performance. Indeed, why should we expect optimal policy in some possible world very different from our own to bear much resemblance to optimal policy in our world? Just because the model delivers an answer we can label "optimal" does not mean we've actually made progress in identifying better policies.
...But why does the finance industry care less about forecasting the effects of a policy change than the Fed does? It just doesn't make sense.
Delete...Not that the Fed relies exclusively or even primarily on DSGE models, either.
I think there's a good reason that finance only cares about unconditional forecasts:
DeleteWhen we make investment decisions, all we care about is what the state of the world will be tomorrow, NOT the optimal state. In academia/central banks we evaluate what would make for a good policy. What does an investment banker care whether there's a wealth effect in people's investment decisions or what the deep parameters of a DSGE look like? All she cares about is GDP up, GDP down, stock prices up, stock prices down.
- Lukas
It may not be that analysts care less than the Fed about the consequences of policy, inasmuch as policy decisions have implications for the forecast, but that they don't care about whether those consequences are good or not. In which case when macroeconomists defend DSGE by saying, "we're not in the business of weather forecasting, we're interested in the consequences of policy" what they're really saying is whether the consequences are good or not. Since financial analysts don't have to worry about that, they don't have to waste time with microfoundations, and have a much more flexible class of models to choose from.
DeleteI've been on the buy and sell side and never saw anyone using any macro model at all. Everyone I knew just looked at recent performance, looked at trends/events they thought would change performance, and guesstimated how big those resulting changes would be. If they were fastidious types they would do this in a spreadsheet and not just in their head. Only large banks and asset managers have somebody doing macro modeling. And most of it is just a somewhat more formalized and rule-based version of what I've described. A model, after all, is just a set of rules. And generally, the guys doing the macro modeling are part of a team which includes others trusting their gut. And generally, the experienced macro analyst will guesstimate better than any set of rules.
ReplyDeleteAll that said, look up Bloomberg's or Reuters' history of surveys of top macro analysts and how often they've predicted correctly the direction of GDP growth. You'd do just as well consulting a coin toss.
This is consistent with everything I've seen and heard!
DeleteYou'd be more convincing if you gave specific examples, like say how JP Morgan modeled that the housing market was turning. Some banks did better than others in their macro predictions and I doubt they were using academic DSGE models.
DeleteThe Fed's models did alright in predicting growth was slowing in 2007, but the FOMC ignored their staff probably going off "gut feelings." They were worried that inflation wouldn't moderate when in fact growth fell off a cliff. Even private markets were pricing in slower growth but the FOMC ignored them too since they had a gut feeling inflation wouldn't moderate. Plus there was anecdotal evidence that inflation wasn't moderating. Just read the FOMC minutes.
Basically I think's Tom's history is BS. "All that said, look up Bloomberg's or Reuters' history of surveys of top macro analysts and how often they've predicted correctly the direction of GDP growth. You'd do just as well consulting a coin toss."
Just look at all the stopped-clock predictions of imminent accelerating inflation these past 6 years. The top macro analysts weren't predicting that.
So their models served them well in that instance.
"I've been on the buy and sell side.." He drank the Kool Aid too.
Peter, what exactly is your complaint about my history? It sounds like you misread me. Maybe I was unclear and should reiterate my points as simply as possible.
Delete- The top macro analysts in finance are not basing their forecasts mainly on models. The bigger the institution, the more likely it is that somebody, usually a junior analyst, is doing some formal modeling. But mainly analysts guesstimate based on recent performance and their assessments of the impacts of important trends and events.
- My personal opinion is that an experienced macro analyst is better than any model. But objectively, top macro analysts as a group are no better than a coin toss. If you don't believe my claim, look it up. Bloomberg at least used to publish the success record of their survey group every quarter, in their article reporting the consensus GDP forecast, published just ahead of the first print. It was always 50 or 51. I remember well because it always gave me a chuckle the way they reported it, absolutely deadpan in the last graph.
As for maverick predictions of accelerating inflation, that is not what I mean by trusting your gut. That's ideology, and I think you'll find that even the funds that sell with that line don't much follow it in their investment strategy.
Likewise, the Fed governors in 2007-2008 were believing in their hopes, not trusting their gut. Finance professionals also are gullible to their own hopes, but being so is more likely to end their careers. Indeed that's what killed the small hedge fund I worked for.
I'm telling you to go look up how often the "consensus" GDP prediction, as calculated by Bloomberg or Reuters in their regular surveys of top analysts, has been correct about the direction of quarterly GDP versus previous quarter. Bloomberg at least used to regularly publish the figure, and as far as I recall it was always 50% or 51%.
Some particular analysts I'm sure had better performance, but in general, the top financial industry macro analysts as a group are no more likely to correctly predict an acceleration or deceleration of GDP growth than a coin toss.
Your characterization of FOMC '07 seems about right, except, the Fed probably had multiple models taking different sides, and the Fed weren't so much trusting their gut as believing in what they wished to be true. That kind of thing also happens a lot in finance and indeed was the death of the small hedge fund I worked for. That might seem like I'm a distinction where no objective distinction exists, but in my mind there's a big difference between trusting your gut and betting on your hopes.
I don't know how JP Morgan modeled the housing market, but modeling a specific situation like that does not fall under what most people mean when they say macro modeling.
Oh heck I garbled that post. Those three middle graphs were meant to be deleted.
DeleteBloomberg doesn't appear to have any such preview story ahead of the April 30 GDP release on its web archive. I'm afraid if you demand proof you'll need a friend with a Bloomberg terminal. I'll bet you anything the success record of the consensus of Bloomberg-surveyed top analysts in predicting the direction of GDP growth relative to previous quarter since the survey was started till now is no higher than 52%.
DeletePS You (Peter) seem to have a strange notion that using models somehow protects from errant ideology. Any ideology, including the simplistic monetarism that says any base money expansion will cause a proportional amount of inflation, can and probably has been incorporated into a model. Actually expectations of a moderate pick-up in inflation were widespread in 2009-2011 and indeed CPI did go over 3 for a spell in what proved to be a China-driven headfake. I think those in finance who correctly predicted low inflation were looking at the low to negative private net credit issuance, not DSGE or any other kind of models.
DeleteThis doesn't really resolve Noah's "why SEM not DSGE" question, but:
DeleteAfter ~20 years on the buy-side, I have to echo Tom. Macro forecasting on the Street is (in my experience) viewed mostly as entertainment (gives us something to write about each Q), not as a useful input. Most forecasts seem to be t+1 = t + small random, with some sort of hand-waved story tacked on. My impression (again, echoing Tom) is that forecast accuracy is a coin toss.
Personally, the only macro issues I spend any time thinking about are tail events. That seems to be where you can materially lose/win.
Also, to the "debate" on inflation prediction post 09 above, there's a claim that the "top macro analysts" were not prediction inflation. That feels circular to me. The "top analysts" looking back from 2014 are inevitably those that were right in retrospect.
I doubt that I am surprised that I object to your intellectual history.
ReplyDeleteI moved my comment over to my personal blog where no one will read it, because the tone is rude.
I agree with Robert's blog post. I'd soften the tone by pointing out that the Fed staffers and business sector forecasters who don't use DSGE and who are rejecting academia's ivory tower economics don't really defend themselves or engage with the false history. They leave it to bloggers like Robert.
DeleteAgain it's an instance where ideas are warped by money and ideology. Private business forecasters have better things to do than take on academia and correct the historical record. The Fed doesn't want to provoke half of Congress, the institution it reports to on a regular basis.
I don't know who is more right about the history (I strongly suspect Dr. Waldman is) so I would very much like to see a reply from Dr. Smith. It seems to me the history should be available to be cited, pro or con.
DeleteRobert's comment is silly. He is saying that we could have saved the SEMs by abandoning them in favor of new SEMs (notice the contradiction) that instead of a trade-off between inflation and unemployment would favor a trade-off between the change in inflation and unemployment. Well, sure, we can keep doing this every time there is a structural change but, especially if this structural change is the result of a change in the policy regime, it is a self-defeating exercise. This was the whole point of the Lucas critique!
DeleteBy the way, Robert conveniently forgets that in the 1970s self-proclaimed Keynesian economists like Tobin were arguing that inflation was a cost-push phenomenon while Friedman and the monetarists were arguing that it was demand-pull and driven by monetary policy. We all know who won that one!
What would finance people use DSGE models for, given that practitioners do not vouch for their predictive accuracy?
ReplyDelete>The standard answer to this is that the "Keynesian" models are OK at unconditional forecasting, but not at policy-conditional forecasting. But since DSGE models should, in theory, be the best models for policy-conditional forecasting, Heckman's question leads naturally to mine...
ReplyDeleteI'm not sure this is quite right. I can build a really detailed model that perfectly predicts a person's income from details of their lives, but that doesn't matter if all I care about is the causal impact of an extra year of education. Models that fit/predict well aren't necessarily identified, and a lot of models that are well identified still have very low R-squared. Once you scale this problem up to DSGE I'd imagine the identification issues are even stronger.
Also, I think "policy-conditional forecasting" is different, as well. As a commenter alludes to above, a well-identified qualitative impact (i.e., getting the sign right) of policy on macro variables is probably more valuable/relevant to macroeconomists.
DeleteNoah writes, (quoting?) Lucas: “We tried (stupid) inflation! It didn’t (dang) work!”
ReplyDeleteNick Rowe points out that the lower the inflation target, the closer you get to communism:
Too dedicated a pursuit of low inflation and the optimum quantity of money leads to communism, with government ownership of everything.
Scott Sumner agrees:
"I like to sometimes tease conservatives who want really low inflation by pointing out that that they are advocating socialism."
What I find remarkable about that Lucas quote, BTW, is that there's no sense of changing circumstance there. I wonder if he were standing on the moon and somebody suggested the measure the acceleration of gravity there, if he would respond: "Dang it to heck, we already measured (dog gone) gravity back on Earth! Why should we friggin' do it again!"
DeleteInsanity: doing the same thing over and over again and expecting different results.
DeleteAlbert Einstein
Yup, Anon - every time we repeat a failed experiment over and over, I find myself exclaiming "This is f'in' nuts!". Of course, usually we figure out why it didn't work all those times and then we get the different result we expected.
DeleteIt is my understanding that the majority of people on Fed (and other major central bank) research staffs are fans of DSGE. However, at least the Fed, and probably most other central banks, keep Keynesian-style structural models and also VAR-type models around for actual forecasting, keeping a few people around to tend to those. What has not picked up much, except in a few places, is any sort of serious agent-based modeling.
ReplyDeleteIt's not very complicated. DSGE models are nothing more than models of the pace of reversion to an (unexplained) trend. And they're not very good at predicting that. So people don't use them.
ReplyDelete"Everyone, and I mean everyone in academia, abandoned SEMs in a very short period of time, and many switched over to DSGEs." Noah Smith, 25 July 2014
ReplyDeleteThis comment is not historically correct, as evidenced by the extensive body of work referenced and abstracted at the following site:
Agent-Based Macroeconomics
http://www.econ.iastate.edu/tesfatsi/amulmark.htm
The above website starts by setting out the following five "Basic Issues," which it would be wonderful to see considered and addressed in a serious thoughtful manner by macroeconomists in general and by the macroeconomic bloggers at Noah's blogsite in particular:
(1) "Is it too much to ask of anyone building a model with `microfoundations' that the microfoundations be `true'? I mean, climatologists have models with microfoundations, but their assumptions about heat transfer actually work…" (Chris Dillow)
(2) Should macroeconomists adhere a priori to the "discipline of equilibrium," regardless of empirical reality? Or should the existence of equilibria be treated as a testable hypothesis, considered in conjunction with basins of attraction?
(3) If you had to specify goals and decision rules for economic agents trying to survive and prosper in a macroeconomy, where these goals and rules can only depend on information and computational capabilities that the agents could reasonably be assumed to possess, and you are not permitted to impose coordination on the agents' activities a priori, e.g., through global market clearing or rational expectations restrictions, how would you go about it?
(4) If the above exercise stumps someone, to what extent can that person be said to possess a scientific understanding of real-world macroeconomic systems?
(5) To gain a constructive understanding of real-world economic processes (e.g., expectation formation, intertemporal planning, production, price discovery, buyer-seller matching, payment settlements, firm entry/exit, longer-term contractual relationships), should not macroeconomists engage more fully with the real-world entities who carry out these processes?
LT
Noah, maybe this will help you answer your question:
ReplyDeleteRead: MACROECONOMICS AFTER THE CRISIS:
TIME TO DEAL WITH THE PRETENSE-OF-KNOWLEDGE SYNDROME
by MIT Prof. Ricardo J. Caballero
Here's the abstract:
In this paper I argue that the current core of macroeconomics—by which I mainly mean the so-called dynamic stochastic general equilibrium approach—has become so mesmerized with its own internal logic that it has begun to confuse the precision it has achieved about its own world with the precision that it has about the real one. This is dangerous for both methodological and policy reasons. On the methodology front, macroeconomic research has been in “fine-tuning” mode within the local-maximum of the dynamic stochastic general equilibrium world, when we should be in “broad-exploration” mode. We are too far from absolute truth to be so specialized and to make the kind of confident quantitative claims that often emerge from the core. On the policy front, this confused precision creates the illusion that a minor adjustment in the standard policy framework will prevent future crises, and by doing so it leaves us overly exposed to the new and unexpected.