Who is responsible when an article gets misread?

How much of the responsibility for understanding lies with the writer of an article, and how much with the reader? This is not an easy question to answer. Obviously both sides bear some responsibility. There are articles so baroque and circuitous that to get the point would require an unreasonable amount of time and effort to parse, even for the smartest reader. And there are readers who skim articles so lazily that even the simplest and most clearly written points are lost. Most cases fall somewhere in between. And the fact that writers don't usually get to write their headlines complicates the issue.

See what you think about this one. The other day, Susan Dynarski wrote an op-ed in the New York Times criticizing school vouchers (a subject I've written about myself). Dynarski opens with the observation that economists are generally less supportive of vouchers than they are of most free-market policies:

You might think that most economists agree with this overall approach, because economists generally like free markets. For example, over 90 percent of the members of the University of Chicago’s panel of leading economists thought that ride-hailing services like Uber and Lyft made consumers better off by providing competition for the highly regulated taxi industry. 

But economists are far less optimistic about what an unfettered market can achieve in education. Only a third of economists on the Chicago panel agreed that students would be better off if they all had access to vouchers to use at any private (or public) school of their choice.

Here's the actual poll: 

As you can see, the modal economist opinion is uncertain about whether vouchers would improve educational quality, while the median is between "uncertain" and "agree". This clearly supports Dynarski's statement that economists are "far less optimistic about vouchers than about Uber and Lyft. 

The headline of the article (which Dynarski of course did not write) might overstate the case a little bit: "Free Market for Education? Economists Generally Don’t Buy It". Whether the IGM survey shows that economists "generally don't buy" vouchers depends on what you think "don't buy" and "generally" mean. It's a little click-bait-y, like most headlines, but in my opinion not too bad. 

Scott Alexander, however, was pretty up in arms about this article. He writes:

By leaving it at “only a third of economists support vouchers”, the article implies that there is an economic consensus against the policy. Heck, it more than implies it – its title is “Free Market For Education: Economists Generally Don’t Buy It”. But its own source suggests that, of economists who have an opinion, a large majority are pro-voucher... 

I think this is really poor journalistic practice and implies the opinion of the nation’s economists to be the opposite of what it really is. I hope the Times prints a correction.

A correction!! Of course no correction will be printed, because no incorrect statements were made. Dynarski said that economists are "far less optimistic" about vouchers than about Uber/Lyft, and this is true. She also reported close to the correct percentage of economists who said they supported the policy in the IGM poll ("a third" for 36%). 

Scott is upset because Dynarski left out other information he considered pertinent - i.e., the breakdown between economists who were "uncertain" and those who "disagree". Scott thinks that information is pertinent because he thinks the article is trying to argue that most economists think vouchers are bad. 

If Dynarski were in fact trying to make that case, then yes, it would have been misleading to omit the breakdown between "uncertain" and "disagree". But she wasn't. In fact, her article was arguing that economists tend to have reservations about vouchers. And she supports her case well with data.

This is a special kind of straw man fallacy. Straw manning is where you present a caricature of your opponent's argument. But there's a particularly insidious kind of straw man where you characerize someone's arguments correctly, but get their thesis wrong. You misread someone's argument, and then criticize them for failing to support your misreading. Other examples of this fallacy might be:

1. You write an article citing Autor et al. to show that the costs of trade can be very high. Someone else says "This doesn't prove autarky is better than free trade!" But of course, you weren't trying to prove that.

2. You write an article arguing that solar is cost-competitive with fossil fuels by pointing out that solar power is expanding rapidly. Someone else says "Solar is still a TINY fraction of global generating capacity!" But of course, you weren't trying to refute that.

3. You write an article saying we shouldn't listen to libertarian calls to dismantle our institutions. Someone else says "Libertarians aren't powerful enough to dismantle our institutions!" But of course, you weren't trying to say they are.

I think Scott is doing this with respect to Dynarski's article. To be fair, his misreading was somewhat assisted by the headline the NYT put on the piece. But once he was reminded of the fact that the headline wasn't Dynarski's, and once he re-read the article itself and realized what its actual thesis was, I think he should have muted his criticism. 

Instead, he doubled down. He argued that most reasonable people, reading the article, would think it was arguing that economists are mostly against vouchers. But his justification for this continues to rely very heavily on the wording of the headline:

First, I feel like you could write exactly the opposite headline. “Public School: Economists Generally Don’t Buy It”... 

Second, the article uses economists “not buying it” as a segue into a description of why economic theory says school choice could be a bad idea... 

In the face of all of this, the New York Times reports the field’s opinion as “Free Market In Education: Economists Generally Don’t Buy It”.

On Twitter, he said: "the actual article is more misleading than the headline." But he appears to say this because he takes the headline - or, more accurately, his reading of it - as defining the thesis that Dynarski is then obligated to defend (when in fact she wrote the piece long before a headline was assigned to it). When he finds that Dynarski doesn't support his reading of a headline she didn't write, it is her article, not the headline, that he calls "misleading".

Of course, the fault here is partly that of the NYT, who used a headline that focused only on one part of Dynarski's article and overstated that part. It's a little harsh for me to say "Come on, man, you should know an article isn't about what its headline says it's about!" Misleading headlines are a problem, it's absolutely true. But after learning that Dynarski didn't write the headline, I think Scott should have been able to then read the article on its own, and go back evaluate the arguments Dynarski actually makes. It's the refusal to do this that seems to me to constitute a straw-man fallacy.

Anyway, one last point: I think Dynarski is actually wrong that economists are more wary of vouchers than other free-market policies. Yes, economists in general are probably wary of voucher schemes. But they're also a lot more favorable to government intervention in a variety of cases than Dynarski claims. Klein and Stern (2006) have some very broad survey data (much broader than IGM). They find that 67.1% of economists support "government production of schooling" at the k-12 level, with 14.4% uncertain and 17.4% opposed. But they also record strong support for a variety of other interventionist policies, such as income redistribution, various types of regulation, and stabilization policy. On many of these issues, economists are more interventionist than the general public! So I think if Dynarski makes a mistake, it's to characterize economists as being generally pro-free-market. Their ambivalence about vouchers doesn't look very exceptional.