Over my morning coffee, I found myself staring at this bulldog graph in the New York Times Magazine (12/11/11). Something was wrong. At first I couldn’t put my finger on it. Then it hit me—the relative size of the two bulldogs couldn’t possibly be correct.
I did a little forensic data analysis (that is, I used a ruler to measure the bodies of the bulldogs and for each computed the area of an ellipse—it turns out that geometry is useful). As I suspected, the area of the bulldog on the right is too small. If the bulldog on the left represents 58% of responses, then the bulldog on the right represents only about 30%. Oops.
Here is how large the bulldog should be. Quite a difference.
The heights of the bulldogs, as they originally appeared, were proportionally correct. That made me wonder. If you change the size of an image using most software, it changes the length and width proportionally—not the area. Is that how the error was made?
The Times wouldn’t make a mistake like that, I reasoned. Maybe the image was supposed to be a stylized bar graph (but in that case the width of the images should have remained constant). In any event, the graph addressed a trivial topic. I went back to my coffee confident in my belief that the New York Times would never make such a blunder on a important issue.
Then I turned the page and found this.
The same mistake. This time the graph was related to an article on US banking policy, hardly a trivial topic. The author wanted to impress upon the reader that a few banks control a substantial share of the market. A pity the image shows the market shares to be far smaller than they really are.
The image below illustrates the nature of the error—confusing proportional change in diameter for a proportional change in the area of a circle.
Below is a comparison of the original inaccurate graph and an accurate revision that I quickly constructed. Note that the area of the original red circle represents a market share of just under 20%—less than half its nominal value.
And here is an alternative graphic using squares instead of circles. A side-by-side presentation of Venn diagrams may allow readers to compare the relative size of market shares more easily than overlaid shapes.
I did a bit of digging and found another person noticed the error and pointed it out to the Times online (you really need to dig to find the comment). To date, the inaccurate graph is still on the Times website and I have not found a printed correction.
Apparently, checking facts does not include checking math.
EvalBlog 
I’m so glad you are talking about this, John. I’ve found that Sky Magazine (what can I say, I fly a lot) is the worst offender at misleading graphic images.
This is great! Thank you for doing the “research” about facts.
When I taught stats, one of our assignments each week was to bring in an example of a bad graph from a newspaper. Usually the students didn’t have to go further than a single issue of USA Today. We had a hall of shame. Thanks for raising the bar.
Thanks John, good job.
Pingback: Similar Situations
I personally like infographics. But I’ve always wondered how accurate size comparisons of images (like the bulldog example) are. Great find!
Just staggering, isn’t it? It’s bad enough all those graphs that have the X-axis cross at nowhere near zero and utterly mislead … but this kind of thing is soooo blatant. Intentional or clueless? Take your pick! Cheers, John, for the example.
I would feel better if all the inaccuracies in graphs were unintended. That would give me hope that training and standards can improve the way we communicate visually. However, some are intentional misrepresentations.
In the NYT Magazine example, the graph of the top banks’ market shares worked against the point the writer was making. I can’t imagine that was intentional. For the WSJ example (a later post) the graph exaggerated the point the writer was making. That makes it more difficult to believe it was unintended, especially given the Journal’s quantitative savvy. But who can know the intentions of others?
For an example of probably intended misinformation, see a Fox News graph about unemployment in which a relatively steep drop off is presented as no change: http://mediamatters.org/blog/201112120005
Wow. That is hard to write off as an innocent error.
Pingback: Bloggers Series: John Gargani on EvalBlog · AEA365
Pingback: How to Evaluate on a Budget: DIY or Outsource? | Nonprofit Capacity Building
I understand your position on accurate math, however from experience I can attest that showing a manager, who doesn’t understand the math, the first response is “that’s not right, fix it.” Even after explaining, the manager still ‘feels’ the graphic is somehow ‘wrong’.
My usual response is to avoid infographics if possible and stick to bar or line graphs.
Todd,
I agree that we need to take our audience into account. If there is some tradition or expectation regarding a graphical presentation (for example, budgets are often represented as pie charts), using a less familiar but more accurate graphic may not work. What we want to avoid is invoking a misleading interpretation of a graphic. Images should quickly and intuitively help readers understand what’s so. If graphics fail to do this–because they are inaccurate or readers believe that accurate representations are wrong–we should not use them.