The ability to create and interpret visual representations has been an important part of the human experience since we began drawing on cave walls at Chauvet.
Today, that ability—what I call visualcy—has even greater importance. We use visuals to discover how the world works, communicate our discoveries, plan efforts to improve the world, and document the success of our efforts.
In short, visualcy affects every aspect of program design and evaluation.
The evolution of our common visual language, sadly, has been shaped by the default settings of popular software, the norms of the conference room, and the desire to attract attention. It is not a language constructed to advance our greater purposes. In fact, much of our common language works against our greater purposes.
An example of a counterproductive element of our visual language is the pie chart.
Consider this curious example from the New York Times Magazine (1/15/2012).
This pie chart has a humble purpose—summarize reader responses to an article on obesity in the US. It failed that purpose stunningly. Here are some reasons why.
(1) Three-dimensionality reduces accuracy: Not only are 3-D graphs harder to read accurately, but popular software can construct them inaccurately. The problem—for eye and machine—arises from the translation of values in 1-D or 2-D space into values in 3-D space. This is a substantial problem with pie charts (imagine computing the area of a pie slice while taking its 3-D perspective into account) as well as other types of graph. Read Stephanie Evergreen’s blog post on the perils the 3-D to see a good example.
(2) Pie charts impede comparisons: People have trouble comparing pie slices by eye. Think you can? Here is a simple pie chart I constructed from the data in the NYT Magazine graph. Which slice is larger—orange or the blue?
This is much clearer.
Note that the the the Y axis ranges from 0% to 100%. That is what makes the bar chart a substitute for the pie chart. Sometimes the Y axis is truncated innocently to save column inches or intentionally to create a false impression, like this:
Differences are exaggerated and large values seem to be closer to 100% than they really are. Don’t do this.
(3) The visual theme is distracting: I suspect the NYT Magazine graph is intended to look like some sort of food. Pieces of a pie? Cake? Cheese? It doesn’t work. This does.
Unless you are evaluating the Pillsbury Bake-Off, however, it is probably not an appropriate theme.
(4) Visual differentiators add noise: Graphs must often differentiate elements. A classic example is differentiating treatment and control group averages using bars of different colors. In the NYT Magazine pie chart, the poor choice of busy patterns makes it very difficult to differentiate one piece of the pie from another. The visual chaos is reminiscent of the results of a “poll” of Iraqi voters presented by the Daily Show in which a very large number of parties purportedly held almost equal levels of support.
(5) Data labels add more noise: Data labels can increase clarity. In this case, however, the swarm of curved arrows connecting labels to pieces of the pie adds to the visual chaos. Even this tangle of labels is better because readers instantly understand that Iraq received a disproportionate amount of the aid provided to many countries.
Do you think I made up these reasons? Then read this report by RAND that investigated graph comprehension using experimental methods. Here is a snippet from the abstract:
We investigated whether the type of data display (bar chart, pie chart, or table) or adding a gratuitous third dimension (shading to give the illusion of depth) affects the accuracy of answers of questions about the data. We conducted a randomized experiment with 897 members of the American Life Panel, a nationally representative US web survey panel. We found that displaying data in a table lead [sic] to more accurate answers than the choice of bar charts or pie charts. Adding a gratuitous third dimension had no effect on the accuracy of the answers for the bar chart and a small but significant negative effect for the pie chart.
There you have it—empirical evidence that it is time to retire the pie chart.
Alas, I doubt that the NYT Magazine, infographic designers, data viz junkies, or anyone with a reporting deadline will do that. As every evaluator knows, it is far easier to present empirical evidence than respond to it.
EvalBlog 