Using Math to Measure Accuracy and Fairness
Tessier, Marie, Investigative Reporters and Editors, Inc. The IRE Journal
As painful as the thought may be to many journalists, fostering a "numbers state of mind" can add depth to daily news stories, shed light on others, and guide longer-term investigative projects in their early stages.
Consider the concept of "average." Journalists mislead their audiences with the term every day. Sometimes that's because sources are deliberately misleading, and sometimes it's because we relish our mathematical ineptitude so much that we encourage stories that are inaccurate and unfair.
Major League Baseball owners, for example, have a reason for talking continually about the average salaries of their players. In baseball, as in many other employment stories, the "average salary" is not a fair or accurate general description of player salaries because highly paid stars like Mark McGwire skew the numbers. Using median salary would be more accurate, yet that figure is rarely used.
Here are three steps to introduce some basic investigative skepticism and a numbers state of mind to improve a lot of daily news stories that use averages.
. When a source makes a claim involving an "average," find out ho that number wa determined and why it was used to support the claim.
Remember that in common usage and in news copy, average refers to the arithmetic mean. Technically speaking, though, "average" can accurately describe three different numbers - the mean, median and mode. Compute them all, and consider what the different numbers tell you.
. Carefully choose which numbers to use in the news story, and use them sparingly.
Now that journalists in all parts of newsrooms are becoming better acquainted with computer spreadsheet software, doing the arithmetic is a breeze. You type in the raw data, and then ask the software to calculate. No problem.
The problem comes when journalists make the calculations without understanding their meaning. Editors who foster a numbers state of mind by asking for basic calculations on many stories can diminish the mindless use of numbers by cutting them off at the pass.
Sports reporters would never use a batting average to describe the overall performance potential of a pitcher. That wouldn't be accurate or fair.
It's the same with mean, median and mode. One or the other is almost always a more fair or more accurate choice for the news. Each concept gives a different way to make a generalization about a group using a single figure. Here are some definitions.
The Associated Press Stylebook defines average" as the "arithmetic mean," or the common sense of the word. So if you want to calculate the average score of journalists taking a math test, you add the scores, then divide by the number of journalists. (Add a series of figures, then divide by the number of figures.) Unless your news organization has a different definition in its style book, you should only use the term "average" to identify the "mean."
The "median" is another way to arrive at a generalization about a whole group. It works by ranking items, then choosing the middle score. So if you rank 15 journalists by their scores on a math test, the 8th highest score is the median score. (If you have an even number of scores, the median is the middle point between the two scores tied for the middle.)
The "mode" generalizes about a group by figuring the most common score or amount. In our math test example, the mode might be 80 if five reporters got that score, and other reporters did not end up in a bigger cluster. The mode can shed light on many campaign finance stories because the most common contributions are usually small, and can illustrate the depth of a candidate's support. …