Reporting with the Tools of Social Science: 'We Had Put the Social Scientists on Notice That Journalists Increasingly Would Be Competitors in Their Field.'
Doig, Stephen K., Nieman Reports
Cynics would say "precision journalism is an oxymoron, like unbiased opinion or civil war. But precision is an ideal to be sought in journalism, though not often achieved.
As defined by Knight Ridder reporter Philip Meyer in his groundbreaking 1973 book of the same name, precision journalism is the use of the tools of social science to replace, or at least supplement, reporters' time-honored methods of citing anecdotal evidence and doing educated guesswork. Today, thanks to Meyer's call, I'm one of hundreds of investigative reporters who have crafted serious stories using such tools as survey research, statistical analysis, experimentation and hypothesis testing. It's social science done on deadline.
As it happens, I got a glimmering of these methods even before I discovered journalism as a career. In my freshman year at Dartmouth in 1966, I slogged my way to a so-so grade in calculus, the last math course I ever took. I remember little calculus today, but I did learn something my professor, John Kemeny, had coauthored two years earlier: the computer language BASIC. I thought it very cool that I could peck a few lines of English-like instructions into a teletype machine and seconds later a mainframe computer somewhere on campus would calculate precinct-level vote percentages for my American Government homework.
However, it was 15 years later before I got a chance to start applying such methods to my journalism. The problem was that much of what Meyer recommended could best be done with a computer, which during the 1970's meant big-iron mainframes that only universities or corporations could afford. It was nearly a decade before personal computers were developed and usable by nontechies like myself.
By then, in 1981, I was a reporter in The Miami Herald's state capital bureau, and I had bought an Atari 800 computer to play with at home. I quickly realized that my expensive toy could help me do my job better. I relearned BASIC, then persuaded my editors to buy one of the new-fangled IBM PCs for me to use at work. At one point, I spent a week writing and debugging a program that would take a legislative roll call vote and produce cross-tabs not only by party but also by such other revealing political demographics as race, gender, geography, leadership position, and source of campaign contributions. It would even write the roll call agate we appended to legislative stories. (Today, of course, such an application could be built in minutes with off-the-shelf database software.)
A couple of years later, I got to meet Meyer for the first time. He did a day-long training seminar at the Herald attended by a few of us, including Richard Morin, who later would go on to be the polling director for The Washington Post for nearly 20 years. Rich and I, in particular, came away from that seminar inspired to be precision journalists.
So I spent the next decade at the Herald teaching myself, in bits and pieces, the social science tools I hadn't had enough sense to study when I was in college, from statistics to cartography. An example of my academic cluelessness came in 1991, when I was working with Knight Ridder colleagues Dan Gillmor and Ted Mellnik on a project about racial segregation in the United States using just-released census data. I spent days trying to noodle together some way to measure the degree of segregation in a community, but nothing useful emerged. I finally mentioned my frustration to Mellnik, who then mentioned it to a friend who was a sociologist. "Oh, you want the dissimilarity index," the friend promptly replied, giving Mellnik a citation for an article describing it in a scholarly journal from the 1950's.
Armed with that already-invented tool, a month after the decennial census data was released we produced our analysis of how segregation had--or hadn't--changed in every state, county and city across the country since 1980. For expert comment on what we had found, Gillmor called William Frey of the University of Michigan, one of the nation's leading demographers. He was stunned. "My god, I had no idea newspapers could do that kind of work,' Frey told Gillmor, adding that he hadn't even started thinking about the grant applications he would write in hopes of doing a similar analysis in the next few years. We had put the social scientists on notice that journalists increasingly would be competitors in their field. And I'm proud to say that we also beat Meyer, by then a Knight Chair at the University of North Carolina in Chapel Hill, to the story; unknown to us until later, he was a consultant to USA Today reporters who were working on a similar analysis.
By then, there was a growing band of investigative reporters doing evermore sophisticated studies using precision journalism techniques. One of the best-known examples at the time was the "Color of Money" project that used hard data to document so-called "redlining" by banks that were denying mortgage loans to black residents of Atlanta, which won a 1989 Pulitzer for young Atlanta Journal and Constitution reporter Bill Dedman.
In a case of chance favoring the prepared mind, my opportunity to use preeision journalism on a huge story arrived on August 24, 1992, when Hurricane Andrew tore across the south end of Miami with winds exceeding 150 miles per hour. Andrew destroyed or heavily damaged more than 80,000 homes--including my own. In the weeks that followed, as Herald reporters were covering the daily events of recovery across the battered region, our investigative team met to see how we might determine whether the extent of this disaster was simply an act of God or the inevitable result of our collective stupidity. I realized that I could merge our growing database of damage reports with the property tax roll, which would allow us to study the storm's damage patterns. When I ran the numbers, I found the first smoking gun of my career. There was no pattern to be seen when I compared storm damage to such variables as home value, type of construction, location or distance from the shore. But there was a strong, counterintuitive pattern connected to the year of construction: The newer the home, the more likely to be destroyed. We went on to uncover the reasons for this in other datasets we gathered, including millions of records of often-hasty building inspections and millions of dollars in campaign contributions to politicians who had approved weakened building codes at the urging of the construction industry. Our report, called "What Went Wrong," was published three months after the storm.
After that, Herald reporters and I investigated other big social problems by using hard data, including an examination of South Florida's overwhelmed criminal justice system and a probe of immigration's effect on our region and the country.
Teaching the Tools
I left the Herald in 1996 to become a professor of journalism. Each semester, I teach our best students some of the precision journalism techniques that I still wish I had learned back in college. I also continue to work with investigative reporters who are following the path first laid out by Meyer. For instance, in advance of the 2000 Census, USA Today database editor Paul Overberg and I taught hundreds of reporters the tools needed to interpret the data--including the dissimilarity index--in a series of workshops around the country organized by Investigative Reporters and Editors (IRE). And with the help of numbers-savvy IRE stalwarts like Jennifer LaFleur and Holly Hacker, (1) I host an annual three-day statistics boot camp for reporters who are ready to move beyond spreadsheets to learn tools like correlation, regression, sampling, hypothesis tests, and probability.
Finally, the use of such tools has become so common that three years ago I joined with IRE to create the Philip Meyer Awards (2) for precision journalism honoring the best investigative reporting done each year using social science techniques. Winners and other strong entries have used clever statistical methods to uncover cheating scandals in public schools, backdated stock options, flaws in ballot designs, origins of the methamphetamine epidemic, and systemic problems with veterans' medical care, among many noteworthy stories. What Meyer dreamed of 35 years ago--a cadre of journalists who use the scientific method--precisely has come to pass.
RELATED ARTICLE: Building a toolbox for precision journalism.
Want to add social science methods to your toolbox of investigative reporting skills? What follows are some ways to consider doing this:
Read "Precision Journalism: A Reporter's Introduction to Social Science Methods" (4th edition) by Philip Meyer.
Fuzzy with math? Read "Numbers in the Newsroom: Using Math and Statistics in News," by Sarah Cohen, the Pulitzer-winning Washington Post database editor.
Get the boss to buy you a copy of SPSS 16.o: This is a widely used statistical analysis software package. It's pricey, at $1,700, but it will do most of the kinds of analyses precision journalists are performing these days. Better yet, the SPSS documentation, manuals and practice data serve as a good tutorial in statistical techniques.
Join Investigative Reporters and Editors (IRE): It offers training in computer-assisted reporting in general and precision journalism in particular. (1)
Subscribe to NICAR-L: This e-mail listserv is frequented by most of the reporters who are using precision journalism tools. (2)
Consider signing up for the annual advanced newsroom statistics boot camp at Arizona State University. (3)
Sign up for the annual "Mapping Data for News Stories" boot camp at IRE's headquarters at the University of Missouri.
Take a statistics course and a geographical information systems course at your local university. Most professors will be intrigued about helping you become a precision journalist.--S.K.D.
(1) For more information on the IRE, go to www.ire.org/. Books mentioned above are available at www.ire.org/store.
(2) To join this listserv go to www.ire.org/membership/subscribe/nicar-l.html. Fast help is available there while climbing the learning curve.
(3) Details can be found at www.ire.org/training/arizonastats08.html.
(1) Hacker and Oklahoma State University journalism professor Stan Ketterer are working on a book about using advanced statistics in reporting.
(2) This year, the Meyer Award was given to The Dallas Morning News for "Faking the Grade." This series, by reporters Joshua Benton, a 2008 Nieman Fellow, and Holly Hacker, uncovered strong evidence of cheating on standardized tests by more than 50,000 students in Texas public and charter schools. [See Nieman Notes, page 107.]
Stephen K. Doig is the Knight Chair in Journalism at Arizona State University's Walter Cronkite School of Journalism and Mass Communication. Precision journalism projects on which he worked during his 19 years at The Miami Herald won the Pulitzer Prize for Public Service, the IRE Award, and the Goldsmith Prize for Investigative Reporting.…
Questia, a part of Gale, Cengage Learning. www.questia.com
Publication information: Article title: Reporting with the Tools of Social Science: 'We Had Put the Social Scientists on Notice That Journalists Increasingly Would Be Competitors in Their Field.'. Contributors: Doig, Stephen K. - Author. Magazine title: Nieman Reports. Volume: 62. Issue: 1 Publication date: Spring 2008. Page number: 47+. © 1999 Harvard University, Nieman Foundation. COPYRIGHT 2008 Gale Group.
This material is protected by copyright and, with the exception of fair use, may not be further copied, distributed or transmitted in any form or by any means.