Polling: A Look Inside the Machinery of Public Opinion Surveys
Tulumello, Jennifer Skalka, The Christian Science Monitor
Polling: Behind the scenes at Gallup, interviewers and editors try to find out how Americans will vote on election day. With the media's dependence on public opinion statistics, news consumers must educate themselves about which surveys provide valuable data and why.
It's dinner time in America.
Somewhere parents and children sit down together to eat, to talk through their respective days, to enjoy precious shared time. The phone rings.
"Hi, I'm Michael Jablonski," says a man whose name is no doubt unfamiliar to the woman who answers, "and I'm calling about a Gallup poll.... We'd like to include your opinions. Can you help us out?"
His inquiry - from a Gallup interviewer sitting in a phone center in a city office park here in Omaha, Neb. - is one of many thousands that citizens will field from interested survey companies as the 2012 White House contest ramps up.
Their answers will mirror the mood of the country, but the mood of the country can very well be affected by the poll itself.
Public opinion surveys have become a ubiquitous element of American political culture. The numbers - some reliable, others less so - are pawed over and interpreted for headlines, insight, and horse-race drama by newspaper reporters, cable news talking heads, and bloggers of all party persuasions.
With each successive election of the modern age, there are more organizations looking to mine what the adult voter believes about the incumbent president and his rival, the state of the nation, and a host of issues from the economy to gay marriage.
With the media's dependence on poll numbers - and the sheer frequency with which those numbers are collected - news consumers must educate themselves about which surveys provide valuable data and why.
"Polling is important because it gives every voter and every nonvoter an equal chance of having their voice represented," says Scott Keeter, the director of survey research at the Pew Research Center. "When properly done, without bias and malice, polls can give you a view of what the public is experiencing or wanting, which you don't get from interest groups or the candidates or even elections, which are very blunt instruments."
Gallup - by the estimation of most in the industry - consistently generates quality data. Editors with the 77-year-old privately held survey company agreed to let the Monitor take an inside look at the process of putting together its June political poll, which assessed the role of religion in the presidential race as well as a range of other matters, including public views on the economy and the two men who want to lead this nation.
The 10-day endeavor of creating and conducting the poll involved Gallup staff in four states and Washington, D.C. Their work shows how important the human element is in shaping and thoughtfully interpreting a survey. Their efforts also illuminate the many junctures at which polls can be manipulated by those more interested in spinning numbers.
For example, question-crafting and order of questions when asked are vital to a valid survey. So are well-trained callers, like Mr. Jablonski, and the quality controls that guard their practices. Proper balance of land lines and cellphones matters in collecting a representative sample. How a data set is weighted - or made, via statistical calculations, to resemble the adult population of the nation - on the back end is critical, too.
Good polling is, in its own way, as intricately detailed as "successful heart surgery," says Frank Newport, the editor in chief of Gallup and the immediate past president of the American Association for Public Opinion Research: "Failures, wrong decisions, or low quality in any of these phases of the process can negatively affect the objective of using carefully selected samples of respondents to accurately represent the attitudes and self-reported behavior of an entire population of citizens. …