Magazine article Computers in Libraries

Data's Role in the Election Surprises of 2016

Magazine article Computers in Libraries

Data's Role in the Election Surprises of 2016

Article excerpt

As I am writing this column, it seems as if all the commentators and critics in the world have had their final say on the U.K.'s Brexit vote, the U.S. presidential election, and swings in voter opinion across the European Union (EU). The turmoil that has engulfed the policy world was brought home for me this past fall, when Matt Sedensky wrote an article on Big Data's "big miss" not long after Donald Trump's surprise win turned conventional wisdom upside down. The article summed up a familiar problem we've long known about--and that keeps changing with the times. This problem has gone by many names over the years, but one in particular has staying power and speaks to contemporary realities: garbage in, garbage out.

Of course, data management realities have definitely changed with the times, but the central dilemma remains the same: The data we access and interpret is only as good as its content. What is more, "garbage" now goes by another name: "unstructured data." That sounds so much more respectable, but, ultimately, the formula continues to operate quite effectively. You might say, "Lots of data in, lots of not-so-useful data out."

I think we could also go one step further and contribute an addendum: The data is only useful when we study the human relationships around it in real time. How else could the entire liberal wing of the global "policy infrastructure" keep missing the mark?

Apparently, it can happen. In light of this big-time "miss," I'm offering up three examples of how social trends are at work in the data-driven world we live in. Perhaps they will challenge us to take another look at data science--and its limits.

Can Data Track Voter Fatigue?

There is no question that American policy wonks, including those in the survey and polling universe, have taken a beating in the wake of the November election. It is especially noteworthy that one of the "star" surveyors, Nate Silver of, did not anticipate the flip in voter sentiment even as he tracked all polling services with sophisticated and time-tested algorithms.

In Sedensky's previously mentioned article ("'Big Data' Questioned in the Wake of Trump's Surprising Victory"), he quoted a number of experts and their theories on what happened. Some border on comedy. My favorite is the idea that those who planned to vote for Donald Trump were "too shy" to say so ahead of time, to friends as well as to pollsters. Trump voters hardly seemed shy in the run-up to Election Day. Both

the media and our own collective circles of friends bear this out--but with some interesting nuances.

California is about as "blue" as a U.S. state can be. Even so, all of my friends have extended family across those bright-red "flyover" states. More than a few of us were reading the tea leaves at least as effectively as our best and brightest surveyors. It was common to hear stories of families no longer speaking to each other, a lone Hillary Clinton enthusiast keeping his silence in Missouri or a Trump fan in Oakland, Calif.--yes, I know one--who felt misunderstood by his friends. The clues are showing up everywhere in hindsight. Clinton also had the benefit of then President Barack Obama's star election experts embedded in her campaign, which gave further assurance to many analysts on the strength of her campaign. So why did the data miss the depth of voter sentiment, particularly in swing states?

Here's my guess: Unstructured data is a whole lot harder to analyze than anyone anticipated. Parsing tons of sales and market data in search of the perfect product is terrific fun, and this data can predict choices such as purchasing one product over another. But elections can hinge on fickle, emotional states of mind. This key dynamic was (apparently) missed in all of the data analysis that occurred during the election cycle. To be fair, perhaps we just do not know how to extract such insights from data. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.