The Statisticians on the Bus

Article excerpt

Byline: Andrew Romano

How a nerd named Nate Silver changed political reporting forever.

There was a lot at stake on Nov. 6. The shape of the economy. The contours of the tax code. The survival of Obamacare. And oh, yeah: the reputation of some nerd named Nate Silver, too.

One of the strangest things about the final days of the 2012 presidential campaign was that the battle between Barack Obama and Mitt Romney became something of a sideshow, at least in the greenrooms of Manhattan and Washington, D.C., and on the Acela trains that shuttled the creme de la press corps between them. The hot topic wasn't the election itself. It was who would best predict the election's results.

The fight was familiar to anyone who lived through baseball's Moneyball revolution. In one corner were the "quants," such as Silver, a sabermetrician turned New York Times blogger who used a proprietary statistical model to average and weigh public polls and produce probabilistic forecasts of which candidate was more likely to win on Election Day. In the other corner were the sort of pundits paid to pontificate about who they think will win on Election Day: the hosts, columnists, retired politicians, and former operatives who prefer to make predictions from the "gut."

For months, Silver's model had shown Obama as the odds-on favorite to recapture the White House. But then a funny thing happened: as Election Day approached and Obama's swing-state advantage solidified, his chances of winning started to rise into the 70s, 80s, and even 90s--and the Gut Brigade began to get testy.

Suddenly, Joe Scarborough was bashing Silver on MSNBC. "Anybody [who] thinks that this race is anything but a toss-up right now is such an ideologue," Scarborough scoffed. "They're jokes." Meanwhile, New York Times columnist David Brooks went so far as to characterize Silver and his imitators as delusional "wizard[s]" and citizens of "silly land."

The fight was fun while it lasted. The future is unverifiable, at least at first, so everyone is free to predict whatever outcome they like. But the future always arrives eventually, as it did on Nov. 6, and someone is proven right. In this case, that someone was Silver. When the dust settled, his model had called 49 states correctly (Florida had yet to be officially declared at press time) and prophesied the popular vote to within a half a percentage point. Not one traditional pundit had come close.

And so the Moneyball Election was settled. The quants won. The question now is whether the rest of the punditocracy will learn anything from its loss--or just keep chattering away, oblivious.

Much of what currently passes for campaign coverage is deliberately unenlightening. On TV and at websites like Politico, commentators incessantly hype individual polls and bluster about their "sense" of "where things stand," creating the illusion that the candidates are trading the lead and that the contest is a so-called toss-up. Sure, the Everything Is Breaking News All the Time business model keeps readers and viewers entertained. But it also makes them cranky, suspicious, and misinformed.

So perhaps it's time to rethink how we cover these contests. After all, innumerate forecasting isn't our only problem. Advertising revenue is scarce. Embedding on a campaign plane is expensive. Blanketing a national convention is even costlier. Neither endeavor is particularly rewarding, newswise; the candidates themselves have never been less accessible or spontaneous. And the media's reputation isn't getting any better.

Which is where the whole Silver skirmish comes in. My guess is that the journalists who will stand out in 2016 will be the ones who stick to a simple plan: if you can't beat 'em, join 'em. Most political hacks aren't equipped to become quants. But the best of them will become more quantlike. Grapple with the data. Absorb the political science. …