Nate Silver’s punditry revolution

The new methods for predicting the outcome of the US elections are changing the face of media analysis.

By CAMERON S. BROWN
Romney poll graphic (photo credit: REUTERS/REUTERS GRAPHICS)
Romney poll graphic
(photo credit: REUTERS/REUTERS GRAPHICS)
Whether or not US President Barack Obama is re-elected will not just determine the future of politics, it is also likely to influence the future of political punditry as well.
The media has been following the US elections with two very, very different narratives. Apart from the obvious – i.e. Democrat versus Republican—opinions were divided between those who thought the race was way too close to call and those who predicted with great confidence that Obama was far more likely to win.
Most of the media—including the most respected outlets—continued with the traditional focus on the national polls. Even when state-level polls were cited, analyses often cherry-picked the more interesting polls or ones that suited the specific bias of the pundit. While on occasion this analysis has given one candidate or another an advantage of several points, more often than not they declared the race “neck-and-neck,” and well within any given poll’s margin of error.
The other narrative was born out of a new, far more sophisticated political analysis than has ever been seen in the mass media before. Nate Silver, a young statistician who was part of the “Moneyball” revolution in professional baseball, has brought that same methodological rigor to political analysis. The result is that Silver is leading a second revolution: this time in the world of political punditry.
In his blog, “Five Thirty-Eight,” (picked up this year by the New York Times), Silver put his first emphasis on the fact that whoever wins the national vote does not actually determine who wins the presidency. Instead, the presidency is decided by the anachronistic Electoral College system, where each state votes for electors and they in turn choose the president. As a result, Silver rightfully argues, what matters most is the state of play in each individual state, and to total those numbers.
Silver increased the complexity to his mathematical model by mixing state and national polls, and by giving each particular poll a different statistical weight (i.e. changing the amount it impacted results) based on its methodology and bias. Finally, the model was dynamic: it began the election season by giving some weight to economic factors, but towards the end focused almost solely on the opinion polling.
On a daily basis his blog calculated these factors and spat out a prediction about where the candidates stood and the level of uncertainty therein. What has concerned Republicans and cheered Democrats is that Silver’s calculus, through good times and bad, consistently predicted a substantially bigger advantage for Obama than what the national polls suggested. On November 6, his model predicted a 92 percent chance of an Obama victory (315 electors to 222 for Romney, plus or minus 47). A totally different picture than the 49 percent to 48 percent Romney win predicted, for example, by Gallup’s poll.
(Silver is not the only pundit to have developed such intricate models. Princeton University’s Sam Wang, for example, also has developed a similar sort of model, though the two do differ slightly on what is the ideal recipe for the best predictive model. Wang’s results were actually even more highly stacked in Obama’s favor than Silver’s.)
Not everyone is pleased with Silver. Many of his detractors have been traditional pundits like Politico’s Dylan Byers and The New York Times’ own David Brooks, though some pushback has also come from statistically-minded people, like The New Yorker’s John Cassidy and Columbia University’s professor of statistics, Andrew Gelman.
Post-election, people are bound to wonder whether punditry benefits from a more quantitative approach or whether analysis essentially based on large amounts of anecdotal evidence better captures reality. Perhaps even more critically, a question arises as to which approach is more effective in the dark arts of predicting the future. The “answer,” unfortunately, is likely to be decided by how close the election results match Silver’s predictions (and no one will look at that “plus or minus 47” electors, just his 315 vs. 222).
Should Silver’s predictions prove accurate, the demand for more sophisticated, quantitative-based punditry is likely to increase substantially, perhaps even spilling into other areas of political analysis. Should the “Romney-by-a-nose” analysis better match election results, it will be very difficult to present a quantitative model in public on any subject without drawing references about being “a Silver.” As a result, the insights these models produce could remain locked up in the ivory tower, shared only among a small pool of political scientists and economists.
Having avoided math during my undergraduate and masters’ degrees, it was not until halfway through my PhD that I finally came to understand what statistical analysis can give us that relying solely on anecdotal evidence cannot.
The truth is that some areas of politics are absolutely ripe for this sort of analysis. Elections in Western democracies are now data-rich environments, giving sophisticated statisticians plenty of great material to work with. Even in less-than-ideal environments, such as understanding the dynamics of war and peace, statistics are increasingly becoming a useful tool for analyzing events.
However, the fear is that partisans on both sides of the fence are likely to overlook the fact that rigorous quantitative analysis—no less than traditional punditry—is ultimately based on theories about how the world works. And good theories, more often than not, are born of precisely the same intuition that produces “gut feelings.” Silver’s “scientific” model has to decide what to include and what to exclude, and how much importance to assign each factor— i.e. it is at least as much art as it is science. By the same token, the brilliance of a solid historical analogy is one that contains the same key elements as the situation under discussion.
Regardless of method, the most informative pundits will remain those who can convincingly demonstrate that their intuitions best capture the essential motivations of human behavior.
The writer is Neubauer Research Fellow at the Institute for National Security Studies (INSS), Tel Aviv University.