December 4, 2012
By now you are likely aware that Nate Silver of the New York Times correctly predicted the results for all 50 states (plus DC) in this year’s presidential election and all but two Senate races. Silver’s predictive capabilities across a range of disciplines have made him a near-deity among those whose livelihood depends on accurate forecasting – from poker players to counter-terrorism units. It’s clear why: His methods work – at least in some cases. And their strengths and limitations carry important lessons for financial advisors.
Already a lightning rod for controversy throughout this year’s election cycle, Silver has been making the rounds to publicly examine his methods – and raise questions about them going forward. His recent high profile owes in part to his election-day successes but also the corresponding increase in sales of his book: “The Signal and The Noise: Why So Many Predictions Fail – But Some Don’t.”
Published September 27th, the book surveys a number of recent failures and successes in forecasting across an array of fields: politics, the economy, baseball, and the weather, to name a few. Exploring statistical theory and its many applications in plain English, the book effectively renders its complex subject matter accessible to a broad audience.
The book’s core message is inclusive as well. Rather than simply presenting a new way of thinking, Silver implores his readers to think for themselves: challenge their assumptions, recognize opposing viewpoints, and reevaluate their beliefs as circumstances change.
Let’s look first at Silver’s overarching approach to data analysis and statistical predictions. We’ll then turn to how his methods apply to a decision that financial advisors face on a regular basis: selecting an actively-managed mutual fund that is likely to outperform its relevant benchmark.
Nate Silver: Lord of the Algorithm
Neatly encapsulating the accolades that Silver has received, Jon Stewart bestowed upon him the title of “Lord of the Algorithm” when Silver appeared on Stewart’s show the night after the election. But Silver’s book contains more philosophy than mathematics. His thinking is grounded in Bayesian statistics, which emphasize the importance of prior beliefs when weighing evidence for or against a hypothesis. He primarily blames predictive failings on our culture’s emphasis on certainty and widespread unwillingness to be honest about our biases.
Though he has gained mainstream recognition as a political prognosticator, Silver’s philosophy first took root in very different soil: the study of baseball. He first glimpsed the power of statistical analysis while developing PECOTA, an algorithmic system for predicting player performance. By recognizing his own dismissive attitude towards scouts and their subjective evaluations of players’ skills, Silver was able to create a more accurate statistical model that accounted for such personal analysis.
Silver’s political prediction blog, fivethirtyeight.com, was born from the ashes of his fascination with poker, which ended abruptly in 2006 when the 109th congress enacted laws that crippled the then-burgeoning online poker industry. Feeling lucky to have gotten out of the game when he did, Silver in his book examines his failings as a gambler and admits to being ignorant of them while he was still playing. Self-evaluation, which Silver emphasizes as necessary to prevent overconfidence, is central to his predictive philosophy.
Would you like to send this article to a friend?Remember, if you have a question or comment, send it to .