Nate Silver, a statistician who earned celebrity status after accurately predicting the results of the 2012 presidential election, stopped by theCUBE at this week’s Tableau Customer Conference to share his unique insights into Big Data analytics.
Silver says that individuals often misinterpret data in an effort to validate their personal views or apply it to an issue that they are trying to solve. He admits that he himself has been accused of bias, but notes that the accuracy of his predictions proved skeptics of his objectivity wrong.
Asked to share his perspective on prediction markets, speculative markets that revolve around probabilities such as a presidential election, Silver says that they often don’t attract enough liquidity to justify an investment. He adds that on a macrolevel, the result of an election can generate hundreds of billions of dollars in stock gains, or send shares spiraling down just as easily.
Markets are far from flawless, Silver stresses, and they start to collapse once the notion that they adhere to a standard of perfection gets propagated.
Some Big Data leaders argue that correlations between datasets take precedence over causality, making the latter redundant. Silver believes that both are equally important for data-driven enterprises, as is having a goal-oriented organizational culture. He explains that the value of Big Data cannot be unlocked without analysts who can separate the signal from the noise and decision makers who keep a finger on the pulse of business.
“A lot of technologists look at economic models where you have different factors of productivity. It’s not usually an adding relationship, it’s more of a multiplying relationship,” Silver says. “So if you have a lot of data that people are not good at analyzing, if you have a lot of data but it’s unstructured and unscrutinized, you’re not going to get particularly good results from it.”
Silver wraps up the interview by drilling down into social media analytics. In his view, this space is moving far too rapidly to bear fruit: by the time data scientists come up with a new model for tracking consumer sentiment, the underlying patterns will have changed.
—
Nate Silver, FiveThirtyEight, ESPN, at Tableau Customer Conference 2013 with Dave Vellante and Jeff Kelly