Data Skeptics is a popular monthly Meetup in New York City that covers the mathematical, ethical, and business aspects of data from a skeptical perspective and hosted by Steven Mornelli.
In this talk given Feb 5th, Claudia Perlich, Chief Data Scientist, whom designs, develops, analyzes and optimizes the machine learning that drives digital advertising at Distillary speaks about the deeply symbiotic relationship between predictive modeling and Big Data.
“All The Data and Still Not Enough” By Claudia Perlich
Machine learning theory asserts that the more data the better. Empirical observations suggest that more granular data, a hallmark of Big Data, further improves performance. On the other hand, predictive modeling is also one of the core techniques that measurably delivers value across many industries and demonstrates the value of Big Data and justifies its investments. Models have become an integral, and when successful invisible part of our life whether through personalization in retail and entertainment, targeted marketing and effective CRM, our city’s efforts to improve their citizen’s safety and emergency response, and even supply management for citibikes.
However, there is a surprising paradox of predictive modeling: when you need models most, even all the data is not enough or just not suitable. The foundation of predictive modeling requires that you have enough training data with the respective outcomes: But there are only so many people buying luxury cars online to inform my targeting models. I can never observe what happens BOTH when I treat you AND when I don’t – which is what I need to make causal claims and measure the impact of strategic decisions. To allocate sales resources I love to know what a customer’s budget is – but maybe even he does not know. I need to predict who is most likely going to respond to an ad BEFORE I have ever shown one.
So in the days and age of Big Data there remains an art to predictive modeling in situation where the right data is scarce. This talk will present a number of cases where enough of the right data is fundamentally not obtainable. In those instances we discuss some of the tricks of the trade including transfer learning and quantile estimation.