The famous words of Edward Deming, the father of the Quality movement of the mid-eighties. Deming was referring to the use of data to improve manufacturing quality. Data was paramount to the process of improving and maintaining quality in the Zero defects quality movement.
Today, Deming's principles are no longer needed in manufacturing. The quality principles he espoused are a given, very much like all those things in your car that are now standard.
However, Deming rings true today than ever before in this era of ever expanding big data. Big data is a revolution that is moving at evolutionary not revolutionary speeds. 2011 was supposed to be the year of Big Data, we are fast approaching 2016. What are the impediments to the big data explosion that is such a long time coming?
1. Big data was already here when it arrived. Companies like American Express were doing what people do today with Big Data a very long time before the term was coined. Granted, the data was structured data in the form of credit card transactions. However, the company was a pioneer in the application of predictive analytics, using the data to predict what consumers will want, what purchase motivations may be triggered and then tailoring offerings from themselves and their partners accordingly.
2. Big data is NOT looking for a needle in a haystack. The haystack is the needle. Big data is about looking at the hay to see if you can make sense of it. This is the big challenge of big data is that you are looking for something when you don't know what that something is. In statistics, the word a priori signifies that there is a hypothesis being tested, that the data analysis will prove or disprove. Not so with most Big Data assignments; the task is typically to look for a new pattern may not be apparent till it is found. In our market research work, we often refer to "peripheral vision"-looking for tangential insights that are new in addition to the subject being investigated or the hypothesis being tested, In Big Data tasks, the peripheral vision observation is not tangential to the main task, it is the main task. I should also mention here that in the area of market segmentation, there are many examples that tie in well together with Big Data assignments. Posthoc segmentation is one such example where, unlike apriori, segmentation, you simply gather a lot of data about your customers or prospects and then use that data to create a basis for segmentation. The classic marketing example here is Procter & Gamble's introduction of Crest Toothpaste. The benefit of good dental checkups (no cavities) was uncovered almost by accident and was not the traditional benefits sought by which toothpaste brands segmented themselves (fresh breath, etc.) The Crest brand was created in response to an unserved and sizable enough segment of those who sought the benefit of good dental checkups.
3. It’s not just the haystack, it’s the entire barn. The mentality is that this is where we will search. All the data, including the garbage or "data exhaust" as it is called. Which also brings to mind, a classic market research example, the first ever market research project in 1873, Campbell's Soup was introduced to the Boston Area, the two major daily newspapers of the day the Boston Globe and the Boston Herald were vying for the lucrative advertising account. There were two opposing hypothesis-one that because the idea of soup in a can was novel, it would appeal to upper class readers of the conservative Herald. The other was that because of its practicality, it would appeal to time pressed working class families as an alternative to long evening meal preparations. The research project consisting of combing through garbage literally in both upper class and working class neighborhoods to collect and count empty Campbell's soup cans. Who won? The Herald hands down. Upper class households embraced the novelty. Working class families, mostly recent European migrants shunned the innovation as an affront to their traditions of preparing home cooked meals. Big Data has indeed been around a long time!
4. It’s more inspector Clouseau than Sherlock Holmes. Not much more to be said about this one but the task is not one that can be logically organized with inductive reasoning and deductions as Holmes would approach his sleuthing. If you have seen any of the Pink Panther movies, Clouseau bumbles through most of the two hour movie, missing clues and creating his own brand of mayhem in the process but eventually solves the crime. And so it is with Big Data, a fact that challenges and frustrates many Business Intelligence departments.
Alan Nazarelli is President and CEO of Silicon Valley Research Group, a global market research and strategy development firm focused on the needs of technology companies.
Do you have questions, or would you like to hear from Silicon Valley Research Group on a specific topic in a future blog publication? Contact us.
If you'd like to start a dialogue about issues you want to address with market research, click the Find Out More button. We will provide you with a free quote on research solutions tailored to your needs.
Topics: Big data
