14 rules for data-driven, not data-deluded, marketing

We’re now in the crossfire of a peaking hype cycle for big data and its inevitable backlash. One marketer the other night was yelling on Twitter, “Big Data People need to GET OFF MY LAWN.”

I confess, I empathize with his frustration. Of course, I believe that data — the right data, used in the right ways — is immensely powerful in modern marketing. As I wrote in a post about pragmatic marketing, “Relying solely on gut-based, experience-driven decision-making in marketing is foolish in the digital age.”

But I also concluded: “The sensible answer for most companies is a balance of data analytics and human judgement.”

My favorite big picture marketing columnist, Gord Hotchkiss, recently penned a piece on Marketing in the Middle, advocating for such a balanced approach in many of the debated dichotomies in marketing:

There are no absolutes in marketing; there is just a lot of mushiness in the middle. We need hedgehogs for the “big ideas” that make great marketing great. But we also need foxes to help us navigate through the middle successfully. In fact, the more time I’ve spent in marketing (trying assiduously to avoid becoming an “expert”), the more I’ve realized that the middle is where all the action is: between quantitative and qualitative, between strategy and big data, between creative branding and direct marketing, between science and art.

(Here’s a quick background on hedgehogs and foxes, if you aren’t already familiar with the metaphor. In short: The fox knows many things. The hedgehog knows one big thing.)

So what are the many foxy things you should know about using data well?

In the spirit of a balanced approach, I heartily encourage you to embrace data in marketing — but here are 14 rules of thumb to keep that data in perspective, to support a pragmatic approach to data-driven marketing (and avoid data-drowned or data-deluded marketing):

#1. Customer-driven > data-driven.

Data-driven marketing is good. But the purpose of marketing is to win customers (and keep them by making sure they have a great experience) — therefore, customer-driven marketing is better. They don’t have to be mutually exclusive, of course. But one is the means and the other is the end. It helps to remember that before chasing a glittering piece of data down a rabbit hole. Stop and ask, “How is this good for customers?”

#2. All data is not created equal.

“Data ends arguments.” It would be nice if that were an absolute truth. But with so much data swirling around these days, it’s absurdly easy to find data to support nearly any side of an argument. But data differs in its accuracy and its relevance. Consider this tale of how in a big data world, marketers know shockingly little about us. A better motto may be: the most accurate and relevant data ends arguments. But in truth, it’s decisions that end arguments.

#3. Data is only history.

Data tells us what happened. It does not tell us what is going to happen — despite all the oversimplified hype around predictive analytics to the contrary. Yes, we should learn from history. Yes, we can extrapolate trends from the past to make predictions about the future — possibly even very accurate predictions. But the world is constantly changing, and the circumstances of history are not necessarily the circumstances of today or tomorrow. Yes, there are black swans, Virginia. The way it was is not always the way it will be.

#4. Data is always incomplete.

Sure, any one specific set of data can be complete. Give me quarterly sales figures for the past three years: that’s a complete data set. But nothing exists in a vacuum. When you’re using data to make decisions — and isn’t that the whole point? — the data you have is not the only data that could be relevant to your decision. (“We go to market with the data we have, not the data want.”) There’s always more data out there. And you can never have all of it. Acknowledging this helps us realize that although the data we have may paint a picture, it’s an Impressionist painting at best. Often, it’s more like modern art.

#5. Data is objective, but its collection and interpretation are subjective.

Data gives a dangerous illusion of objectivity. Objectively, everyone can look at the same data in a spreadsheet. If I show you a Net Promoter score of 7, there’s no argument that the number I’m showing you is indeed a “7.” But what data was collected, when and how it was collected, and from whom are almost all subjective choices. And on the other side, how we choose to interpret that data is subjective too. This article on the hidden biases in big data gives several good examples. It also introduces the brilliant phrase “data fundamentalism.”

#6. Any one set of data supports an infinite number of narratives.

Marketers are storytellers, and that’s generally a good thing. Data can be used to make stories more compelling. But, since we can subjectively choose how to interpret data, we can invent almost any narrative we want around it. Granted, some stories are more credible than others. But that’s often a fuzzy line. This is known as the problem of induction, and it’s vexed scientists and philosophers for centuries. Needless to say, there’s no easy answer. But it helps to keep perspective: any story presented around data is never the only story that could be told about it.

#7. Strategy is choice. Good data helps us make those choices.

I recently expounded on the difference between strategic data vs. data theater, so I won’t rehash it here. But a good strategy is a framework for making choices. In turn, good data is information that helps us make those choices. Data that isn’t helping us make choices in pursuit of our strategy is a distraction.

#8. Experimentation is the gold standard of causation.

Correlation is not causation. Every data scientist worth their salt will tell you this. But as marketers, it’s usually causation that we’re after — we want to know what we can do that will cause more customers to do more business with us. So what do we do when data shows a correlation that may reveal such a cause? We run a controlled experiment. Keep all other variables constant (as much as is practically feasible) and test the alternatives to prove or disprove our hypothesis. Google runs over 10,000 such experiments every year. It’s the most powerful data you can generate, which is why big testing will be bigger than big data.

#9. Watch the dashboard but also look out the windshield.

One of my favorite quotes from Gord is this great metaphor: “Quantitative is watching the dashboard while you drive. Qualitative is looking out the windshield.” Obviously, we should do both. And in practice, we balance the quantitative (data) and the qualitative (experience) naturally when we’re driving — well, ahem, at least most of us do. We should strive for that same equilibrium in marketing too. Don’t miss the inspiring scenery, or the oncoming truck barreling towards you, because you’re myopically glued to your dashboard.

#10. The accuracy and relevance of data decay with time (and often quite quickly).

Most data has a relatively short shelf life, particularly in marketing. By all means, target car ads at me during the week when I’m searching for a new car — but six months later, that data that identified me as in the market for a car is just a fossil of the past. There may be some value in knowing that I was in the market for a car, but believing I still am is worthless. Worse than worthless, because it may lead you to make bad decisions about me. Accuracy and relevance are what make data valuable, but those attributes change with time.

#11. Data can be used for exploration (“why?”) or confirmation (“what?”).

Data should be treated differently when it’s being used for exploration versus confirmation. With exploration, you’re seeking patterns, insights, ideas, discoveries — inspiration for new hypotheses. With confirmation, you’re verifying that something happened (or not) and to what extent. However, unless you were confirming a hypothesis tested with a controlled experiment, confirmation tells you only what happened, not why. The same data may be used for confirmation in one context and exploration in another. Know which you’re doing.

#12. A man with a watch knows what time it is. A man with two watches is never sure.

This rule is known as Segal’s Law, which is one of the 7 laws of technology every marketer should know. The world is full of conflicting data. Anyone who has ever tried to get metrics from two separate web analytics packages to line up can attest to this truth. Different tools will measure the same phenomenon differently. It may be worth investigating significant variances — understanding why they’re different may lead to valuable insights — but there’s a diminishing return to chasing every minute discrepancy. In many cases, you don’t need perfectly accurate data, but simply sufficiently accurate data to make good decisions.

#13. The model is not reality.

Data is not the reality that it claims to represent. At best, it is a reflection of reality, but one that is susceptible to being warped (see all the rules above). The philosopher and scientist Alfred Korzybski famously said, “The map is not the territory.” Certainly we want to use data — and maps, for that matter. As the great statistician George E. P. Box said, “All models are wrong, but some are useful.” But it’s prudent to maintain a little healthy skepticism about the correctness of the representation. In particular, we want to be alert to other signs — outside of the data — that suggest that reality differs. Follow the Swiss Army Aphorism: If the map and the terrain disagree, trust the terrain. (“The map shows no c-l-l-i-i-i-f-f-f-f…”)

#14. Data visualization can illuminate, obfuscate, or distract.

Data visualization — graphs, charts, infographics, etc. — is a powerful but two-edged sword. By far, it is the most effective way for us humans to see the patterns in data. Unfortunately, either intentionally or accidentally, it can show us patterns that aren’t actually true. Great data visualization is a science and an art unto itself. I’d highly recommend reading Stephen Few, Kaiser Fung, Edward Tufte, Nathan Yau, and Fernanda Viégas and Martin Wattenberg to hone your visual literacy skills. They will contribute some of the best tools to your data insight toolbox.

Bonus summary rule: Keep calm and use data wisely.

Data may be the most underutilized asset in marketing, but we want to utilize it well.